Serverless Data Processing with Dataflow: Operations
In the last installment of the Dataflow course series, we will introduce the components of the Dataflow operational model. We will examine tools and techniques for troubleshooting and optimizing pipeline performance. We will then review testing, deployment, and reliability best practices for Dataflow pipelines. We will conclude with a review of Templates, which makes it easy to scale Dataflow pipelines to organizations with hundreds of users. These lessons will help ensure that your data platform is stable and resilient to unanticipated circumstances.
What you'll learn
In the last installment of the Dataflow course series, we will introduce the components of the Dataflow operational model. We will examine tools and techniques for troubleshooting and optimizing pipeline performance. We will then review testing, deployment, and reliability best practices for Dataflow pipelines. We will conclude with a review of Templates, which makes it easy to scale Dataflow pipelines to organizations with hundreds of users. These lessons will help ensure that your data platform is stable and resilient to unanticipated circumstances.
Table of contents
- Testing and CI/CD Overview 5m
- Unit Testing 6m
- Integration Testing 2m
- Artifact Building 1m
- Deployment 13m
- Lab: Serverless Data Processing with Dataflow - Testing with Apache Beam (Java) 0m
- Lab: Serverless Data Processing with Dataflow - Testing with Apache Beam (Python) 0m
- Lab: Serverless Data Processing with Dataflow - CI/CD with Dataflow 0m
- Additional Resources 0m
