Call : (+91) 968636 4243
Mail :

Serverless Data Processing with Dataflow

( Duration: 3 Days )

This Serverless Data Processing with Dataflow training course is intended for big data practitioners who want to further their understanding of Dataflow in order to advance their data processing applications. Beginning with foundations, this course explains how Apache Beam and Dataflow work together to meet our data processing needs without the risk of vendor lock-in. The section on developing pipelines covers how you convert your business logic into data processing applications that can run on Dataflow. This course culminates with a focus on operations, which reviews the most important lessons for operating a data application on Dataflow, including monitoring, troubleshooting, testing, and reliability.

By attending Serverless Data Processing with Dataflow workshop, delegates will learn to:

  • Demonstrate how Apache Beam and Dataflow work together to fulfill your organization’s data processing needs. Summarize the benefits of the Beam Portability Framework and enable it for your Dataflow pipelines.
  • Enable Shuffle and Streaming Engine, for batch and streaming pipelines respectively, for maximum performance. Enable Flexible Resource Scheduling for more cost-efficient performance.
  • Select the right combination of IAM permissions for your Dataflow job.
  • Implement best practices for a secure data processing environment.
  • Select and tune the I/O of your choice for your Dataflow pipeline.
  • Use schemas to simplify your Beam code and improve the performance of your pipeline.
  • Develop a Beam pipeline using SQL and DataFrames.
  • Perform monitoring, troubleshooting, testing and CI/CD on Dataflow pipelines.

  • Building Batch Data Pipelines
  • Building Resilient Streaming Analytics Systems

The Serverless Data Processing with Dataflow class is ideal for:

  • Data engineers
  • Data analysts and data scientists aspiring to develop data engineering skills


Day 1

  • Introduction
  • Beam Portability
  • Separating Compute and Storage with Dataflow
  • IAM, Quotas, and Permissions
  • Security
  • Beam Concepts Review

Day 2

  • Windows, Watermarks, Triggers
  • Sources and Sinks
  • Schemas
  • State and Timers
  • Best Practices
  • Dataflow SQL and DataFrames
  • Beam Notebooks

Day 3

  • Monitoring
  • Logging and Error Reporting
  • Troubleshooting and Debug
  • Performance
  • Testing and CI/CD
  • Reliability
  • Flex Templates

Encarta Labs Advantage

  • One Stop Corporate Training Solution Providers for over 6,000 various courses on a variety of subjects
  • All courses are delivered by Industry Veterans
  • Get jumpstarted from newbie to production ready in a matter of few days
  • Trained more than 50,000 Corporate executives across the Globe
  • All our trainings are conducted in workshop mode with more focus on hands-on sessions

View our other course offerings by visiting

Contact us for delivering this course as a public/open-house workshop/online training for a group of 10+ candidates.