Call : (+91) 968636 4243
Mail : info@EncartaLabs.com
EncartaLabs

StreamSets Developer

( Duration: 2 Days )

This StreamSets Developer training course dives into solving use cases that span across data ingest (Data Collector) and data engineering (Transformer). The course also covers deeper topics such as advanced expression language, CI/CD, pipeline testing using the StreamSets SDK, complex pipeline development, custom processors, and best practices.

  • General knowledge of operating systems, networking, programming concepts, and databases.
  • Solid understanding of the objectives covered in the following fundamentals courses:
    • Control Hub Fundamentals
    • Data Collector Fundamentals
    • Transformer Fundamentals

The StreamSets Developer workshop is ideal for:

  • Data engineers, developers, data flow engineers, and systems architects who will be architecting and developing advanced pipelines.

COURSE AGENDA

1

Quick Review

  • Control Hub
  • Data Collector
  • Transformer
2

Expression Language

  • Constants
  • Literals
  • Variables & Fields
  • Functions & Operators
  • Runtime EL
  • Regular expressions
3

SDC Evaluator Processors

  • Record processing with scripts
  • Individual record inspection
  • Logging and error handling
4

Transformer Spark Processors & Evaluators

  • Spark SQL Query Processor
  • Spark SQL Expression Processor
  • PySpark Processor
5

End to End Pipelines

  • Data ingest & transformation use cases
  • Complex pipelines
  • Advanced parameterization
  • Connection Catalog
  • Orchestration pipelines
  • Topologies
  • Custom component development
  • Kafka Pipelines
  • CDC Pipelines
6

Pipeline Testing & Automation

  • The Pipeline Repository
  • Pipeline development lifecycle
  • Testing pipelines
  • Pipeline CI/CD
  • SCH REST APIs
7

Capstone Project

  • Develop an end-to-end solution given a complex use case
  • Optimize & tune solution

Encarta Labs Advantage

  • One Stop Corporate Training Solution Providers for over 6,000 various courses on a variety of subjects
  • All courses are delivered by Industry Veterans
  • Get jumpstarted from newbie to production ready in a matter of few days
  • Trained more than 50,000 Corporate executives across the Globe
  • All our trainings are conducted in workshop mode with more focus on hands-on sessions

View our other course offerings by visiting https://www.encartalabs.com/course-catalogue-all.php

Contact us for delivering this course as a public/open-house workshop/online training for a group of 10+ candidates.

Top
Notice
X