Call : (+91) 968636 4243
Mail :

Apache Spark

The Apache Spark - Essentials training course provides a solid technical introduction to the Spark architecture and how Spark works. You will learn the basic building blocks of Spark, including RDDs and the distributed compute engine, as well as higher-level constructs that provide a simpler and more capable interface, including Spark SQL and DataFrames. This course also covers more advanced capabilities such as the use of Spark Streaming to process streaming data, and provides an overview of Spark Graph Processing (GraphX and GraphFrames) and Spark Machine Learning (SparkML Pipelines). Finally, the class explores possible performance issues, troubleshooting, cluster deployment techniques, and strategies for optimization.

The Apache Spark - Advanced training course teaches the advanced Spark skills. You will discover how to integrate Spark with Cassandra, cluster data workflows, measure performance, and more.

By attending Apache Spark - Essentials workshop, delegates will learn to:

  • Understand the need for Spark in data processing
  • Understand the Spark architecture and how it distributes computations to cluster nodes
  • Be familiar with basic installation / setup / layout of Spark
  • Use the Spark for interactive and ad-hoc operations
  • Use Dataset/DataFrame/Spark SQL to efficiently process structured data
  • Understand basics of RDDs (Resilient Distributed Datasets), and data partitioning, pipelining, and computations
  • Understand Spark's data caching and its usage
  • Understand performance implications and optimizations when using Spark
  • Be familiar with Spark Graph Processing and SparkML machine learning

By attending Apache Spark - Advanced workshop, delegates will learn to:

  • Build on Spark fundamentals to gain a deeper understanding of Spark internals
  • Learn the operational tweaks to generate the maximum performance from Spark
  • Discover how to use GraphX and MLib for machine learning

For Apache Spark - Essentials

  • Knowledge of Python or Scala.

For Apache Spark - Advanced

  • Developers who have knowledge of Spark Programming.


Apache Spark - Essentials
(Duration : 3 Days)


Scala Ramp Up

  • Scala Introduction, Variables, Data Types, Control Flow
  • The Scala Interpreter
  • Collections and their Standard Methods (e.g. map())
  • Functions, Methods, Function Literals
  • Class, Object, Trait

Introduction to Spark

  • Overview, Motivations, Spark Systems
  • Spark Ecosystem
  • Spark vs. Hadoop
  • Typical Spark Deployment and Usage Environments

RDDs and Spark Architecture

  • RDD Concepts, Partitions, Lifecycle, Lazy Evaluation
  • Working with RDDs - Creating and Transforming (map, filter, etc.)
  • Caching - Concepts, Storage Type, Guidelines

DataSets/DataFrames and Spark SQL

  • Introduction and Usage
  • Creating and Using a DataSet
  • Working with JSON
  • Using the DataSet DSL
  • Using SQL with Spark
  • Data Formats
  • Optimizations: Catalyst and Tungsten
  • DataSets vs. DataFrames vs. RDDs

Creating Spark Applications

  • Overview, Basic Driver Code, SparkConf
  • Creating and Using a SparkContext/SparkSession
  • Building and Running Applications
  • Application Lifecycle
  • Cluster Managers
  • Logging and Debugging

Spark Streaming

  • Overview and Streaming Basics
  • Structured Streaming
  • DStreams (Discretized Steams),
  • Architecture, Stateless, Stateful, and Windowed Transformations
  • Spark Streaming API
  • Programming and Transformations

Performance Characteristics and Tuning

  • The Spark UI
  • Narrow vs. Wide Dependencies
  • Minimizing Data Processing and Shuffling
  • Caching - Concepts, Storage Type, Guidelines
  • Using Caching
  • Using Broadcast Variables and Accumulators

Spark GraphX Overview

  • Introduction
  • Constructing Simple Graphs
  • GraphX API
  • Shortest Path Example

MLLib Overview

  • Introduction
  • Feature Vectors
  • Clustering / Grouping, K-Means
  • Recommendations
  • Classifications
Apache Spark - Advanced
(Duration : 3 Days)




Spark integration with Cassandra (other compatible NoSQL implementations can be substituted if supported)


Advanced Spark SQL and Spark Streaming


Implementing Spark on DataStax and Hortonworks


Cluster resource requirements


Debugging/troubleshooting Spark apps


Developing data workflows


Performance metrics


Cases studies

Encarta Labs Advantage

  • One Stop Corporate Training Solution Providers for over 6,000 various courses on a variety of subjects
  • All courses are delivered by Industry Veterans
  • Get jumpstarted from newbie to production ready in a matter of few days
  • Trained more than 50,000 Corporate executives across the Globe
  • All our trainings are conducted in workshop mode with more focus on hands-on sessions

View our other course offerings by visiting

Contact us for delivering this course as a public/open-house workshop/online training for a group of 10+ candidates.