Real-Time Stream Processing Using Apache Spark 3 for Scala Developers

Preview this course

Learn the process to design and develop big data engineering projects using Apache Spark. This example-driven advanced-level course will help you understand real-time stream processing using Apache Spark and you can apply that knowledge to build real-time stream processing solutions.

Unlimited access to 750+ courses.
Enjoy a Free Trial. Cancel Anytime.

- OR -

30-Day Money-Back Guarantee
Full Lifetime Access.
37 on-demand videos & exercises
Level: All Levels
3hrs 23mins
Access on mobile, web and TV

What to know about this course

Since its inception, Apache Spark has seen rapid adoption by enterprises across a wide range of industries. So, mastering Apache Spark opens a wide range of professional opportunities. If you are a software engineer or architect and want to design or build your own projects, then this is the right course for you.

This is a hands-on, example-driven, advanced course with demonstrations and coding sessions. This course will help you understand real-time stream processing using Apache Spark and later, you will be able to apply that knowledge to build real-time stream processing solutions. This course covers everything from scratch, which involves installing Apache Spark and seeing how to set up and run Apache Kafka. Furthermore, it introduces stream processing and how to work with files and directories. You will also explore Kafka serialization and deserialization for Spark and how to work with Kafka AVRO Source. And finally, the course wraps up with streaming Watermark and outer joints.

By the end of this course, you will be able to design and develop big data engineering projects. You will be able to create real-time stream processing applications with Apache Spark. This course will also help you further your growth in real-time stream processing. All resources and code files are placed here:

Who's this course for?

This course is designed for software engineers and architects who aspire to develop big data engineering projects using Apache Spark.
Also, if you are a programmer and developer who wants to grow and learn data engineering using Apache Spark, then this course is for you.

Another group of people that can opt for this course are the managers and architects who might not directly work with Spark implementation but still work with the people who implement Apache Spark at the ground level.

What you'll learn

  • Create arbitrary streaming sinks
  • Explore Kafka Source and integrate Spark with Kafka
  • Learn state-less and state-full streaming transformations
  • Learn to handle memory problems with streaming joins
  • Learn to work with file streams
  • Explore windowing aggregates using Spark Stream

Key Features

  • Deep dive into Spark structured streaming APIs and architecture
  • Discover streaming joins and aggregation
  • Explore real-time stream processing concepts

Course Curriculum

About the Author


ScholarNest is a small team of people passionate about helping others learn and grow in their careers by bridging the gap between their existing and required skills. Together, they have over 40+ years of experience in IT as a developer, architect, consultant, trainer, and mentor. They have worked with international software services organizations on various data-centric and Big Data projects.
It is a team of firm believers in lifelong continuous learning and skill development. To popularize the importance of continuous learning, they started publishing free training videos on their YouTube channel. They conceptualized the notion of continuous learning, creating a journal of our learning under the Learning Journal banner.

40% OFF! Unlimited Access to 750+ Courses. Redeem Now.