Data Engineering - Advanced

Spark Structured Streaming

Design fault-tolerant streaming pipelines with event time, windows, watermarks, and checkpoints.

Instructor: OSEKOO ACADEMY - 3 units

This module can be taken independently, in any order.

Course overview

Build on Spark fundamentals to process unbounded data safely. Handle late events, state, recovery, observability, and production deployment constraints.

Standalone unit guides

Every unit is designed to be followed independently: review the guide, complete the lessons in order, finish the practical lab, pass the checkpoint, and deliver the unit project.

1. Streaming Fundamentals3 lessons

Lesson sequence

  1. Batch versus stream processing - Free preview
  2. Sources, sinks, and triggers
  3. Event time and processing time
2. Stateful Processing3 lessons

Lesson sequence

  1. Windows and aggregations
  2. Watermarks and late data
  3. State management and deduplication
3. Production Streaming3 lessons

Lesson sequence

  1. Checkpointing and recovery
  2. Monitoring streaming queries
  3. Capstone: real-time metrics pipeline
Verified learner reviews

Learner feedback

No published reviews yet. Enrolled learners can submit the first review.