SPARK

Structured Streaming basics

So far you've processed data in batches, which was a finite dataset in, a result out. Data keeps arriving forever, and your job keeps processing it forever. Spark's Structured Streaming lets you write streaming code that looks almost exactly like batch code, using the same DataFrame API. Learn the model, understand triggers and output modes, and run a real streaming query.

ontent


What's next

Now go and try this out in a live environment — boot a fresh cluster and play with the manifests above.

Start Spark
Spec 2 CPU / 4 GiB ·Disk 25 GiB
Sign in to launch this environment
Required 1 VM · 2 CPU · 4 GB · 25 GiB disk
Available 1 VM · 1 CPU · 2 GB · 10 GiB disk
Sign in