From Tables to Streams: Apache Flink for SQL Developers
A presentation at Amsterdam JUG in in Amsterdam, Netherlands by Viktor Gamov
From Tables to Streams Apache Flink for SQL Developers Viktor Gamov · @gamussa · Amsterdam JUG X / Bluesky: @gamussa
Same SQL. Different semantics. X / Bluesky: @gamussa
Viktor Gamov · @gamussa I build streaming things (and the toy you just saw — selectstar.stream) Flink, Kafka, real-time data X / Bluesky: @gamussa
Everything from tonight Everything from tonight lives here ↓ speaking.gamov.io · selectstar.stream · sql.selectstar.stream scan → speaking.gamov.io X / Bluesky: @gamussa
Apache Flink The open-source engine for stateful stream processing — and batch. Distributed · event-time aware · exactly-once · driven by SQL. X / Bluesky: @gamussa
Sources → Flink → Sinks Kafka, databases, files in. Results out. Flink is the processing in between. X / Bluesky: @gamussa
Unbounded data · event-time · state · exactly-once · scale The hard parts of streaming — handled, so you write SQL. X / Bluesky: @gamussa
SQL — declared dead since 1974 (SEQUEL, IBM Research). X / Bluesky: @gamussa
Table = the harbor (data at rest) Stream = the open sea (data in motion, never done) You stop being the harbormaster. You become the captain. X / Bluesky: @gamussa
You don’t need a Kafka PhD. You need SELECT. X / Bluesky: @gamussa
SELECT / JOIN / GROUP BY — on water that never stops. X / Bluesky: @gamussa
⎯ ship’s log ⎯ GROUP BY, two ways SELECT card_id, COUNT(*) AS n FROM txns GROUP BY card_id; batch → card_3, 3 (one final row) stream → +I (card_3, 1) -U (card_3, 1) +U (card_3, 2) -U (card_3, 2) +U (card_3, 3) … never done X / Bluesky: @gamussa
X / Bluesky: @gamussa
Windows. Late data. Watermarks. X / Bluesky: @gamussa
A watermark says: “I’ll wait a reasonable while for stragglers — then I sail without you.” X / Bluesky: @gamussa
⎯ ship’s log ⎯ temporal join SELECT t.txn_id, t.amount, r.rate_to_eur, ROUND(t.amount * r.rate_to_eur, 2) AS amount_eur FROM txn_events AS t JOIN fx_rates FOR SYSTEM_TIME AS OF t.event_time AS r ON t.ccy = r.ccy; X / Bluesky: @gamussa
When SQL runs out of road · and real-time AI X / Bluesky: @gamussa
⎯ ship’s log ⎯ Table API table.groupBy($(“card_id”)).select($(“card_id”), $ (“amount”).sum()); // same semantics, programmatic control X / Bluesky: @gamussa
Scoring a 3am batch isn’t realtime. It’s a horoscope. X / Bluesky: @gamussa
Card in Amsterdam. 4 seconds later, Tokyo. X / Bluesky: @gamussa
Stream when freshness changes the decision. Batch when the answer can wait. X / Bluesky: @gamussa
Same syntax. Different semantics. Know both. X / Bluesky: @gamussa
Play now: selectstar.stream · sql.selectstar.stream Learn: Apache Flink docs · Flink SQL cookbook All of it: speaking.gamov.io X / Bluesky: @gamussa
as always, have a nice day
What if your SQL queries could process data as it happens instead of after it’s stored? Apache Flink speaks SQL fluently, but it’s not a database – think of it as a conductor orchestrating endless streams of data rather than a librarian managing stored records.
This session bridges the gap between traditional SQL and stream processing.
We’ll explore:
The Mental Model Shift – From “query the database” to “continuously process the stream” using intuitive analogies Flink SQL in Action: How familiar operations like SELECT, JOIN, and GROUP BY work on infinite data streams, plus temporal joins, time windows, and watermarks Table API: Programmatic control with declarative simplicity, bridging SQL and complete programming flexibility Flink AI: Real-time feature engineering and model inference on streaming data using SQL-like patterns Real-World Patterns: Fraud detection during transactions, live analytics dashboards, and event-driven architectures You’ll leave understanding not just how to use Flink SQL, but when and why it transforms traditional database skills into real-time data superpowers.
Perfect for: SQL developers and data engineers ready to make their queries travel through time.
Resources
The following resources were mentioned during the presentation or are useful additional information.
