The Missing Protocol: How MCP Bridges LLMs and Data Streams

Who’s here? ————————————————————————————————— • • • • How many of you are building AI agents right now? (raise your hand) Keep them up - how many are using MCP to connect those agents to external systems? How many of you have streaming data - Kafka, Flink, event-driven architectures? How many have tried to make an agent react to real-time events? (this is the talk for you) X/Bluesky: @gamussa SF JUG, April 2026 2 / 44

The Hype ————————————————————————————————— Everyone is talking about MCP. I have no idea why. (just kidding, I totally know why) Let’s look at the data… X/Bluesky: @gamussa SF JUG, April 2026 3 / 44

Google Trends: “Model Context Protocol” ————————————————————————————————— X/Bluesky: @gamussapened in NoSF JUG, April 2026 4 / 44

Who Am I? ————————————————————————————————— Viktor Gamov Principal Developer Advocate at Confluent, Java Champion, book author I’ve been streaming data since before streaming was cool. (and definitely before it was on the LinkedIn influencer curriculum) X/Bluesky: @gamussa SF JUG, April 2026 5 / 44

Slides and Resources ————————————————————————————————— Everything from this talk - slides, links, resources speaking.gamov.io X/Bluesky: @gamussa SF JUG, April 2026 6 / 44

Part 1: The Problem ————————————————————————————————— X/Bluesky: @gamussa SF JUG, April 2026 7 / 44

Your Agents Are Impressive. And Blind. ————————————————————————————————— Your agents can: • • • • Query databases (stale by the time you ask) Search the web (everyone else’s data) Read your docs (written last quarter) Call APIs (request/response, point-in-time) But they cannot see what’s happening in your systems right now. X/Bluesky: @gamussa SF JUG, April 2026 8 / 44

The Evolution (and the Gap) ————————————————————————————————— Every layer solves one problem. But the API is still stateless. Each call starts fresh. RAG gives context, agents give actions MCP gives a standard way to connect both. X/Bluesky: @gamussa SF JUG, April 2026 9 / 44

Why Streaming Data? ————————————————————————————————— Your business doesn’t run in batch. Events happen continuously. What’s streaming? • • • • Orders placed Sensors reporting Users clicking Payments processing Where does it live? • • • • This is the data your agents are missing. X/Bluesky: @gamussa SF JUG, April 2026 Apache Kafka (most common) Apache Flink (processing) Schema Registry (contracts) Topics, not tables 10 / 44

The Integration Nightmare ————————————————————————————————— Every AI tool vendor says: “Just use our SDK!” So you write custom code for: • • • • • Claude API OpenAI API Gemini API Llama via Ollama Mistral API • • • • • Your Your Your Your Your vector DB Kafka cluster Flink jobs monitoring internal APIs That’s N × M integrations. Does it make sense? No. It’s a nightmare. X/Bluesky: @gamussa SF JUG, April 2026 11 / 44

Remember USB-A? ————————————————————————————————— Have you ever tried to put a USB-A cable in correctly on the first try? I’ll give you a moment. You turn it one way. Doesn’t fit. You turn it the other way. Still doesn’t fit. You turn it back. Now it fits. Thank you, European Union. We finally have USB-C. MCP is USB-C for AI integrations. X/Bluesky: @gamussa SF JUG, April 2026 12 / 44

Part 2: Enter MCP ————————————————————————————————— X/Bluesky: @gamussa SF JUG, April 2026 13 / 44

What is MCP? ————————————————————————————————— Model Context Protocol - an open standard by Anthropic • • • • • A protocol, not a product JSON-RPC based (I know, I know… JSON-RPC in 2026… but hear me out) Client-Server architecture Language agnostic One protocol to connect LLMs to everything X/Bluesky: @gamussa SF JUG, April 2026 14 / 44

MCP Architecture ————————————————————————————————— X/Bluesky: @gamussa SF JUG, April 2026 15 / 44

The Three Primitives ————————————————————————————————— ▒▒▒▒ Tools ▒▒▒▒ Resources ▒▒▒▒ Prompts Functions the LLM can call Data the LLM can read Templates the LLM can use × query_topic() × send_message() × create_pipelin e() X/Bluesky: @gamussa × topic://orders × schema://avro/ user × metrics://lag SF JUG, April 2026 × debug_consumer × analyze_throug hput × explain_schema 16 / 44

Part 3: MCP Meets Data Streaming ————————————————————————————————— X/Bluesky: @gamussa SF JUG, April 2026 17 / 44

What If… ————————————————————————————————— What if your AI agent could: • • • • • Subscribe to a Kafka topic and react to events Query a Flink materialized view in real time Understand your Avro schemas without you explaining them Monitor consumer lag and alert you before things break Create new streaming pipelines from natural language X/Bluesky: @gamussa SF JUG, April 2026 18 / 44

MCP + Kafka Architecture ————————————————————————————————— X/Bluesky: @gamussa SF JUG, April 2026 19 / 44

Real Example: Anomaly Detection ————————————————————————————————— You: “Hey Claude, anything weird happening in the orders topic?” Claude (via MCP): ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ I checked the orders topic. In the last 5 minutes: * Order volume dropped 73% compared to the hourly average * Consumer group payment-processor has lag of 45,000 messages * Last successful commit was 8 minutes ago This looks like the payment service is down. Want me to check the payment-service-health topic? X/Bluesky: @gamussa streaming SF JUG, April 2026 20 / 44

Part 4: Let’s Build It ————————————————————————————————— X/Bluesky: @gamussa SF JUG, April 2026 21 / 44

Kafka MCP Server Config ————————————————————————————————— { “mcpServers”: { “confluent-kafka”: { “command”: “npx”, “args”: [ “-y”, “@confluentinc/mcp-confluent”, “-e”, “.env” ] } } } X/Bluesky: @gamussa SF JUG, April 2026 22 / 44

Two Flavors, Same API ————————————————————————————————— ▒▒▒▒ Confluent Cloud ▒▒▒▒ OSS / Self-managed @confluentinc/mcp-confluent mcp-kafka • • • Managed Kafka, Flink, Schema Registry Confluent Cloud authentication Full platform integration • • • Any Kafka cluster Open source Same tools, same API surface github.com/gamussa/mcp-kafka github.com/confluentinc/mcp-c onfluent X/Bluesky: @gamussa SF JUG, April 2026 They’re tool-compatible - same MCP interface, swap the server, 23 / 44

What Tools Does It Expose? ————————————————————————————————— × list-topics cluster × consume-messages × produce-message × list-schemas Registry × create-flink-statement × get-consumer-group-lag health × create-connector × diagnostics → See all topics in the → Read from any topic → Write to any topic → Browse Schema → Run Flink SQL → Monitor consumer → Deploy connectors → Troubleshoot errors X/Bluesky: @gamussa discovers SF JUG, April 2026tically. 24 / 44

Flink SQL via MCP ————————————————————————————————— Ask your agent: “Create a real-time dashboard for flight delays” The agent writes and submits: INSERT INTO flight_dashboard SELECT airline_name, CASE WHEN dep_delay > 0 THEN ‘DELAYED’ ELSE ’ ON_TIME’ END AS status, CASE WHEN HOUR(dep_time) < 12 THEN ‘morning’ WHEN HOUR(dep_time) < 17 THEN ‘afternoon’ X/Bluesky: @gamussa ELSE ‘evSF JUG, April 2026 END AS time_of_day, 25 / 44

The Streaming Lakehouse Pipeline ————————————————————————————————— Kafka + Flink + Tableflow + Iceberg MCP doesn’t replace any of these. It orchestrates them. X/Bluesky: @gamussa SF JUG, April 2026 26 / 44

Demo: The Brewmaster Agent ————————————————————————————————— X/Bluesky: @gamussa SF JUG, April 2026 27 / 44

Meet the Brewmaster ————————————————————————————————— An AI agent that manages a craft brewery’s streaming platform. The setup • • • • Confluent Cloud cluster Topics for sensors, alerts, metrics Flink SQL for real-time analytics Schema Registry for Avro schemas The agent • • • • LangChain4J + TUI4J + Anthropic APIs @confluentinc/mcp-confl uent Natural language only No code, no dashboards Let’s see what happens when you talk to your data platform like a X/Bluesky: @gamussa SF JUG, April 2026 28 / 44

Act 1: Set Up the Brewery ————————————————————————————————— ▍ “Create three topics — brewery-sensors, brewery-alerts, and ▍ brewery-metrics. Then tag them all with a brewery-pipeline tag.” (live demo - agent calls create-topics, create-topic-tags, add-tags-to-topic) Notice: the agent doesn’t just execute — it explains what it built and why. Infrastructure as conversation. X/Bluesky: @gamussa SF JUG, April 2026 29 / 44

Act 2: Fill the Fermenters ————————————————————————————————— ▍ “I have 3 fermenters — an IPA in fermenter-1, a Stout in ▍ fermenter-2, and a Pilsner in fermenter-3. Produce realistic ▍ sensor data for all of them.” (live demo - first produce registers an Avro schema automatically. Agent produces ~30 messages with style-appropriate temperatures) Watch: first message registers the Avro schema with Schema Registry. Every subsequent message is validated against it. No free-form JSON. X/Bluesky: @gamussa SF JUG, April 2026 30 / 44

Act 3: Watch the Brewery ————————————————————————————————— ▍ “Create a Flink SQL job that monitors for temperature anomalies ▍ — alert if any fermenter drifts more than 2 degrees from its ▍ style’s normal range in a 10-second window.” (live demo - agent writes Flink SQL with CASE logic for style-specific thresholds, submits it, verifies it’s running) Watch the SQL stream in token by token. Style-specific thresholds — it knows IPA is different from Pilsner. This is AI writing real-time stream processing. X/Bluesky: @gamussa SF JUG, April 2026 31 / 44

Act 4: Sabotage the IPA ————————————————————————————————— ▍ “Oh no — fermenter-1’s cooling system just failed! Produce ▍ readings showing the temperature climbing fast — 22, 25, 28, 31 ▍ degrees.” (live demo - agent produces escalating readings) ▍ “Did the alerts fire?” (live demo - agent consumes from brewery-alerts — Flink caught the spike) X/Bluesky: @gamussa SF JUG, April 2026 32 / 44

Act 5: Diagnose and Fix ————————————————————————————————— ▍ “What’s the current state of all fermenters? What should we do ▍ about fermenter-1?” (live demo - agent reads latest data, diagnoses IPA as critical, gives real brewing advice) It’s not just a data tool — it’s a domain expert. The Belgian Saison suggestion is real brewing knowledge. X/Bluesky: @gamussa SF JUG, April 2026 33 / 44

Act 6: Schema Evolution ————————————————————————————————— ▍ “The brewery just installed humidity sensors. Show me the ▍ current schema, then evolve it to add humidity_percent, and ▍ verify the new version registered.” Three beats: 1. list-schemas → v1 schema (10 fields) 2. Produce with new field → Schema Registry accepts backward-compatible v2 3. list-schemas → v2 registered alongside v1 X/Bluesky: @gamussaversion 2. SF JUG, April 2026d-compatible. No downtime, no breaking consumers. 34 / 44

Part 5: Honest Trade-offs ————————————————————————————————— X/Bluesky: @gamussa SF JUG, April 2026 35 / 44

What Works ————————————————————————————————— • • • • • ✓ Schema-aware queries - agent understands Avro/Protobuf automatically ✓ Ad-hoc exploration - “show me the last 10 messages from topic X” ✓ Pipeline creation - natural language to Flink SQL ✓ Cross-system correlation - “why is consumer lag growing?” ✓ Debugging - agent reads logs, schemas, configs in context X/Bluesky: @gamussa SF JUG, April 2026 36 / 44

What Doesn’t (Yet) ————————————————————————————————— • • • • ✗ High-throughput consumption - MCP is request/response, not streaming ✗ Latency guarantees - LLM reasoning adds seconds, not milliseconds ✗ Autonomous production writes - do you really want an LLM producing to your orders topic? ✗ Complex stateful processing - Flink is better at this than any agent MCP is for orchestration and exploration, not replacing your streaming pipeline. X/Bluesky: @gamussa SF JUG, April 2026 37 / 44

When to Use What ————————————————————————————————— Use Case │ Tool ───────────────────────────────┼───────────────────── Real-time event processing │ Flink Persistent streaming pipelines │ Kafka + Flink Ad-hoc data exploration │ MCP + LLM Pipeline creation & debugging │ MCP + LLM Anomaly investigation │ MCP + LLM Automated alerting │ Flink (not an agent) “What happened at 3am?” │ MCP + LLM The right tool for the right job. Not everything needs AI. But some things really do. X/Bluesky: @gamussa SF JUG, April 2026 38 / 44

Why This Matters ————————————————————————————————— We’re at an inflection point. Before MCP - AI and streaming are separate worlds. Your dashboards show data. Your agents answer questions. They don’t talk to each other. After MCP - AI agents become first-class citizens of your data platform. They can read streams, understand schemas, create pipelines, and investigate incidents. The streaming platform becomes the nervous system. MCP becomes the interface between human intent and machine execution. X/Bluesky: @gamussa SF JUG, April 2026 39 / 44

What to Do Monday Morning ————————————————————————————————— 1. Install the Confluent MCP server - npx @confluentinc/mcp-confluent 2. Connect Claude Desktop to your dev cluster (not prod… yet) 3. Ask it questions - “What topics do I have? What’s the schema for X?” 4. Build an agent for your own streaming use case 5. Come talk to me - I’ll be around after X/Bluesky: @gamussa SF JUG, April 2026 40 / 44

Resources ————————————————————————————————— • • • • • MCP Spec - modelcontextprotocol.io Confluent MCP github.com/confluentinc/mcp-conflu ent OSS Kafka MCP github.com/gamussa/mcp-kafka Confluent Cloud confluent.io/confluent-cloud This talk - speaking.gamov.io X/Bluesky: @gamussa SF JUG, April 2026 41 / 44

As always, have a nice day. ————————————————————————————————— Viktor Gamov - X/Bluesky: @gamussa

Questions? ————————————————————————————————— I’ll be around after. Find me near the pizza. Find me on X/Bluesky: @gamussa

X/Bluesky: @gamussa SF JUG, April 2026 44 / 44