# JIT Chapter 82: Event-Driven Architecture with Kafka (/docs/deploying-agent-factories-in-the-cloud/82-event-driven-kafka/overview)

---
type: Document
title: "JIT Chapter 82: Event-Driven Architecture with Kafka"
description: "Build your Kafka skill first, then learn to understand and refine it into a production-ready asset"
status: stable
order: 8200
generated:
  by: ag-archive/0.1.0
  at: 2026-08-26T00:00:00Z
ksor:
  audience: [public]
  approval:
    by: ag-archive/0.1.0
    at: 2026-08-26T00:00:00Z
trust_tier: unverified
build_id: sha256:c8912c3008ae345a7fdc33ace656531ad74f37c065177057c390c7d759f1e970
source_commit: dad4758bb59919c7b9130444778ade062a2dce3e
ksor_version: 0.0.47
---

You build the `kafka-events` skill first, then use each lesson to test and deepen it—from EDA fundamentals to production-grade operations. Kafka 4.0+ KRaft mode (no ZooKeeper) is the default.

---

## Goals

- Understand event-driven architecture and Kafka’s core model (topics, partitions, consumer groups)
- Deploy Kafka with Strimzi in KRaft mode
- Build reliable producers/consumers with delivery guarantees and transactions
- Integrate Kafka with FastAPI using schemas (Avro + schema registry)
- Apply advanced patterns: Connect, CDC with Debezium, agent events, saga
- Operate Kafka: production config, monitoring, debugging
- Capture everything in a reusable Kafka skill

---

## Lesson Progression

| # | Lesson | Focus |
|---|--------|-------|
| **0** | Build Your Kafka Skill | Scaffold from docs |
| 1-3 | EDA Foundations | Why events, EDA patterns, Kafka mental model |
| 4-8 | Kafka Core | Strimzi deploy, producers, consumers, groups |
| 9-13 | Production Patterns | Async FastAPI integration, schemas, delivery semantics, transactions, reliability config |
| 14-17 | Advanced Patterns | Connect, Debezium CDC, agent event design, saga orchestration |
| 18-19 | Operations | Production Strimzi, monitoring/debugging |
| 20 | AI-Assisted Development | Use AI to generate/tune configs |
| **21** | Capstone: Event-Driven Notifications | Production pipeline |

Each lesson ends with a skill reflection: test, find gaps, and improve.

---

## Outcome & Method

You finish with a production-ready Kafka deployment, reliable producer/consumer code integrated with FastAPI, and a Kafka skill for future projects. The chapter follows the 4-Layer approach: foundations → production patterns → AI-assisted authoring → spec-driven capstone.

---

## Prerequisites

- Chapters 79-81: container image and Kubernetes/Helm familiarity
- Ability to run a local Kubernetes cluster (e.g., Docker Desktop) for Strimzi
- **Implement reliable producers**: acks semantics, retries, idempotent producer, error handling
- **Implement robust consumers**: Consumer groups, rebalancing, offset management, lag monitoring
- **Integrate with FastAPI**: Async producers/consumers, lifespan events, background tasks
- **Design event schemas**: Avro with Schema Registry, schema evolution, breaking change prevention
- **Apply delivery guarantees**: At-least-once, at-most-once, exactly-once semantics and trade-offs
- **Use transactions**: Consume-process-produce pattern, zombie fencing, read_committed isolation
- **Build data pipelines**: Kafka Connect, Debezium CDC, outbox pattern for microservices
- **Implement agent patterns**: Task events, notification fanout, audit logs, saga pattern
- **Run Kafka on Kubernetes**: Strimzi operator, Kafka CRDs, KRaft mode, production configuration
- **Debug production issues**: Consumer lag, under-replicated partitions, rebalancing storms

## Technology Choices

| Component | Choice | Rationale |
|-----------|--------|-----------|
| **Kafka Operator** | Strimzi | CNCF project, industry standard for Kafka on K8s |
| **Kafka Mode** | KRaft (no ZooKeeper) | Kafka 4.0+ default, simpler architecture |
| **Python Client** | confluent-kafka-python | Best performance, native async, Schema Registry support |
| **Schemas** | Avro + Confluent Schema Registry | Industry standard, evolution support |
| **Platform** | Docker Desktop Kubernetes | Consistent with Chapters 79-81 |
| **CDC** | Debezium | Best-in-class change data capture |

## What's NOT Covered

This chapter focuses on **developer skills**, not SRE operations:

- Docker Compose — we use Kubernetes throughout Part 7
- Multi-datacenter replication (MirrorMaker 2)
- Security deep dive (SASL, SSL, ACLs) — covered at overview level only
- Kafka Streams framework — separate advanced topic
- Broker hardware sizing and tuning
- ZooKeeper — removed in Kafka 4.0

## Looking Ahead

This chapter teaches Kafka directly. Chapter 83 (Dapr) shows how to abstract pub/sub behind Dapr's API, making your code portable across message brokers while retaining the concepts you learned here.