Building Event Driven Services with Apache Kafka and Kafka Streams Ben Stopford @benstopford 1 There is a book! http://bit.ly/designing-event-driven-systems 2 Event Driven Architectures Event Sourcing (DDD) Stream Processing 3 Today’s ecosystems get pretty big • 2.2 trillion messages per day (6 Petabytes) • Up to 400 Microservices pre cluster. • 20-200 Brokers per cluster 4 4 Today’s ecosystems get pretty big • 1 billion messages per day • 20,000 messages per second • 100 teams 5 5 Event Driven Architectures Event Sourcing (DDD) Events Stream Processing 6 Notification Data replication 7 Notification Data replication 8 Streaming Platforms Kafka Streams / KSQL KAFKA Serving Layer (Cassandra etc.) High Throughput Data is embedded in Clustered Messaging each engine Java App 9 Streaming Pipeline apps_opened (a) Apps Opened, Per App, Per day opened_per_day (c) Unstable Applications unstable_apps crashed_per_day (b) Crashes, app_crashes Per App, Per day 10