Jakub Korab

Ameliant

The Myth of the Magical Messaging Fabric

To most developers messaging systems are disregarded as 'just a pipe'. Without understanding the reasoning behind them and the underlying design decision, we are frustrated by how we think they should work in production. In this session we will compare and contrast Apache ActiveMQ and Apache Kafka, and how their designs address message distribution, reliability, scalability, and performance.

Full abstract

Messaging systems underpin architectures ranging from stream processing, through to event sourcing and communication between loosely-coupled microservices. To most developers the technology is tangential to the business they work in, and is often disregarded with much hand waving as “just a pipe” for message distribution. Without a clear understanding of their capabilities and underlying design decisions, we imagine myths about how these systems “should” work and are frustrated in production by behaviours emerging from their design trade-offs.

In this session you will learn how to think about messaging as we compare and contrast the approaches taken by two similar but fundamentally different messaging platforms – Apache ActiveMQ and Apache Kafka. We will discuss how their underlying philosophies impact message distribution, reliability, scalability, and performance.

Along the way we will dip into how IOPS, latency and bandwidth affect throughput, why messaging systems do not play nicely with containers, and the unicorn of once-and-only-once delivery in distributed systems.