The Apache Kafka distributed streaming platform features an architecture that – ironically, given the name – provides application messaging that is markedly clearer and less Kafkaesque when compared with alternatives. In this article, we’ll take a detailed look at how Kafka’s architecture accomplishes this.

The Kafka Components – Universal Modeling Language (UML)

Kafka’s main architectural components include Producers, Topics, Consumers, Consumer Groups, Clusters, Brokers, Partitions, Replicas, Leaders, and Followers. This simplified UML diagram describes the ways these components relate to one another:

It’s important to note the relationships between broker, replica, and partition components that are highlighted, such as:

  • Kafka clusters can have one or more brokers.
  • Brokers can host multiple replicas.
  • Topics can have one or more partitions.
  • A broker can host zero or one replica per partition.
  • A partition has one leader replica and zero or more follower replicas.
  • Each replica for a partition needs to be on a separate broker.
  • Every partition replica needs to fit on a broker, and a partition can’t be divided over multiple brokers.
  • Every broker can have one or more leaders, covering different partitions and topics.

 

This article originally appeared on insidebigdata.com.  To read the full article, click here.

 

Nastel Comments:

AutoPilot for Apache Kafka provides a single point of instant analysis and visibility into business services and application ecosystems spanning your entire enterprise.  To learn more, click here.