Apache Kafka: Creating Real-Time Distributed Microservices

Apache Kafka is proving its mettle in the burgeoning microservices space Research values the global microservices industry at $1.8 billion, and microservices deployments continue to rise across the board. With that sort of interest, there is now a great collection of frameworks, libraries and other packages designed to help construct elegant, reactive services.  One of [...]

By |Apache, Kafka|

Confluent, an Apache Kafka service provider adopts a new license to fight against cloud service providers

A common trend of software firms limiting their software licenses to prevent cloud service providers from exploiting their open source code is all the rage these days. One such software firm to have joined this move is Confluent, an Apache Kafka service provider, who announced its new Confluent Community License, two weeks back. The new license [...]

By |Apache, Kafka|

Expanding Visibility With Apache Kafka

Expanding Visibility With Apache Kafka - What would you do if you had terabytes of operational data being generated in production each day, and hundreds of engineering teams wanting to use that data to improve their services … but no way to connect the two? A Broader Vision This started us thinking bigger. There are a vast [...]

By |Kafka, logging|

How Apache Kafka Inspired Our Platform Events Architecture

There’s a class of scalability challenges that are best approached using an asynchronous, event-driven architecture — particularly when it comes to doing data integrations. To support these uses cases Salesforce recently added a feature called “Platform Events” which exposes a time-ordered immutable event stream to our customers. Our multitenant setup posed some challenges for traditional messaging systems and luckily Apache Kafka [...]

By |Kafka, logging, Middleware|

Developing a Deeper Understanding of Apache Kafka Architecture

The Apache Kafka distributed streaming platform features an architecture that – ironically, given the name – provides application messaging that is markedly clearer and less Kafkaesque when compared with alternatives. In this article, we’ll take a detailed look at how Kafka’s architecture accomplishes this. The Kafka Components – Universal Modeling Language (UML) Kafka’s main architectural components include Producers, Topics, [...]

By |Kafka|

Nastel’s AutoPilot Monitoring for Apache Kafka

Nastel's® AutoPilot® for Apache Kafka provides operational and transactional monitoring for Apache Kafka, the open-source stream processing platform developed by the Apache Software Foundation, written in Scala and Java. Apache Kafka is a unified, high-throughput, low-latency platform for handling real-time data feeds. It utilizes a massively scalable pub/sub message queue—designed as a distributed transaction log—as its [...]

By |Apache, Kafka, open-source|