Contact Us
WhatsApp
SaaS Log InXRay Login
Apache Kafka

Integration with Apache Kafka

Nastel Technologies®
September 19, 2022

You can integrate Edge Flow Manager (EFM) with Apache Kafka and forward agent heartbeats to defined Kafka topics. Learn how to perform the integration with Apache Kafka.

 

To integrate EFM with Kafka, you need to configure Kafka and EFM properties.

 

EFM supports the forwarding of agent heartbeats and acknowledges messages exchanged on the C2 protocol between the EFM server and MiNiFi agents. You can also enable two-way TLS or authentication using SASL_SSL for secure communication between EFM and Kafka brokers.

 

The integration is not on critical path so, if there is any communication issue between the server and the Kafka broker(s), it results in error logs but not in core functionality degradation. When connection between EFM and Kafka is restored, the accumulated messages are forwarded. You can fine tune the buffering related settings.

 

Configuring Kafka for use by EFM
To integrate Edge Flow Manager (EFM) with Apache Kafka, you need to create Kafka topics that receive the heartbeats and acknowledgments from MiNiFi agents.

 

Configuring the EFM properties
To integrate EFM with Kafka, you need to configure the EFM properties in the efm.properties file.

 

This article originally appeared on doc.cloudera.com. To read the full article, click here.

Nastel Technologies is the global leader in Integration Infrastructure Management (i2M). It helps companies achieve flawless delivery of digital services powered by integration infrastructure by delivering tools for Middleware Management, Monitoring, Tracking, and Analytics to detect anomalies, accelerate decisions, and enable customers to constantly innovate, to answer business-centric questions, and provide actionable guidance for decision-makers. It is particularly focused on IBM MQ, Apache Kafka, Solace, TIBCO EMS, ACE/IIB and also supports RabbitMQ, ActiveMQ, Blockchain, IOT, DataPower, MFT, IBM Cloud Pak for Integration and many more.

 

The Nastel i2M Platform provides:

Comments

Write a comment
Leave a Reply
Your email address will not be published. Required fields are marked *
Comment * This field is required!
First name * This field is required!
Email * Please, enter valid email address!
Subscribe

Schedule a Meeting to Learn More