Apache Kafka has grown a lot in functionality and reach in last couple of years. It’s used in production by one third of the Fortune 500, including seven of the top 10 global banks, eight of the top 10 insurance companies, and nine of the top 10 U.S. telecom companies.
Imagine you run a travel website. The price of hotels and flights keeps changing all the time. A few components of your system (price alerts, analytics) need to be informed of these changes. You post the changes on Kafka topics, and each component that needs to be notified acts as a subscriber. All nodes of a single subscriber system form a single consumer group. A given message is sent to only one node in the consumer group. This way each component gets the copy of the message, plus work gets effectively divided inside each component.
Website activity (page views, searches, or other actions users may take) can be tracked and analysed through Kafka. In fact, this was the original use case for which Kafka was invented at LinkedIn. Website activities are published to central topics with one topic per activity type. The feed can be processed in real time to gain insights into user engagement, drop-offs, page flows, and so on.
Imagine you have location data coming in from GPS beacons or smartphone devices, and you want to process it in real time to show vehicle path, distance traveled, and so on. Incoming data can be published on Kafka topics and processed with Stream API. Stateful processing with windowing comes in handy when you need to extract and process all location data of a given user for a certain period of time.
This article originally appeared on blog.amirathi.com. To read the full article, click here.
Nastel Technologies uses machine learning to detect anomalies, behavior and sentiment, accelerate decisions, satisfy customers, innovate continuously. To answer business-centric questions and provide actionable guidance for decision-makers, Nastel’s AutoPilot® for Analytics fuses:
- Advanced predictive anomaly detection, Bayesian Classification and other machine learning algorithms
- Raw information handling and analytics speed
- End-to-end business transaction tracking that spans technologies, tiers, and organizations
- Intuitive, easy-to-use data visualizations and dashboards
Nastel Technologies is the global leader in Integration Infrastructure Management (i2M). It helps companies achieve flawless delivery of digital services powered by integration infrastructure by delivering Middleware Management, Monitoring, Tracking, and Analytics to detect anomalies, accelerate decisions, and enable customers to constantly innovate, to answer business-centric questions, and provide actionable guidance for decision-makers. It is particularly focused on IBM MQ, Apache Kafka, Solace, TIBCO EMS, ACE/IIB and also supports RabbitMQ, ActiveMQ, Blockchain, IOT, DataPower, MFT and many more.
The Nastel i2M Platform provides:
- Secure self-service configuration management with auditing for governance & compliance
- Message management for Application Development, Test, & Support
- Real-time performance monitoring, alerting, and remediation
- Business transaction tracking and IT message tracing
- AIOps and APM
- Automation for CI/CD DevOps
- Analytics for root cause analysis & Management Information (MI)
- Integration with ITSM/SIEM solutions including ServiceNow, Splunk, & AppDynamics