Grab Reduces Traffic Cost for Kafka Consumers on AWS to Zero
Grab took advantage of the ability of Apache Kafka consumers to connect to the broker node in the same availability zone (AZ) introduced in Kafka 2.3 and reduced the traffic cost on AWS to zero for re...
Data Infrastructure at IFTTT
Data is a big deal at IFTTT. Our business development and marketing teams rely on data to make critical business decisions. The product team relies on data to run tests, learn about product usage, and...
KNet: A New Way to Use Apache Kafka from .NET
KNet offers a new approach for using Apache Kafka from .NET by executing the Java code of the validated Apache Kafka distribution directly, rewriting APIs only. With KNet, users have access to the sam...
KNet: the Apache Kafka .NET suite
KNet is a comprehensive .NET suite for Apache Kafka APIs providing all features: Producer, Consumer, Admin, Streams, Connect, backends (ZooKeeper and Kafka). This project aims to create a set of libra...
Producing and Consuming Kafka Messages in CloudEvents Format Using the C# SDK
In this article, Vasil Kosturski presents a simple Kafka producer and consumer using the C# CloudEvents SDK. The article covers a practical example of how to use CloudEvents with Kafka and provides in...
Testing Kafka-based Asynchronous Workflows Using OpenTelemetry
Asynchronous architectures are common in cloud native applications due to their decoupling services and improving scalability and reliability. Testing these workflows, especially with Apache Kafka, po...
Effortless Kafka integration for your web services
FastKafka is a Python library that simplifies the process of writing producers and consumers for Kafka topics. It makes it easy to prototype and develop high-performance Kafka-based services with mini...
Integration Tests for your Kafka Producer with “Testcontainers” in C#
In this article, Vasil Kosturski presents an approach to building integration tests for your Kafka producer to verify that messages are produced correctly using Docker containers. The testing pattern ...
Problem Solving Strategies for Microservice Architecture Part III
This article explores the use of the CQRS pattern for implementing complex queries in a microservice architecture. CQRS offers advantages such as independent scaling, improved separation of concerns, ...
Real-Time Query Across Multiple Microservices Using Kafka
In a microservices architecture, services work independently with their own databases and communicate using messaging. However, querying data across multiple services can be a challenge. One solution ...
Kafka Retries and Maintaining the Order of Retry Events
In a microservices system, retries are crucial especially when a service goes down. But with an event-driven architecture and Kafka consuming events, retry strategies become more complex. This article...
Build a Real-time Data Analytics Pipeline with Airbyte, Kafka, and Pinot
Learn how to use Airbyte, Kafka, and Pinot to build a real-time data pipeline for a user-facing analytics dashboard. This article explores three popular open-source products in the data space, Airbyte...
Event-Driven Architecture with Apache Kafka for .NET Developers Part 1 - Event Producer
Learn how to implement an event-driven architecture using Apache Kafka as the event router. This article covers setting up the development environment and publishing messages to Kafka. Kafka is a hori...
Event-Driven Architecture with Apache Kafka for .NET Developers Part 2 - Event Consumer
In this article, the author discusses the implementation of an event consumer using Kafka for .NET developers with a specific focus on the Manager service. The article also covers the necessary NuGet ...