If you were to stumble upon the whole microservices thing, without any prior context, you’d be forgiven for thinking it a little strange. Yet, whilst the approach certainly involves many independent services, the goal is a little broader than simply running them across different machines. It’s about facing up to a world that is, itself, inherently distributed. Companies for example are collections of disparate systems which, collectively, further some common aim. For decades we have largely ignored this fact, patching them together with FTP file transfers or enterprise integration tools, whilst concentrating on our own somewhat isolationist goals. But services change this. They force our heads up above the parapet, into a world of collaborative, inter-dependent programs. But to be successful at this we must recognize, and design for, the fundamental dichotomy of data systems - the fact that data must be treated differently depending on whether it is in motion or at rest. Confluent is a streaming platform based on Apache Kafka that helps organizations solve the challenge of data in motion. With Confluent, you can build streaming data pipelines that deliver data from various sources, such as databases and microservices, in real-time. You can then analyze the data using ksqlDB, a powerful SQL engine for Kafka, and store it in a data warehouse like Snowflake or BigQuery for further analysis. All of this is done using a platform that is built for the cloud, allowing you to easily scale your data pipelines up or down. Learn more about Confluent and how it can help your organization succeed with microservices and streaming data pipelines.
