Blog Logo
TAGS

Processing data in parallel using Channels

Thanks to the Task Asynchronous Programming model writing asynchronous code in .NET is usually straightforward. Channels give us a way to communicate between concurrent (async) operations in .NET. Lets start with a theoretical scenario. Imagine you have a web application that periodically receives data from a set of devices at frequent intervals. The data then must be analyzed, and the result of the analysis should be persisted in some kind of storage. Because the analysis can take a long time, it has to be done in the background so as not to make the devices wait for their results. The data from each device is independent and therefore can be processed in parallel to increase the efficiency of your application. And thats what we will be building today! Lets start with a simple wrapper for the data that also contains a Key property, which is used for grouping the data. Two DataWithKey instances with the same Key cannot be processed in parallel. With the scenario above in mind, the Key would indicate the device identifier. The IDataProcessor interface will be used to schedule the data for processing in the background. The data processor should be available throughout the whole lifetime of the application, we can use the BackgroundService base class and register it as a hosted service.