Reading Many-to-One Asynchronously
Perhaps instead of a single slow datasource, you have several slow datasources that produce complementary data.
In this scenario, you can use the
AsyncMultiReader class to read from all sources concurrently. This reader uses
a separate thread to load each upstream source into its buffer. Like the
AsyncReader, downstream requests
won't have to wait when pulling from the buffer unless it's empty.
This example reads from two CSV files in parallel and merges their data into a union to be written out downstream. Each
CSVReader can be replaced with other readers, including filters, transformers, and aggregators.