Connecting Pipelines to an Event Bus

While the event bus is general enough to be used in any application, it comes with built-in connectors for reading and writing record to a bus.

EventBusWriter

The EventBusWriter allows you to send records from any pipeline to an event bus. When constructing an EventBusWriter, you need to supply it with the event bus you are targeting, plus a topic. The topic can be a string, enum or other object and is used by the bus to distinguish one stream of records from another.

EventBusReader

The EventBusReader pulls records from an event bus into your pipeline. It's constructor takes the source event bus, plus zero or more topics to read from. If the topic is null or missing, the reader will read all records regardless of their topic.

One advantage of using an event bus is that you can connect any number of readers to a single topic. You can also connect any number of writers or the same or different topics. In many cases, and event bus can be a more flexible option to DeMux-based multi-threading.

Event Bus Pipelines

The event bus treats data as a flowing stream. Unlike its bigger, Enterprise Service Bus (ESB) cousin, the event bus does not persist records. If your reader isn't connected to the bus when the event is published, it will not be able to see it later. That's why it's important to connect your readers to the bus prior to sending records from a writer.

When records are published to the bus, the writer send it immediately to the bus which queues it in its buffer or blocks the writer if the buffer is full. On the delivery side, the bus puts records into the reader's queue, unless the reader is full in which case the bus will block.

This example uses a single writer to send records to the bus using the purchases topic (lines 23-26). Two separate readers then read from purchases topic and write to their individual targets (lines 11-13 and 16-19).