Stream Operators

Once data is flowing into the pipeline, Stream Operators provide the ability to filter, enrich, transform, and summarize data in-flight.

Stream Operators are not a true third concept. Instead, they're a special kind of reader or writer that operates on other readers and writers. They use the decorator pattern to wrap other readers and writers to intercept (and possibly alter) records going into them or coming out from them.

This allows stream operators to transform and process data in-flight, in a common way, regardless of the data's original source or ultimate target. This also allows simple operators to be chained together to form complex processing sequences.