Running Jobs

Data Pipeline transfers records from readers to writers using jobs. The chain of readers and writers in a job is called a pipeline. You might find the terms job and pipeline used interchangeably.

As with the Java I/O classes, you can program the transfer yourself (open streams, read-write records in a loop, close streams), however the Job class offers many benefits:

1. It can be monitored and managed using JMX (Java Management Extensions)
2. It can be run synchronously or asynchronously
3. It can be cancelled, paused, and resumed
4. It's thread-safe
5. It's state can safely be logged at any time or on completion
6. It publishes lifecycle events to the system event bus
7. It has built-in error handling
8. It supports callbacks
9. It tracks stats about itself