Running Jobs

Data Pipeline transfers records from readers to writers using jobs. The chain of readers and writers in a job is called a pipeline. You might find the terms job and pipeline used interchangeably.

As with the Java I/O classes, you can program the transfer yourself (open streams, read-write records in a loop, close streams), however the Job class offers many benefits:

  1. It can be monitored and managed using JMX (Java Management Extensions)
  2. It can be run synchronously or asynchronously
  3. It can be cancelled, paused, and resumed
  4. It's thread-safe
  5. It's state can safely be logged at any time or on completion
  6. It publishes lifecycle events to the system event bus
  7. It has built-in error handling
  8. It supports callbacks
  9. It tracks stats about itself
Mobile Analytics