Jobs Logging
A fourth way of monitoring your jobs is to provide a DataWriter to the job where it can record its stats once the job finishes. However, you can request the job log
its current stats any time by calling the log() method. Calling log too often will cause your job to run slow since the job momentarily stops running to take a snapshot of its entire state.
Here's an example of the data sent to the log writer once the job finishes.
| Fields | Description |
|---|---|
| 0 | The timestamp for this log entry. |
| 1-15 | Field names beginning with "job_" are job-level stats and will be the same for all records logged a the same time. |
| 16-24 | Field names beginning with "endpoint_" are reader/writer-level stats. One record will be emitted for each reader/writer in this job's pipeline.
The first record corresponds to the first reader in the pipeline chain and the last record corresponds to the last writer in the chain. Field 16 (endpoint_index) identifies the reader or writer's position in the pipeline. Field's 17 and 18 (endpoint_simple_type and endpoint_qualified_type) indicate the reader/writer class for that step of the pipeline. |
