Use Retry Operation With Retry Condition

This example demonstrates how to retry an operation based on a custom retry condition using the RetryingOperation class.

Java Code

package com.northconcepts.datapipeline.examples.cookbook;

import com.northconcepts.datapipeline.avro.AvroReader;
import com.northconcepts.datapipeline.core.DataReader;
import com.northconcepts.datapipeline.core.DataWriter;
import com.northconcepts.datapipeline.csv.CSVWriter;
import com.northconcepts.datapipeline.job.Job;
import com.northconcepts.datapipeline.retry.RetryStrategy;
import com.northconcepts.datapipeline.retry.RetryingOperation;

import java.io.File;
import java.io.IOException;

public class UseRetryingOperationWithRetryCondition {


    public static void main(String[] args) throws Throwable {
        RetryingOperation retryingOperation = new RetryingOperation<>();
        retryingOperation.setInitialRetryDelay(1000L);
        retryingOperation.setMaxRetryCount(3);
        retryingOperation.setStrategy(RetryStrategy.EXPONENTIAL_BACKOFF);
        retryingOperation.setRetryPredicate(context -> context.getLastException() instanceof IOException);

        retryingOperation.call(() -> {
            DataReader reader = new AvroReader(new File("example/data/input/twitter.avro"));
            DataWriter writer = new CSVWriter(new File("example/data/output/twitter.csv"));

            Job job = Job.run(reader, writer);
            return job.getRunningTimeAsString();
        });
    }
}

Code Walkthrough

1. RetryingOperation is created to wrap an operation that may fail and needs retry logic.
2. setInitialRetryDelay() configures the delay before the first retry attempt. The first retry will wait 1000 milliseconds (1 second) before attempting again.
3. setMaxRetryCount() defines how many retry attempts are allowed. Here, the operation is limited to three retries.
4. setStrategy(RetryStrategy.EXPONENTIAL_BACKOFF) sets the retry strategy where the delay between retries increases exponentially. For example:
   • 1st retry → 1 second
   • 2nd retry → 2 seconds,
   • 3rd retry → 4 seconds
5. setRetryPredicate() configures a specific condition for triggering a retry. In this case, the operation only retries if the last error was an IOException; all other exceptions will result in an immediate failure.
6. The logic to be retried is encapsulated and executed within the call() method.
7. AvroReader is initialized to read data from twitter.avro.
8. Job.run() is used to transfer records from reader to CSVWriter.
9. getRunningTimeAsString() returns the total execution time of the job in a human-readable format.