com.northconcepts.datapipeline.retry

Class RetryingReader

java.lang.Object

com.northconcepts.datapipeline.core.DataObject

com.northconcepts.datapipeline.core.Endpoint

com.northconcepts.datapipeline.core.DataEndpoint

com.northconcepts.datapipeline.core.DataReader

com.northconcepts.datapipeline.core.ProxyReader

com.northconcepts.datapipeline.retry.RetryingReader

public class RetryingReader
extends ProxyReader

A proxy that attempts to continue reading on failure.

Nested Class Summary

Nested classes/interfaces inherited from class com.northconcepts.datapipeline.core.DataEndpoint

DataEndpoint.State

Field Summary

Fields inherited from class com.northconcepts.datapipeline.core.DataReader

fieldLineage, recordLineage

Fields inherited from class com.northconcepts.datapipeline.core.DataEndpoint

lastRecord, PRODUCT, PRODUCT_VERSION, VENDOR, XML_INPUT_FACTORY_KEY

Fields inherited from class com.northconcepts.datapipeline.core.Endpoint

BUFFER_SIZE, captureElapsedTime, DEFAULT_READ_BUFFER_SIZE

Fields inherited from class com.northconcepts.datapipeline.core.DataObject

id, log, name, TIMESTAMP_FORMAT

Constructor Summary

Constructors

Constructor and Description
RetryingReader(DataReader nestedDataReader)

Method Summary

Modifier and Type	Method and Description
DataException	addExceptionProperties(DataException exception) Adds this endpoint's current state to a DataException.
void	close() Indicates that this endpoint has finished reading or writing.
String	getDiscardReasonFieldName() Returns the field in the record to place the description of the exception that was thrown by the failing read or null if one should not be added.
DataWriter	getDiscardWriter() Returns the discard sink for failing records or null if one was not assigned.
long	getExceptionCount() Returns the total number of exceptions/failures seen by this reader.
Throwable	getFirstException() Returns the first exception thrown for any individual read attempt.
long	getInitialRetryDelay() Returns the amount of time to wait (in milliseconds) after the first error, before retrying (default 1 second).
Throwable	getLastException() Returns the last exception thrown for any individual read attempt.
long	getMaxErrorCount() Returns the number of retries allowed across all read attempts (default max long - 1).
int	getMaxRetryCount() Returns the number of retries allowed for any individual read attempt (default 5).
IRetryStrategy	getStrategy() Returns the retry strategy (default RetryStrategy.EXPONENTIAL_BACKOFF).
protected void	onFailure(Record record, Throwable exception)
void	open() Makes this endpoint ready for reading or writing.
protected Record	readImpl() Overridden by subclasses to read the next record from this DataReader.
RetryingReader	setDiscardReasonFieldName(String discardReasonFieldName) Specifies the field in the record to place the description of the exception that was thrown by the failing read or null if one should not be added.
RetryingReader	setDiscardWriter(DataWriter writer) Assign a discard sink for failing records.
RetryingReader	setInitialRetryDelay(long initialRetryDelay) Sets the amount of time to wait (in milliseconds) after the first error, before retrying (default 1 second).
RetryingReader	setMaxErrorCount(long maxErrorCount) Sets the number of retries allowed across all read attempts (default max long -1).
RetryingReader	setMaxRetryCount(int maxRetryCount) Sets the number of retries allowed for any individual read attempt (default 5).
RetryingReader	setStrategy(IRetryStrategy strategy) Sets the retry strategy (default RetryStrategy.EXPONENTIAL_BACKOFF).

Methods inherited from class com.northconcepts.datapipeline.core.ProxyReader

available, getNestedReader, interceptRecord, map, map, setNestedDataReader, setNestedDataReader

Methods inherited from class com.northconcepts.datapipeline.core.DataReader

addLineage, getBufferSize, getNestedEndpoint, getReader, getRootEndpoint, getRootReader, isExhausted, isLineageSupported, isSaveLineage, peek, pop, push, read, setSaveLineage, skip

Methods inherited from class com.northconcepts.datapipeline.core.DataEndpoint

decrementRecordCount, enableJmx, getLastRecord, getRecordCount, getRecordCountAsBigInteger, getRecordCountAsString, incrementRecordCount, isRecordCountBigInteger, resetRecordCount, toString

Methods inherited from class com.northconcepts.datapipeline.core.Endpoint

addElapsedtime, assertClosed, assertNotOpened, assertOpened, finalize, getClosedOn, getDescription, getElapsedTime, getElapsedTimeAsString, getOpenedOn, getOpenElapsedTime, getOpenElapsedTimeAsString, getSelfTime, getSelfTimeAsString, getState, isCaptureElapsedTime, isClosed, isOpen, setCaptureElapsedTime, setDescription

Methods inherited from class com.northconcepts.datapipeline.core.DataObject

exception, exception, exception, getId, getName, resetID

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

RetryingReader

public RetryingReader(DataReader nestedDataReader)

Method Detail

getMaxRetryCount

public int getMaxRetryCount()

Returns the number of retries allowed for any individual read attempt (default 5).

setMaxRetryCount

public RetryingReader setMaxRetryCount(int maxRetryCount)

Sets the number of retries allowed for any individual read attempt (default 5).

getMaxErrorCount

public long getMaxErrorCount()

Returns the number of retries allowed across all read attempts (default max long - 1).

setMaxErrorCount

public RetryingReader setMaxErrorCount(long maxErrorCount)

Sets the number of retries allowed across all read attempts (default max long -1).

getInitialRetryDelay

public long getInitialRetryDelay()

Returns the amount of time to wait (in milliseconds) after the first error, before retrying (default 1 second).

setInitialRetryDelay

public RetryingReader setInitialRetryDelay(long initialRetryDelay)

Sets the amount of time to wait (in milliseconds) after the first error, before retrying (default 1 second).

getStrategy

public IRetryStrategy getStrategy()

Returns the retry strategy (default RetryStrategy.EXPONENTIAL_BACKOFF).

setStrategy

public RetryingReader setStrategy(IRetryStrategy strategy)

Sets the retry strategy (default RetryStrategy.EXPONENTIAL_BACKOFF).

getExceptionCount

public long getExceptionCount()

Returns the total number of exceptions/failures seen by this reader. This value is also the number of retries as long as maxErrorCount hasn't been exceeded.

getFirstException

public Throwable getFirstException()

Returns the first exception thrown for any individual read attempt. This value will reset on the next read attempt after a successful retry.

getLastException

public Throwable getLastException()

Returns the last exception thrown for any individual read attempt. This value will reset on the next read attempt after a successful retry.

getDiscardWriter

public DataWriter getDiscardWriter()

Returns the discard sink for failing records or null if one was not assigned.

setDiscardWriter

public RetryingReader setDiscardWriter(DataWriter writer)

Assign a discard sink for failing records.

Parameters:

writer - the discard sink

getDiscardReasonFieldName

public String getDiscardReasonFieldName()

Returns the field in the record to place the description of the exception that was thrown by the failing read or null if one should not be added.

setDiscardReasonFieldName

public RetryingReader setDiscardReasonFieldName(String discardReasonFieldName)

Specifies the field in the record to place the description of the exception that was thrown by the failing read or null if one should not be added.

open

public void open()
          throws DataException

Description copied from class: DataEndpoint

Makes this endpoint ready for reading or writing.

Overrides:

open in class ProxyReader

Throws:

DataException

close

public void close()
           throws DataException

Description copied from class: DataEndpoint

Indicates that this endpoint has finished reading or writing.

Overrides:

close in class ProxyReader

Throws:

DataException

readImpl

protected Record readImpl()
                   throws Throwable

Description copied from class: DataReader

Overridden by subclasses to read the next record from this DataReader. The default implementation of DataReader.read() now insures that this method will not be called again after it returns a null.

If no record is available, null will be returned.

Overrides:

readImpl in class ProxyReader

Throws:

Throwable

onFailure

protected void onFailure(Record record,
                         Throwable exception)

addExceptionProperties

public DataException addExceptionProperties(DataException exception)

Description copied from class: Endpoint

Adds this endpoint's current state to a DataException. Since this method is called whenever an exception is thrown, subclasses should override it to add their specific information.

Overrides:

addExceptionProperties in class ProxyReader