com.northconcepts.datapipeline.core
Class RemoveDuplicatesReader

java.lang.Object
  com.northconcepts.datapipeline.core.DataEndpoint
      com.northconcepts.datapipeline.core.DataReader
          com.northconcepts.datapipeline.core.ProxyReader
              com.northconcepts.datapipeline.core.RemoveDuplicatesReader

public class RemoveDuplicatesReader
extends ProxyReader

A proxy that removes duplicate records. Duplicates can be determined by checking all fields (RemoveDuplicatesReader(DataReader)) or subset of fields (RemoveDuplicatesReader(DataReader, FieldList)).

Nested Class Summary

Nested classes/interfaces inherited from class com.northconcepts.datapipeline.core.DataEndpoint

DataEndpoint.State

Field Summary

Fields inherited from class com.northconcepts.datapipeline.core.DataEndpoint

BUFFER_SIZE, log

Constructor Summary

RemoveDuplicatesReader(DataReader targetDataReader)

RemoveDuplicatesReader(DataReader targetDataReader, FieldList fields)

Method Summary

protected Record

interceptRecord(Record record)

Methods inherited from class com.northconcepts.datapipeline.core.ProxyReader

addExceptionProperties, available, close, getNestedReader, open, readImpl, setNestedDataReader, setNestedDataReader

Methods inherited from class com.northconcepts.datapipeline.core.DataReader

getBufferSize, getRootReader, peek, pop, push, read, skip

Methods inherited from class com.northconcepts.datapipeline.core.DataEndpoint

assertNotOpened, assertOpened, decrementRecordCount, exception, exception, exception, finalize, getRecordCount, getRecordCountAsBigInteger, getRecordCountAsString, getState, incrementRecordCount, isClosed, isOpen, isRecordCountBigInteger, resetRecordCount, toString

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

RemoveDuplicatesReader

public RemoveDuplicatesReader(DataReader targetDataReader,
                              FieldList fields)

RemoveDuplicatesReader

public RemoveDuplicatesReader(DataReader targetDataReader)

Method Detail

interceptRecord

protected Record interceptRecord(Record record)
                          throws Throwable

Overrides:

interceptRecord in class ProxyReader

Throws:

Throwable