com.northconcepts.datapipeline.twitter

Class TwitterUserLookupReader

java.lang.Object

com.northconcepts.datapipeline.core.DataObject

com.northconcepts.datapipeline.core.Endpoint

com.northconcepts.datapipeline.core.DataEndpoint

com.northconcepts.datapipeline.core.DataReader

com.northconcepts.datapipeline.core.ProxyReader

com.northconcepts.datapipeline.internal.lang.IntegrationProxyReader

com.northconcepts.datapipeline.twitter.TwitterProxyReader

com.northconcepts.datapipeline.twitter.TwitterUserLookupReader

public class TwitterUserLookupReader
extends TwitterProxyReader

Nested Class Summary

Nested classes/interfaces inherited from class com.northconcepts.datapipeline.core.DataEndpoint

DataEndpoint.State

Field Summary

Fields

Modifier and Type	Field and Description
static int	MAX_BATCH_SIZE

Fields inherited from class com.northconcepts.datapipeline.twitter.TwitterProxyReader

EOF

Fields inherited from class com.northconcepts.datapipeline.core.DataReader

fieldLineage, recordLineage

Fields inherited from class com.northconcepts.datapipeline.core.DataEndpoint

lastRecord, PRODUCT, PRODUCT_VERSION, VENDOR, XML_INPUT_FACTORY_KEY

Fields inherited from class com.northconcepts.datapipeline.core.Endpoint

BUFFER_SIZE, captureElapsedTime, DEFAULT_READ_BUFFER_SIZE

Fields inherited from class com.northconcepts.datapipeline.core.DataObject

id, log, name, TIMESTAMP_FORMAT

Constructor Summary

Constructors

Constructor and Description
TwitterUserLookupReader(String consumerKey, String consumerSecret, DataReader nestedDataReader, String sourceIdFieldName)
TwitterUserLookupReader(String consumerKey, String consumerSecret, String accessToken, String accessTokenSecret, DataReader nestedDataReader, String sourceIdFieldName)
TwitterUserLookupReader(TwitterProvider provider, DataReader nestedDataReader, String sourceIdFieldName)

Method Summary

Modifier and Type	Method and Description
DataException	addExceptionProperties(DataException exception) Adds this endpoint's current state to a DataException.
void	close() Indicates that this endpoint has finished reading or writing.
int	getBatchSize()
DataWriter	getDiscardWriter() Returns the discard sink for records where the lookup failed or null if one was not assigned.
boolean	isLookupById()
boolean	isLookupByScreenName()
boolean	isReturnOrphans() Indicates if records failing the lookup should be returned — instead of being deleted (default is false).
protected void	onUserFound(Long userId, String screenName, Record record) Called for each user found.
protected void	onUserNotFound(Long userId, String screenName, Record record) Called for each user not found.
void	open() Makes this endpoint ready for reading or writing.
protected Record	readImpl() Overridden by subclasses to read the next record from this DataReader.
TwitterUserLookupReader	setBatchSize(int batchSize)
TwitterUserLookupReader	setDiscardWriter(DataWriter writer) Assign a discard sink for records where the lookup failed.
TwitterUserLookupReader	setLookupByScreenName(boolean lookupByScreenName)
TwitterUserLookupReader	setReturnOrphans(boolean returnOrphans) Indicates if records failing the lookup should be returned — instead of being deleted (default is false).

Methods inherited from class com.northconcepts.datapipeline.twitter.TwitterProxyReader

getApiLimitPolicy, getConverter, getEntityExpansionPolicy, getProvider, getRateLimit, getRateLimit, isRateLimitExceeded, onRateLimitExceeded, reloadRateLimit, setApiLimitPolicy, setConverter, setEntityExpansionPolicy, setRateLimit, setRateLimit, setRateLimitExceeded

Methods inherited from class com.northconcepts.datapipeline.core.ProxyReader

available, getNestedReader, interceptRecord, map, map, setNestedDataReader, setNestedDataReader

Methods inherited from class com.northconcepts.datapipeline.core.DataReader

addLineage, getBufferSize, getNestedEndpoint, getRootEndpoint, getRootReader, isExhausted, isLineageSupported, isSaveLineage, peek, pop, push, read, setSaveLineage, skip

Methods inherited from class com.northconcepts.datapipeline.core.DataEndpoint

decrementRecordCount, enableJmx, getLastRecord, getRecordCount, getRecordCountAsBigInteger, getRecordCountAsString, incrementRecordCount, isRecordCountBigInteger, resetRecordCount, toString

Methods inherited from class com.northconcepts.datapipeline.core.Endpoint

addElapsedtime, assertClosed, assertNotOpened, assertOpened, finalize, getClosedOn, getDescription, getElapsedTime, getElapsedTimeAsString, getOpenedOn, getOpenElapsedTime, getOpenElapsedTimeAsString, getSelfTime, getSelfTimeAsString, getState, isCaptureElapsedTime, isClosed, isOpen, setCaptureElapsedTime, setDescription

Methods inherited from class com.northconcepts.datapipeline.core.DataObject

exception, exception, exception, getId, getName, resetID

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

MAX_BATCH_SIZE

public static final int MAX_BATCH_SIZE

See Also:

Constant Field Values

Constructor Detail

TwitterUserLookupReader

public TwitterUserLookupReader(String consumerKey,
                               String consumerSecret,
                               DataReader nestedDataReader,
                               String sourceIdFieldName)

TwitterUserLookupReader

public TwitterUserLookupReader(String consumerKey,
                               String consumerSecret,
                               String accessToken,
                               String accessTokenSecret,
                               DataReader nestedDataReader,
                               String sourceIdFieldName)

TwitterUserLookupReader

public TwitterUserLookupReader(TwitterProvider provider,
                               DataReader nestedDataReader,
                               String sourceIdFieldName)

Method Detail

isLookupByScreenName

public boolean isLookupByScreenName()

isLookupById

public boolean isLookupById()

setLookupByScreenName

public TwitterUserLookupReader setLookupByScreenName(boolean lookupByScreenName)

getDiscardWriter

public DataWriter getDiscardWriter()

Returns the discard sink for records where the lookup failed or null if one was not assigned.

setDiscardWriter

public TwitterUserLookupReader setDiscardWriter(DataWriter writer)

Assign a discard sink for records where the lookup failed.

Parameters:

writer - the discard sink

isReturnOrphans

public boolean isReturnOrphans()

Indicates if records failing the lookup should be returned — instead of being deleted (default is false).

setReturnOrphans

public TwitterUserLookupReader setReturnOrphans(boolean returnOrphans)

Indicates if records failing the lookup should be returned — instead of being deleted (default is false).

getBatchSize

public int getBatchSize()

setBatchSize

public TwitterUserLookupReader setBatchSize(int batchSize)

open

public void open()
          throws DataException

Description copied from class: DataEndpoint

Makes this endpoint ready for reading or writing.

Overrides:

open in class TwitterProxyReader

Throws:

DataException

close

public void close()
           throws DataException

Description copied from class: DataEndpoint

Indicates that this endpoint has finished reading or writing.

Overrides:

close in class ProxyReader

Throws:

DataException

readImpl

protected Record readImpl()
                   throws Throwable

Description copied from class: DataReader

Overridden by subclasses to read the next record from this DataReader. The default implementation of DataReader.read() now insures that this method will not be called again after it returns a null.

If no record is available, null will be returned.

Overrides:

readImpl in class ProxyReader

Throws:

Throwable

onUserFound

protected void onUserFound(Long userId,
                           String screenName,
                           Record record)

Called for each user found.

Parameters:

userId -

onUserNotFound

protected void onUserNotFound(Long userId,
                              String screenName,
                              Record record)

Called for each user not found.

addExceptionProperties

public DataException addExceptionProperties(DataException exception)

Description copied from class: Endpoint

Adds this endpoint's current state to a DataException. Since this method is called whenever an exception is thrown, subclasses should override it to add their specific information.

Overrides:

addExceptionProperties in class TwitterProxyReader