com.northconcepts.datapipeline.twitter2

Class AbstractTwitterReader

java.lang.Object

com.northconcepts.datapipeline.core.DataObject

com.northconcepts.datapipeline.core.Endpoint

com.northconcepts.datapipeline.core.DataEndpoint

com.northconcepts.datapipeline.core.DataReader

com.northconcepts.datapipeline.internal.lang.IntegrationReader

com.northconcepts.datapipeline.twitter2.AbstractTwitterReader

Direct Known Subclasses:

TwitterFilterStreamReader, TwitterFollowerReader, TwitterFollowingReader, TwitterSearchReader, TwitterTimelineReader

public abstract class AbstractTwitterReader
extends IntegrationReader

Nested Class Summary

Nested classes/interfaces inherited from class com.northconcepts.datapipeline.core.DataEndpoint

DataEndpoint.State

Field Summary

Fields

Modifier and Type	Field and Description
protected TwitterV2Client	client
protected static int	DEFAULT_RETRY_AFTER_SECONDS
protected TwitterCredentials	twitterCredentials

Fields inherited from class com.northconcepts.datapipeline.core.DataReader

fieldLineage, recordLineage

Fields inherited from class com.northconcepts.datapipeline.core.DataEndpoint

lastRecord, PRODUCT, PRODUCT_VERSION, VENDOR, XML_INPUT_FACTORY_KEY

Fields inherited from class com.northconcepts.datapipeline.core.Endpoint

BUFFER_SIZE, captureElapsedTime, DEFAULT_READ_BUFFER_SIZE

Fields inherited from class com.northconcepts.datapipeline.core.DataObject

id, log, name, TIMESTAMP_FORMAT

Constructor Summary

Constructors

Constructor and Description
AbstractTwitterReader(TwitterCredentials twitterCredentials)

Method Summary

Modifier and Type	Method and Description
DataException	addExceptionProperties(DataException exception) Adds this endpoint's current state to a DataException.
int	getMaxResults() Indicates the maximum number of records that will be obtained by the endpoint.
protected String	getNextToken()
protected Record	readImpl() Overridden by subclasses to read the next record from this DataReader.
protected boolean	readNextBatch()
protected abstract Response	search()
protected DataException	setException(DataException exception)
AbstractTwitterReader	setMaxResults(int maxResults) Indicates the maximum number of records that will be obtained by the endpoint.

Methods inherited from class com.northconcepts.datapipeline.internal.lang.IntegrationReader

open

Methods inherited from class com.northconcepts.datapipeline.core.DataReader

addLineage, available, getBufferSize, getNestedEndpoint, getNestedReader, getReader, getRootEndpoint, getRootReader, isExhausted, isLineageSupported, isSaveLineage, peek, pop, push, read, setSaveLineage, skip

Methods inherited from class com.northconcepts.datapipeline.core.DataEndpoint

close, decrementRecordCount, enableJmx, getLastRecord, getRecordCount, getRecordCountAsBigInteger, getRecordCountAsString, incrementRecordCount, isRecordCountBigInteger, resetRecordCount, toString

Methods inherited from class com.northconcepts.datapipeline.core.Endpoint

addElapsedtime, assertClosed, assertNotOpened, assertOpened, finalize, getClosedOn, getDescription, getElapsedTime, getElapsedTimeAsString, getOpenedOn, getOpenElapsedTime, getOpenElapsedTimeAsString, getSelfTime, getSelfTimeAsString, getState, isCaptureElapsedTime, isClosed, isOpen, setCaptureElapsedTime, setDescription

Methods inherited from class com.northconcepts.datapipeline.core.DataObject

exception, exception, exception, getId, getName, resetID

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

DEFAULT_RETRY_AFTER_SECONDS

protected static final int DEFAULT_RETRY_AFTER_SECONDS

See Also:

Constant Field Values

client

protected final TwitterV2Client client

twitterCredentials

protected final TwitterCredentials twitterCredentials

Constructor Detail

AbstractTwitterReader

public AbstractTwitterReader(TwitterCredentials twitterCredentials)

Method Detail

search

protected abstract Response search()

getMaxResults

public int getMaxResults()

Indicates the maximum number of records that will be obtained by the endpoint.

setMaxResults

public AbstractTwitterReader setMaxResults(int maxResults)

Indicates the maximum number of records that will be obtained by the endpoint.

getNextToken

protected String getNextToken()

readImpl

protected Record readImpl()
                   throws Throwable

Description copied from class: DataReader

Overridden by subclasses to read the next record from this DataReader. The default implementation of DataReader.read() now insures that this method will not be called again after it returns a null.

If no record is available, null will be returned.

Specified by:

readImpl in class DataReader

Throws:

Throwable

readNextBatch

protected boolean readNextBatch()
                         throws Throwable

Throws:

Throwable

setException

protected DataException setException(DataException exception)

addExceptionProperties

public DataException addExceptionProperties(DataException exception)

Description copied from class: Endpoint

Adds this endpoint's current state to a DataException. Since this method is called whenever an exception is thrown, subclasses should override it to add their specific information.

Overrides:

addExceptionProperties in class DataReader