com.northconcepts.datapipeline.twitter

Class TwitterPageableReader<E,C extends TwitterResponse & CursorSupport>

java.lang.Object

com.northconcepts.datapipeline.core.DataObject

com.northconcepts.datapipeline.core.Endpoint

com.northconcepts.datapipeline.core.DataEndpoint

com.northconcepts.datapipeline.core.DataReader

com.northconcepts.datapipeline.internal.lang.IntegrationReader

com.northconcepts.datapipeline.twitter.TwitterReader

com.northconcepts.datapipeline.twitter.TwitterPageableReader<E,C>

Direct Known Subclasses:

TwitterFollowerIDsReader, TwitterFollowerListReader, TwitterFollowingIDsReader, TwitterFollowingListReader, TwitterListMemberReader, TwitterListOwnershipReader

public abstract class TwitterPageableReader<E,C extends TwitterResponse & CursorSupport>
extends TwitterReader

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	TwitterPageableReader.Direction

Nested classes/interfaces inherited from class com.northconcepts.datapipeline.core.DataEndpoint

DataEndpoint.State

Field Summary

Fields

Modifier and Type	Field and Description
static int	FIRST_PAGE

Fields inherited from class com.northconcepts.datapipeline.twitter.TwitterReader

EOF

Fields inherited from class com.northconcepts.datapipeline.core.DataReader

fieldLineage, recordLineage

Fields inherited from class com.northconcepts.datapipeline.core.DataEndpoint

lastRecord, PRODUCT, PRODUCT_VERSION, VENDOR, XML_INPUT_FACTORY_KEY

Fields inherited from class com.northconcepts.datapipeline.core.Endpoint

BUFFER_SIZE, captureElapsedTime, DEFAULT_READ_BUFFER_SIZE

Fields inherited from class com.northconcepts.datapipeline.core.DataObject

id, log, name, TIMESTAMP_FORMAT

Constructor Summary

Constructors

Constructor and Description
TwitterPageableReader(String rateLimitResource, String rateLimitKey, String consumerKey, String consumerSecret, int pageSize)
TwitterPageableReader(String rateLimitResource, String rateLimitKey, String consumerKey, String consumerSecret, String accessToken, String accessTokenSecret, int pageSize)
TwitterPageableReader(String rateLimitResource, String rateLimitKey, TwitterProvider provider, int pageSize)

Method Summary

Modifier and Type	Method and Description
DataException	addExceptionProperties(DataException exception) Adds this endpoint's current state to a DataException.
protected abstract void	addPageToBuffer(C list, LinkedList<E> buffer)
protected abstract void	copyToRecord(E element, Record record, IEntityExpansionPolicy entityExpansionPolicy)
TwitterPageableReader.Direction	getDirection()
long	getPage()
protected abstract C	getPageImpl(long page, int pageSize)
int	getPageSize()
long	getStartingOffset()
void	open() Makes this endpoint ready for reading or writing.
protected Record	readImpl() Overridden by subclasses to read the next record from this DataReader.
TwitterPageableReader<E,C>	setApiLimitPolicy(IApiLimitPolicy apiLimitPolicy)
TwitterPageableReader<E,C>	setDirection(TwitterPageableReader.Direction direction)
TwitterPageableReader<E,C>	setStartingOffset(long startingOffset)

Methods inherited from class com.northconcepts.datapipeline.twitter.TwitterReader

getApiLimitPolicy, getConverter, getEntityExpansionPolicy, getProvider, getRateLimit, getRateLimit, isRateLimitExceeded, onRateLimitExceeded, reloadRateLimit, setConverter, setEntityExpansionPolicy, setRateLimit, setRateLimit, setRateLimitExceeded

Methods inherited from class com.northconcepts.datapipeline.core.DataReader

addLineage, available, getBufferSize, getNestedEndpoint, getNestedReader, getReader, getRootEndpoint, getRootReader, isExhausted, isLineageSupported, isSaveLineage, peek, pop, push, read, setSaveLineage, skip

Methods inherited from class com.northconcepts.datapipeline.core.DataEndpoint

close, decrementRecordCount, enableJmx, getLastRecord, getRecordCount, getRecordCountAsBigInteger, getRecordCountAsString, incrementRecordCount, isRecordCountBigInteger, resetRecordCount, toString

Methods inherited from class com.northconcepts.datapipeline.core.Endpoint

addElapsedtime, assertClosed, assertNotOpened, assertOpened, finalize, getClosedOn, getDescription, getElapsedTime, getElapsedTimeAsString, getOpenedOn, getOpenElapsedTime, getOpenElapsedTimeAsString, getSelfTime, getSelfTimeAsString, getState, isCaptureElapsedTime, isClosed, isOpen, setCaptureElapsedTime, setDescription

Methods inherited from class com.northconcepts.datapipeline.core.DataObject

exception, exception, exception, getId, getName, resetID

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

FIRST_PAGE

public static final int FIRST_PAGE

See Also:

Constant Field Values

Constructor Detail

TwitterPageableReader

public TwitterPageableReader(String rateLimitResource,
                             String rateLimitKey,
                             TwitterProvider provider,
                             int pageSize)

TwitterPageableReader

public TwitterPageableReader(String rateLimitResource,
                             String rateLimitKey,
                             String consumerKey,
                             String consumerSecret,
                             int pageSize)

TwitterPageableReader

public TwitterPageableReader(String rateLimitResource,
                             String rateLimitKey,
                             String consumerKey,
                             String consumerSecret,
                             String accessToken,
                             String accessTokenSecret,
                             int pageSize)

Method Detail

getPageSize

public int getPageSize()

getStartingOffset

public long getStartingOffset()

setStartingOffset

public TwitterPageableReader<E,C> setStartingOffset(long startingOffset)

getPage

public long getPage()

getDirection

public TwitterPageableReader.Direction getDirection()

setDirection

public TwitterPageableReader<E,C> setDirection(TwitterPageableReader.Direction direction)

setApiLimitPolicy

public TwitterPageableReader<E,C> setApiLimitPolicy(IApiLimitPolicy apiLimitPolicy)

Overrides:

setApiLimitPolicy in class TwitterReader

open

public void open()
          throws DataException

Description copied from class: DataEndpoint

Makes this endpoint ready for reading or writing.

Overrides:

open in class TwitterReader

Throws:

DataException

getPageImpl

protected abstract C getPageImpl(long page,
                                 int pageSize)
                          throws Throwable

Throws:

Throwable

addPageToBuffer

protected abstract void addPageToBuffer(C list,
                                        LinkedList<E> buffer)
                                 throws Throwable

Throws:

Throwable

copyToRecord

protected abstract void copyToRecord(E element,
                                     Record record,
                                     IEntityExpansionPolicy entityExpansionPolicy)
                              throws Throwable

Throws:

Throwable

readImpl

protected Record readImpl()
                   throws Throwable

Description copied from class: DataReader

Overridden by subclasses to read the next record from this DataReader. The default implementation of DataReader.read() now insures that this method will not be called again after it returns a null.

If no record is available, null will be returned.

Specified by:

readImpl in class DataReader

Throws:

Throwable

addExceptionProperties

public DataException addExceptionProperties(DataException exception)

Description copied from class: Endpoint

Adds this endpoint's current state to a DataException. Since this method is called whenever an exception is thrown, subclasses should override it to add their specific information.

Overrides:

addExceptionProperties in class TwitterReader