com.northconcepts.datapipeline.group

Class GroupByReader

java.lang.Object

com.northconcepts.datapipeline.core.DataObject

com.northconcepts.datapipeline.core.Endpoint

com.northconcepts.datapipeline.core.DataEndpoint

com.northconcepts.datapipeline.core.DataReader

com.northconcepts.datapipeline.core.ProxyReader

com.northconcepts.datapipeline.group.GroupByReader

public class GroupByReader
extends ProxyReader

A proxy that divides records into groups and applies summary operations to each group; similar to "group by" in SQL, but applied to streaming data.

Nested Class Summary

Nested classes/interfaces inherited from class com.northconcepts.datapipeline.core.DataEndpoint

DataEndpoint.State

Field Summary

Fields inherited from class com.northconcepts.datapipeline.core.DataReader

fieldLineage, recordLineage

Fields inherited from class com.northconcepts.datapipeline.core.DataEndpoint

lastRecord, PRODUCT, PRODUCT_VERSION, VENDOR, XML_INPUT_FACTORY_KEY

Fields inherited from class com.northconcepts.datapipeline.core.Endpoint

BUFFER_SIZE, captureElapsedTime, DEFAULT_READ_BUFFER_SIZE

Fields inherited from class com.northconcepts.datapipeline.core.DataObject

id, log, name, TIMESTAMP_FORMAT

Constructor Summary

Constructors

Constructor and Description
GroupByReader(DataReader reader, FieldList groupByFields)
GroupByReader(DataReader reader, int queueSize, FieldList groupByFields)
GroupByReader(DataReader reader, int queueSize, String... groupByFields)
GroupByReader(DataReader reader, String... groupByFields)

Method Summary

Modifier and Type	Method and Description
GroupByReader	add(GroupOperation<?> operation)
DataException	addExceptionProperties(DataException exception) Adds this endpoint's current state to a DataException.
GroupByReader	avg(String sourceFieldName, String targetFieldName)
GroupByReader	avg(String sourceFieldName, String targetFieldName, int scale, RoundingMode roundingMode)
void	close() Indicates that this endpoint has finished reading or writing.
GroupByReader	count(String targetFieldName)
GroupByReader	count(String targetFieldName, boolean excludeNulls)
GroupByReader	first(String sourceFieldName, boolean excludeNulls)
GroupByReader	first(String sourceFieldName, String targetFieldName)
GroupByReader	first(String sourceFieldName, String targetFieldName, boolean excludeNulls)
List<Window>	getClosedWindows()
CloseWindowStrategy	getCloseWindowStrategy()
CreateWindowStrategy	getCreateWindowStrategy()
FieldList	getGroupByFields()
List<Window>	getOpenedWindows()
List<GroupOperation<?>>	getOperations()
long	getWindowsClosed()
long	getWindowsCreated()
boolean	isDebug()
boolean	isExcludeNulls()
GroupByReader	last(String sourceFieldName, boolean excludeNulls)
GroupByReader	last(String sourceFieldName, String targetFieldName)
GroupByReader	last(String sourceFieldName, String targetFieldName, boolean excludeNulls)
GroupByReader	max(String sourceFieldName)
GroupByReader	max(String sourceFieldName, boolean excludeNulls)
GroupByReader	max(String sourceFieldName, String targetFieldName)
GroupByReader	max(String sourceFieldName, String targetFieldName, boolean excludeNulls)
GroupByReader	min(String sourceFieldName)
GroupByReader	min(String sourceFieldName, boolean excludeNulls)
GroupByReader	min(String sourceFieldName, String targetFieldName)
GroupByReader	min(String sourceFieldName, String targetFieldName, boolean excludeNulls)
void	open() Makes this endpoint ready for reading or writing.
protected Record	readImpl() Overridden by subclasses to read the next record from this DataReader.
GroupByReader	setCloseWindowStrategy(CloseWindowStrategy closeWindowStrategy)
GroupByReader	setCreateWindowStrategy(CreateWindowStrategy createWindowStrategy)
GroupByReader	setDebug(boolean debug)
GroupByReader	setExcludeNulls(boolean excludeNulls) Indicates if null groups should be excluded from results (defaults to false).
GroupByReader	sum(String sourceFieldName)
GroupByReader	sum(String sourceFieldName, String targetFieldName)
String	toString()

Methods inherited from class com.northconcepts.datapipeline.core.ProxyReader

available, getNestedReader, interceptRecord, map, map, setNestedDataReader, setNestedDataReader

Methods inherited from class com.northconcepts.datapipeline.core.DataReader

addLineage, getBufferSize, getNestedEndpoint, getRootEndpoint, getRootReader, isExhausted, isLineageSupported, isSaveLineage, peek, pop, push, read, setSaveLineage, skip

Methods inherited from class com.northconcepts.datapipeline.core.DataEndpoint

decrementRecordCount, enableJmx, getLastRecord, getRecordCount, getRecordCountAsBigInteger, getRecordCountAsString, incrementRecordCount, isRecordCountBigInteger, resetRecordCount

Methods inherited from class com.northconcepts.datapipeline.core.Endpoint

addElapsedtime, assertClosed, assertNotOpened, assertOpened, finalize, getClosedOn, getDescription, getElapsedTime, getElapsedTimeAsString, getOpenedOn, getOpenElapsedTime, getOpenElapsedTimeAsString, getSelfTime, getSelfTimeAsString, getState, isCaptureElapsedTime, isClosed, isOpen, setCaptureElapsedTime, setDescription

Methods inherited from class com.northconcepts.datapipeline.core.DataObject

exception, exception, exception, getId, getName, resetID

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

GroupByReader

public GroupByReader(DataReader reader,
                     String... groupByFields)

GroupByReader

public GroupByReader(DataReader reader,
                     int queueSize,
                     String... groupByFields)

GroupByReader

public GroupByReader(DataReader reader,
                     FieldList groupByFields)

GroupByReader

public GroupByReader(DataReader reader,
                     int queueSize,
                     FieldList groupByFields)

Method Detail

getGroupByFields

public FieldList getGroupByFields()

isExcludeNulls

public boolean isExcludeNulls()

setExcludeNulls

public GroupByReader setExcludeNulls(boolean excludeNulls)

Indicates if null groups should be excluded from results (defaults to false). For example, when grouping on country and city, any cases where both country and city are null will be skipped if set to true.

isDebug

public boolean isDebug()

setDebug

public GroupByReader setDebug(boolean debug)

getOperations

public List<GroupOperation<?>> getOperations()

add

public GroupByReader add(GroupOperation<?> operation)

count

public GroupByReader count(String targetFieldName)

count

public GroupByReader count(String targetFieldName,
                           boolean excludeNulls)

sum

public GroupByReader sum(String sourceFieldName,
                         String targetFieldName)

sum

public GroupByReader sum(String sourceFieldName)

avg

public GroupByReader avg(String sourceFieldName,
                         String targetFieldName)

avg

public GroupByReader avg(String sourceFieldName,
                         String targetFieldName,
                         int scale,
                         RoundingMode roundingMode)

first

public GroupByReader first(String sourceFieldName,
                           String targetFieldName,
                           boolean excludeNulls)

first

public GroupByReader first(String sourceFieldName,
                           String targetFieldName)

first

public GroupByReader first(String sourceFieldName,
                           boolean excludeNulls)

last

public GroupByReader last(String sourceFieldName,
                          String targetFieldName,
                          boolean excludeNulls)

last

public GroupByReader last(String sourceFieldName,
                          String targetFieldName)

last

public GroupByReader last(String sourceFieldName,
                          boolean excludeNulls)

max

public GroupByReader max(String sourceFieldName,
                         String targetFieldName,
                         boolean excludeNulls)

max

public GroupByReader max(String sourceFieldName,
                         String targetFieldName)

max

public GroupByReader max(String sourceFieldName,
                         boolean excludeNulls)

max

public GroupByReader max(String sourceFieldName)

min

public GroupByReader min(String sourceFieldName,
                         String targetFieldName,
                         boolean excludeNulls)

min

public GroupByReader min(String sourceFieldName,
                         String targetFieldName)

min

public GroupByReader min(String sourceFieldName,
                         boolean excludeNulls)

min

public GroupByReader min(String sourceFieldName)

getCloseWindowStrategy

public CloseWindowStrategy getCloseWindowStrategy()

setCloseWindowStrategy

public GroupByReader setCloseWindowStrategy(CloseWindowStrategy closeWindowStrategy)

getCreateWindowStrategy

public CreateWindowStrategy getCreateWindowStrategy()

setCreateWindowStrategy

public GroupByReader setCreateWindowStrategy(CreateWindowStrategy createWindowStrategy)

getOpenedWindows

public List<Window> getOpenedWindows()

getClosedWindows

public List<Window> getClosedWindows()

getWindowsCreated

public long getWindowsCreated()

getWindowsClosed

public long getWindowsClosed()

open

public void open()

Description copied from class: DataEndpoint

Makes this endpoint ready for reading or writing.

Overrides:

open in class ProxyReader

close

public void close()
           throws DataException

Description copied from class: DataEndpoint

Indicates that this endpoint has finished reading or writing.

Overrides:

close in class ProxyReader

Throws:

DataException

readImpl

protected Record readImpl()
                   throws Throwable

Description copied from class: DataReader

Overridden by subclasses to read the next record from this DataReader. The default implementation of DataReader.read() now insures that this method will not be called again after it returns a null.

If no record is available, null will be returned.

Overrides:

readImpl in class ProxyReader

Throws:

Throwable

addExceptionProperties

public DataException addExceptionProperties(DataException exception)

Description copied from class: Endpoint

Adds this endpoint's current state to a DataException. Since this method is called whenever an exception is thrown, subclasses should override it to add their specific information.

Overrides:

addExceptionProperties in class ProxyReader

toString

public String toString()

Overrides:

toString in class DataEndpoint