com.northconcepts.datapipeline.foundations.pipeline.dataset

Class MemoryDataset

java.lang.Object

com.northconcepts.datapipeline.foundations.core.Bean

com.northconcepts.datapipeline.foundations.core.FoundationObject

com.northconcepts.datapipeline.foundations.pipeline.dataset.Dataset

com.northconcepts.datapipeline.foundations.pipeline.dataset.MemoryDataset

All Implemented Interfaces:

JsonSerializable, RecordSerializable, XmlSerializable, Serializable

public class MemoryDataset
extends Dataset

Caches the dataset's records in RAM.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class com.northconcepts.datapipeline.foundations.pipeline.dataset.Dataset

Dataset.ColumnsDataReader

Field Summary

Fields inherited from class com.northconcepts.datapipeline.foundations.core.FoundationObject

internalId, internalName, log, TIMESTAMP_FORMAT

Fields inherited from interface com.northconcepts.datapipeline.core.RecordSerializable

SERIALIZED_CLASS_NAME, TYPE

Fields inherited from interface com.northconcepts.datapipeline.core.XmlSerializable

XML_SERIALIZED_CLASS_NAME

Constructor Summary

Constructors

Constructor and Description
MemoryDataset(AbstractPipeline pipeline)

Method Summary

Modifier and Type	Method and Description
protected void	beforeLoad(Integer maxRecords)
void	close()
protected DataWriter	createDataWriter() Writes records to this dataset's cache after clearing it.
Column	getColumn(int index)
Column	getColumn(String name)
long	getColumnCount()
List<String>	getColumnNames()
List<Column>	getColumns()
protected Column	getOrCreateColumn(String name, int index)
Record	getRecord(long index)
long	getRecordCount()

Methods inherited from class com.northconcepts.datapipeline.foundations.pipeline.dataset.Dataset

addField, cancelLoad, createColumnsDataReader, createDataReader, createDataReader, finalize, fromRecord, getColumnStatsReaderThreads, getJob, getMaxColumnStatsRecords, getPipeline, getRecordList, isInferStringTypes, load, load, load, setColumnStatsReaderThreads, setInferStringTypes, setMaxColumnStatsRecords, toRecord, updateColumns, waitForColumnStatsToLoad, waitForColumnStatsToLoad, waitForRecordsToLoad, waitForRecordsToLoad

Methods inherited from class com.northconcepts.datapipeline.foundations.core.FoundationObject

addExceptionProperties, assertValid, assertValid, clone, exception, exception, exception, getInternalId, getInternalName, resetInternalId

Methods inherited from class com.northconcepts.datapipeline.foundations.core.Bean

toString

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface com.northconcepts.datapipeline.core.RecordSerializable

fromArrayValue, fromArrayValue, fromArrayValue, fromJson, fromJson, newInstanceFromRecord, newInstanceFromRecord, toArrayValue, toJson, toJson, toJson, toRecord

Methods inherited from interface com.northconcepts.datapipeline.core.JsonSerializable

newInstanceFromJson

Methods inherited from interface com.northconcepts.datapipeline.core.XmlSerializable

fromXml, fromXml, fromXmlElement, getAttribute, getAttribute, getAttribute, getAttribute, getAttribute, getAttribute, getAttribute, getAttribute, getAttribute, getAttribute, getChildElement, getChildElements, newInstanceFromXml, newInstanceFromXml, setAttribute, toXml, toXml, toXml, toXml, toXml, toXmlElement, writeXml, writeXml

Constructor Detail

MemoryDataset

public MemoryDataset(AbstractPipeline pipeline)

Method Detail

close

public void close()

Overrides:

close in class Dataset

getRecordCount

public long getRecordCount()

Specified by:

getRecordCount in class Dataset

getRecord

public Record getRecord(long index)

Specified by:

getRecord in class Dataset

getColumnCount

public long getColumnCount()

Specified by:

getColumnCount in class Dataset

getColumnNames

public List<String> getColumnNames()

Specified by:

getColumnNames in class Dataset

getColumn

public Column getColumn(int index)

Specified by:

getColumn in class Dataset

getColumn

public Column getColumn(String name)

Specified by:

getColumn in class Dataset

getOrCreateColumn

protected Column getOrCreateColumn(String name,
                                   int index)

Specified by:

getOrCreateColumn in class Dataset

getColumns

public List<Column> getColumns()

Specified by:

getColumns in class Dataset

beforeLoad

protected void beforeLoad(Integer maxRecords)

Specified by:

beforeLoad in class Dataset

createDataWriter

protected DataWriter createDataWriter()

Description copied from class: Dataset

Writes records to this dataset's cache after clearing it.

Specified by:

createDataWriter in class Dataset