com.northconcepts.datapipeline.orc.input

Class OrcPipelineInput

java.lang.Object

com.northconcepts.datapipeline.foundations.core.Bean

com.northconcepts.datapipeline.foundations.core.FoundationObject

com.northconcepts.datapipeline.foundations.pipeline.PipelineObject

com.northconcepts.datapipeline.foundations.pipeline.PipelineInput

com.northconcepts.datapipeline.foundations.pipeline.input.FileSourcePipelineInput

com.northconcepts.datapipeline.orc.input.OrcPipelineInput

All Implemented Interfaces:

DataExceptionContributor, JsonSerializable, RecordSerializable, XmlSerializable, DataReaderFactory, JavaCodeGenerator, Serializable

public class OrcPipelineInput
extends FileSourcePipelineInput

See Also:

Serialized Form

Field Summary

Fields inherited from class com.northconcepts.datapipeline.foundations.core.FoundationObject

internalId, internalName, log, TIMESTAMP_FORMAT

Fields inherited from interface com.northconcepts.datapipeline.core.RecordSerializable

SERIALIZED_CLASS_NAME, TYPE

Fields inherited from interface com.northconcepts.datapipeline.core.XmlSerializable

XML_SERIALIZED_CLASS_NAME

Constructor Summary

Constructors

Constructor and Description
OrcPipelineInput()

Method Summary

Modifier and Type	Method and Description
OrcPipelineInput	addConfig(Map<String,String> config) Sets the Orc configuration parameters.
OrcPipelineInput	addConfig(String key, String value)
DataReader	createDataReader()
OrcPipelineInput	fromRecord(Record source) Loads this instance's state from a record and returns this (for fluid API call chaining).
OrcPipelineInput	fromXmlElement(Element pipelineInputElement)
void	generateJavaCode(JavaCodeBuilder javaCodeBuilder)
int	getBatchSize()
FieldList	getColumns() Indicates the columns to read from ORC file.
Map<String,String>	getConfig() Returns the Orc configuration parameters.
FileSource	getFileSource()
String	getName()
TypeDescription	getSchema() Indicates the schema used to read the file.
boolean	isLineageSupported()
boolean	isSaveLineage()
OrcPipelineInput	setBatchSize(int batchSize)
OrcPipelineInput	setColumns(FieldList columns) Indicates the columns to read from ORC file.
OrcPipelineInput	setFileSource(FileSource fileSource)
OrcPipelineInput	setSaveLineage(boolean saveLineage)
OrcPipelineInput	setSchema(String schema) Sets the schema used to read the file as a string.
OrcPipelineInput	setSchema(TypeDescription schema) Indicates the schema used to read the file.
Record	toRecord()
Element	toXmlElement(Document document)

Methods inherited from class com.northconcepts.datapipeline.foundations.pipeline.PipelineInput

getNestedPipelineInput, getRootPipelineInput

Methods inherited from class com.northconcepts.datapipeline.foundations.core.FoundationObject

addExceptionProperties, assertValid, assertValid, clone, exception, exception, exception, getInternalId, getInternalName, resetInternalId

Methods inherited from class com.northconcepts.datapipeline.foundations.core.Bean

toString

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface com.northconcepts.datapipeline.core.RecordSerializable

fromArrayValue, fromArrayValue, fromArrayValue, fromJson, fromJson, newInstanceFromRecord, newInstanceFromRecord, toArrayValue, toJson, toJson, toJson, toRecord

Methods inherited from interface com.northconcepts.datapipeline.core.JsonSerializable

newInstanceFromJson

Methods inherited from interface com.northconcepts.datapipeline.core.XmlSerializable

forEachGrandChildElement, fromXml, fromXml, getAttribute, getAttribute, getAttribute, getAttribute, getAttribute, getAttribute, getAttribute, getAttribute, getAttribute, getAttribute, getChildElement, getChildElements, newInstanceFromXml, newInstanceFromXml, setAttribute, toXml, toXml, toXml, toXml, toXml, writeXml, writeXml

Methods inherited from interface com.northconcepts.datapipeline.core.DataExceptionContributor

addExceptionProperties, contributeExceptionProperties, contributeExceptionProperties

Constructor Detail

OrcPipelineInput

public OrcPipelineInput()

Method Detail

getName

public String getName()

Overrides:

getName in class FileSourcePipelineInput

getFileSource

public FileSource getFileSource()

Specified by:

getFileSource in class FileSourcePipelineInput

setFileSource

public OrcPipelineInput setFileSource(FileSource fileSource)

Specified by:

setFileSource in class FileSourcePipelineInput

isSaveLineage

public boolean isSaveLineage()

Overrides:

isSaveLineage in class PipelineInput

setSaveLineage

public OrcPipelineInput setSaveLineage(boolean saveLineage)

Overrides:

setSaveLineage in class PipelineInput

createDataReader

public DataReader createDataReader()

generateJavaCode

public void generateJavaCode(JavaCodeBuilder javaCodeBuilder)

isLineageSupported

public boolean isLineageSupported()

Overrides:

isLineageSupported in class PipelineInput

getBatchSize

public int getBatchSize()

setBatchSize

public OrcPipelineInput setBatchSize(int batchSize)

getSchema

public TypeDescription getSchema()

Indicates the schema used to read the file.

setSchema

public OrcPipelineInput setSchema(String schema)

Sets the schema used to read the file as a string.

setSchema

public OrcPipelineInput setSchema(TypeDescription schema)

Indicates the schema used to read the file.

getColumns

public FieldList getColumns()

Indicates the columns to read from ORC file.

setColumns

public OrcPipelineInput setColumns(FieldList columns)

Indicates the columns to read from ORC file.

getConfig

public Map<String,String> getConfig()

Returns the Orc configuration parameters.

addConfig

public OrcPipelineInput addConfig(Map<String,String> config)

Sets the Orc configuration parameters.

addConfig

public OrcPipelineInput addConfig(String key,
                                  String value)

toRecord

public Record toRecord()

Specified by:

toRecord in interface RecordSerializable

Overrides:

toRecord in class PipelineInput

fromRecord

public OrcPipelineInput fromRecord(Record source)

Description copied from interface: RecordSerializable

Loads this instance's state from a record and returns this (for fluid API call chaining). For fluid API call chaining, the overridden method should change the declared return type to its class.

Specified by:

fromRecord in interface RecordSerializable

Overrides:

fromRecord in class PipelineInput

Returns:

this instance.

toXmlElement

public Element toXmlElement(Document document)

Specified by:

toXmlElement in interface XmlSerializable

Overrides:

toXmlElement in class PipelineInput

fromXmlElement

public OrcPipelineInput fromXmlElement(Element pipelineInputElement)

Specified by:

fromXmlElement in interface XmlSerializable

Overrides:

fromXmlElement in class PipelineInput