OrcDataWriter (Data Pipeline JavaDoc)

java.lang.Object
- com.northconcepts.datapipeline.core.DataObject
- - com.northconcepts.datapipeline.core.Endpoint
  - - com.northconcepts.datapipeline.core.DataEndpoint
    - - com.northconcepts.datapipeline.core.DataWriter
      - com.northconcepts.datapipeline.internal.lang.IntegrationWriter
        
        com.northconcepts.datapipeline.orc.OrcDataWriter

```
public class OrcDataWriter
extends IntegrationWriter
```
Writes records to Apache ORC columnar files. See Apache ORC columnar storage.

Nested Class Summary
- Nested classes/interfaces inherited from class com.northconcepts.datapipeline.core.DataEndpoint
  DataEndpoint.State

Field Summary
- Fields inherited from class com.northconcepts.datapipeline.core.DataEndpoint
  lastRecord, PRODUCT, PRODUCT_VERSION, VENDOR, XML_INPUT_FACTORY_KEY
- Fields inherited from class com.northconcepts.datapipeline.core.Endpoint
  BUFFER_SIZE, captureElapsedTime, DEFAULT_READ_BUFFER_SIZE
- Fields inherited from class com.northconcepts.datapipeline.core.DataObject
  id, log, name, TIMESTAMP_FORMAT

Constructor Summary

Constructors
Constructor and Description

OrcDataWriter(File file)
Write ORC data to a file.

OrcDataWriter(Path path)
Write ORC data to a path like S3, Hadoop etc.

Constructors
Constructor and Description
`OrcDataWriter(File file)` Write ORC data to a file.
`OrcDataWriter(Path path)` Write ORC data to a path like S3, Hadoop etc.

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`DataException`	`addExceptionProperties(DataException exception)` Adds this endpoint's current state to a `DataException`.
`void`	`close()` Indicates that this endpoint has finished reading or writing.
`int`	`getBatchSize()` Indicates the maximum number of records to buffer when writing ORC data (default 1024).
`int`	`getBigDecimalPrecision()` Returns default precision value for BigDecimal type of values.
`int`	`getBigDecimalScale()` Indicates the scale used when writing BigDecimal values (default 10).
`CompressionKind`	`getCompressionKind()` Indicates the compression used for writing (default NONE).
`Configuration`	`getConfig()` Returns the Orc configuration parameters.
`Path`	`getPath()` Returns the `Path` of the ORC file being written.
`RoundingMode`	`getRoundingMode()` Indicates the rounding algorithm used for all BigDecimal values (default is `RoundingMode.HALF_UP`).
`RowBatchVersion`	`getRowBatchVersion()` Returns `RowBatchVersion` to store ORC data.
`TypeDescription`	`getSchema()` Indicates the schema used to write the file.
`boolean`	`isOverwrite()` Overwrites the file if true, otherwise, throws an exception if the file already exists.
`boolean`	`isUseTimestampInstant()` Indicates if `Instant`s should be used to store datetime values.
`boolean`	`isUTC()` Indicates if datetime values are adjusted to UTC.
`boolean`	`isWriteUuidAsString()` Indicates whether UUID values are written as strings or bytes.
`void`	`open()` Makes this endpoint ready for reading or writing.
`OrcDataWriter`	`setBatchSize(int batchSize)` Indicates the maximum number of records to buffer when writing ORC data (default 1024).
`OrcDataWriter`	`setBigDecimalPrecision(int bigDecimalPrecision)` Sets default precision value for BigDecimal type of values.
`OrcDataWriter`	`setBigDecimalScale(int bigDecimalScale)` Indicates the scale used when writing BigDecimal values (default 10).
`OrcDataWriter`	`setCompressionKind(CompressionKind compressionKind)` Indicates the compression used for writing (default NONE).
`OrcDataWriter`	`setConfig(Configuration config)` Sets the Orc configuration parameters.
`OrcDataWriter`	`setOverwrite(boolean overwrite)` Overwrites the file if true, otherwise, throws an exception if the file already exists.
`OrcDataWriter`	`setRoundingMode(RoundingMode roundingMode)` Indicates the rounding algorithm used for all BigDecimal values (default is `RoundingMode.HALF_UP`).
`OrcDataWriter`	`setRowBatchVersion(RowBatchVersion rowBatchVersion)` Sets `RowBatchVersion` to store ORC data.
`OrcDataWriter`	`setSchema(TypeDescription schema)` Indicates the schema used to write the file.
`OrcDataWriter`	`setUseTimestampInstant(boolean useTimestampInstant)` Indicates if `Instant`s should be used to store datetime values.
`OrcDataWriter`	`setUTC(boolean isUTC)` Indicates if datetime values are adjusted to UTC.
`OrcDataWriter`	`setWriteUuidAsString(boolean writeUuidAsString)` Indicates whether UUID values are written as strings or bytes.
`protected void`	`writeImpl(Record record)` Overridden by subclasses to write the specified record to this `DataWriter`.

Methods inherited from class com.northconcepts.datapipeline.core.DataWriter
available, getNestedEndpoint, getNestedWriter, getRootEndpoint, getRootWriter, write

Methods inherited from class com.northconcepts.datapipeline.core.DataEndpoint
decrementRecordCount, enableJmx, getLastRecord, getRecordCount, getRecordCountAsBigInteger, getRecordCountAsString, incrementRecordCount, isRecordCountBigInteger, resetRecordCount, toString

Methods inherited from class com.northconcepts.datapipeline.core.Endpoint
addElapsedtime, assertClosed, assertNotOpened, assertOpened, finalize, getClosedOn, getDescription, getElapsedTime, getElapsedTimeAsString, getOpenedOn, getOpenElapsedTime, getOpenElapsedTimeAsString, getSelfTime, getSelfTimeAsString, getState, isCaptureElapsedTime, isClosed, isOpen, setCaptureElapsedTime, setDescription

Methods inherited from class com.northconcepts.datapipeline.core.DataObject
exception, exception, exception, getId, getName, resetID

Methods inherited from class java.lang.Object
clone, equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Constructor Detail
  - OrcDataWriter
```
public OrcDataWriter(File file)
```
    Write ORC data to a file.
  - OrcDataWriter
```
public OrcDataWriter(Path path)
```
    Write ORC data to a path like S3, Hadoop etc.
- Method Detail
  - writeImpl
```
protected void writeImpl(Record record)
                  throws Throwable
```
    Description copied from class: DataWriter
    
    Overridden by subclasses to write the specified record to this DataWriter.
    
    Specified by:
    
    writeImpl in class DataWriter
    
    Throws:
    
    Throwable
  - open
```
public void open()
          throws DataException
```
    Description copied from class: DataEndpoint
    
    Makes this endpoint ready for reading or writing.
    
    Overrides:
    
    open in class IntegrationWriter
    
    Throws:
    
    DataException
  - close
```
public void close()
           throws DataException
```
    Description copied from class: DataEndpoint
    
    Indicates that this endpoint has finished reading or writing.
    
    Overrides:
    
    close in class DataEndpoint
    
    Throws:
    
    DataException
  - getSchema
```
public TypeDescription getSchema()
```
    Indicates the schema used to write the file.
  - setSchema
```
public OrcDataWriter setSchema(TypeDescription schema)
```
    Indicates the schema used to write the file.
  - getConfig
```
public Configuration getConfig()
```
    Returns the Orc configuration parameters.
  - setConfig
```
public OrcDataWriter setConfig(Configuration config)
```
    Sets the Orc configuration parameters.
  - getBatchSize
```
public int getBatchSize()
```
    Indicates the maximum number of records to buffer when writing ORC data (default 1024).
  - setBatchSize
```
public OrcDataWriter setBatchSize(int batchSize)
```
    Indicates the maximum number of records to buffer when writing ORC data (default 1024).
  - isUseTimestampInstant
```
public boolean isUseTimestampInstant()
```
    Indicates if Instants should be used to store datetime values.
  - setUseTimestampInstant
```
public OrcDataWriter setUseTimestampInstant(boolean useTimestampInstant)
```
    Indicates if Instants should be used to store datetime values.
  - isOverwrite
```
public boolean isOverwrite()
```
    Overwrites the file if true, otherwise, throws an exception if the file already exists.
  - setOverwrite
```
public OrcDataWriter setOverwrite(boolean overwrite)
```
    Overwrites the file if true, otherwise, throws an exception if the file already exists.
  - addExceptionProperties
```
public DataException addExceptionProperties(DataException exception)
```
    Description copied from class: Endpoint
    
    Adds this endpoint's current state to a DataException. Since this method is called whenever an exception is thrown, subclasses should override it to add their specific information.
    
    Overrides:
    
    addExceptionProperties in class DataWriter
  - isUTC
```
public boolean isUTC()
```
    Indicates if datetime values are adjusted to UTC.
  - setUTC
```
public OrcDataWriter setUTC(boolean isUTC)
```
    Indicates if datetime values are adjusted to UTC.
  - isWriteUuidAsString
```
public boolean isWriteUuidAsString()
```
    Indicates whether UUID values are written as strings or bytes. If set to false, UUID values will be written as bytes.
    
    Default value is true.
  - setWriteUuidAsString
```
public OrcDataWriter setWriteUuidAsString(boolean writeUuidAsString)
```
    Indicates whether UUID values are written as strings or bytes. If set to false, UUID values will be written as bytes.
    
    Default value is true.
  - getRowBatchVersion
```
public RowBatchVersion getRowBatchVersion()
```
    Returns RowBatchVersion to store ORC data. precision > TypeDescription.MAX_DECIMAL64_PRECISION (18) then DecimalColumnVector is used instead of Decimal64ColumnVector.
  - setRowBatchVersion
```
public OrcDataWriter setRowBatchVersion(RowBatchVersion rowBatchVersion)
```
    Sets RowBatchVersion to store ORC data. If BigDecimal value has precision > TypeDescription.MAX_DECIMAL64_PRECISION (18) then DecimalColumnVector is used instead of Decimal64ColumnVector.
  - getBigDecimalPrecision
```
public int getBigDecimalPrecision()
```
    Returns default precision value for BigDecimal type of values. Default value is HiveDecimal.MAX_PRECISION (38).
  - setBigDecimalPrecision
```
public OrcDataWriter setBigDecimalPrecision(int bigDecimalPrecision)
```
    Sets default precision value for BigDecimal type of values. Default value is HiveDecimal.MAX_PRECISION (38).
  - getBigDecimalScale
```
public int getBigDecimalScale()
```
    Indicates the scale used when writing BigDecimal values (default 10).
  - setBigDecimalScale
```
public OrcDataWriter setBigDecimalScale(int bigDecimalScale)
```
    Indicates the scale used when writing BigDecimal values (default 10).
  - getRoundingMode
```
public RoundingMode getRoundingMode()
```
    Indicates the rounding algorithm used for all BigDecimal values (default is RoundingMode.HALF_UP).
  - setRoundingMode
```
public OrcDataWriter setRoundingMode(RoundingMode roundingMode)
```
    Indicates the rounding algorithm used for all BigDecimal values (default is RoundingMode.HALF_UP).
  - getCompressionKind
```
public CompressionKind getCompressionKind()
```
    Indicates the compression used for writing (default NONE).
  - setCompressionKind
```
public OrcDataWriter setCompressionKind(CompressionKind compressionKind)
```
    Indicates the compression used for writing (default NONE).
  - getPath
```
public Path getPath()
```
    Returns the Path of the ORC file being written.

Class OrcDataWriter

Nested Class Summary

Nested classes/interfaces inherited from class com.northconcepts.datapipeline.core.DataEndpoint

Field Summary

Fields inherited from class com.northconcepts.datapipeline.core.DataEndpoint

Fields inherited from class com.northconcepts.datapipeline.core.Endpoint

Fields inherited from class com.northconcepts.datapipeline.core.DataObject

Constructor Summary

Method Summary

Methods inherited from class com.northconcepts.datapipeline.core.DataWriter

Methods inherited from class com.northconcepts.datapipeline.core.DataEndpoint

Methods inherited from class com.northconcepts.datapipeline.core.Endpoint

Methods inherited from class com.northconcepts.datapipeline.core.DataObject

Methods inherited from class java.lang.Object

Constructor Detail

OrcDataWriter

OrcDataWriter

Method Detail

writeImpl

open

close

getSchema

setSchema

getConfig

setConfig

getBatchSize

setBatchSize

isUseTimestampInstant

setUseTimestampInstant

isOverwrite

setOverwrite

addExceptionProperties

isUTC

setUTC

isWriteUuidAsString

setWriteUuidAsString

getRowBatchVersion

setRowBatchVersion

getBigDecimalPrecision

setBigDecimalPrecision

getBigDecimalScale

setBigDecimalScale

getRoundingMode

setRoundingMode

getCompressionKind

setCompressionKind

getPath