com.northconcepts.datapipeline.sql.postgresql

Class PostgreSqlUpsertWriter

java.lang.Object

com.northconcepts.datapipeline.core.DataObject

com.northconcepts.datapipeline.core.Endpoint

com.northconcepts.datapipeline.core.DataEndpoint

com.northconcepts.datapipeline.core.DataWriter

com.northconcepts.datapipeline.core.AbstractWriter

com.northconcepts.datapipeline.core.TextWriter

com.northconcepts.datapipeline.sql.postgresql.PostgreSqlInsertWriter

com.northconcepts.datapipeline.sql.postgresql.PostgreSqlUpsertWriter

public class PostgreSqlUpsertWriter
extends PostgreSqlInsertWriter

Generates PostgreSQL upsert statement from records.

Nested Class Summary

Nested classes/interfaces inherited from class com.northconcepts.datapipeline.core.DataEndpoint

DataEndpoint.State

Field Summary

Fields inherited from class com.northconcepts.datapipeline.sql.postgresql.PostgreSqlInsertWriter

batchedInserts, batchSize, pretty, tableName

Fields inherited from class com.northconcepts.datapipeline.core.AbstractWriter

currentRecord

Fields inherited from class com.northconcepts.datapipeline.core.DataEndpoint

lastRecord, PRODUCT, PRODUCT_VERSION, VENDOR, XML_INPUT_FACTORY_KEY

Fields inherited from class com.northconcepts.datapipeline.core.Endpoint

BUFFER_SIZE, captureElapsedTime, DEFAULT_READ_BUFFER_SIZE

Fields inherited from class com.northconcepts.datapipeline.core.DataObject

id, log, name, TIMESTAMP_FORMAT

Constructor Summary

Constructors

Constructor and Description
PostgreSqlUpsertWriter(String table, File file, boolean append, List<String> keyFieldNames)
PostgreSqlUpsertWriter(String table, File file, boolean append, String... keyFieldNames)
PostgreSqlUpsertWriter(String table, File file, List<String> keyFieldNames)
PostgreSqlUpsertWriter(String table, File file, String... keyFieldNames)

Method Summary

Modifier and Type	Method and Description
void	close() Indicates that this endpoint has finished reading or writing.
ConflictAction	getConflictAction()
List<String>	getKeyFieldNames()
PostgreSqlUpsertWriter	setBatchSize(int batchSize) Sets the number of records to chunk together in batch upserts.
PostgreSqlUpsertWriter	setConflictAction(ConflictAction conflictAction) Indicates the action to perform on conflict (default ConflictAction.UPDATE).
PostgreSqlUpsertWriter	setPretty(boolean pretty) Indicates if line breaks and indentations should be added to output upsert statement (default false). Example output (pretty = true): INSERT INTO "tableName" ("col1", "col2", "col3") VALUES (val1, val2, val3),
protected void	writeRecord(Record record)

Methods inherited from class com.northconcepts.datapipeline.sql.postgresql.PostgreSqlInsertWriter

addExceptionProperties, getBatchSize, getSqlValueLiteral, getTableName, isBatchMode, isPretty, setFieldNamesInFirstRow

Methods inherited from class com.northconcepts.datapipeline.core.TextWriter

getFile, getWriter, isAppend, isAutoCloseWriter, isFieldNamesInFirstRow, isFlushOnWrite, setAutoCloseWriter, setFlushOnWrite, write

Methods inherited from class com.northconcepts.datapipeline.core.AbstractWriter

open, writeImpl

Methods inherited from class com.northconcepts.datapipeline.core.DataWriter

available, getNestedEndpoint, getNestedWriter, getRootEndpoint, getRootWriter

Methods inherited from class com.northconcepts.datapipeline.core.DataEndpoint

decrementRecordCount, enableJmx, getLastRecord, getRecordCount, getRecordCountAsBigInteger, getRecordCountAsString, incrementRecordCount, isRecordCountBigInteger, resetRecordCount, toString

Methods inherited from class com.northconcepts.datapipeline.core.Endpoint

addElapsedtime, assertClosed, assertNotOpened, assertOpened, finalize, getClosedOn, getDescription, getElapsedTime, getElapsedTimeAsString, getOpenedOn, getOpenElapsedTime, getOpenElapsedTimeAsString, getSelfTime, getSelfTimeAsString, getState, isCaptureElapsedTime, isClosed, isOpen, setCaptureElapsedTime, setDescription

Methods inherited from class com.northconcepts.datapipeline.core.DataObject

exception, exception, exception, getId, getName, resetID

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

PostgreSqlUpsertWriter

public PostgreSqlUpsertWriter(String table,
                              File file,
                              boolean append,
                              String... keyFieldNames)

PostgreSqlUpsertWriter

public PostgreSqlUpsertWriter(String table,
                              File file,
                              boolean append,
                              List<String> keyFieldNames)

PostgreSqlUpsertWriter

public PostgreSqlUpsertWriter(String table,
                              File file,
                              String... keyFieldNames)

PostgreSqlUpsertWriter

public PostgreSqlUpsertWriter(String table,
                              File file,
                              List<String> keyFieldNames)

Method Detail

writeRecord

protected void writeRecord(Record record)
                    throws Throwable

Overrides:

writeRecord in class PostgreSqlInsertWriter

Throws:

Throwable

close

public void close()

Description copied from class: DataEndpoint

Indicates that this endpoint has finished reading or writing.

Overrides:

close in class PostgreSqlInsertWriter

setPretty

public PostgreSqlUpsertWriter setPretty(boolean pretty)

Indicates if line breaks and indentations should be added to output upsert statement (default false).

Example output (pretty = true):


INSERT INTO "tableName" ("col1", "col2", "col3")
VALUES (val1, val2, val3),

       (val1, val2, val3),

       (val1, val2, val3)

ON CONFLICT ("id") DO UPDATE SET "col1" = EXCLUDED."col1", "col2" = EXCLUDED."col2", "col3" = EXCLUDED."col3";
Example output (pretty = false):
INSERT INTO "tableName" ("col1", "col2", "col3") VALUES (val1, val2, val3), (val1, val2, val3), (val1, val2, val3) ON CONFLICT ("id") DO UPDATE SET "col1" = EXCLUDED."col1", "col2" = EXCLUDED."col2", "col3" = EXCLUDED."col3";

Overrides:

setPretty in class PostgreSqlInsertWriter

setBatchSize

public PostgreSqlUpsertWriter setBatchSize(int batchSize)

Sets the number of records to chunk together in batch upserts. Upserts are not batched if this value is 1 or less. PostgreSqlUpsertWriter defaults to non batching upserts.

Example output (batchSize = 2):
INSERT INTO "tableName" ("col1", "col2", "col3")
VALUES (val1, val2, val3),

       (val1, val2, val3),

ON CONFLICT ("id") DO UPDATE SET "col1" = EXCLUDED."col1", "col2" = EXCLUDED."col2", "col3" = EXCLUDED."col3";
Example output (batchSize = 0):
INSERT INTO "tableName" ("col1", "col2", "col3") VALUES (val1, val2, val3) ON CONFLICT ("id") DO UPDATE SET "col1" = EXCLUDED."col1", "col2" = EXCLUDED."col2", "col3" = EXCLUDED."col3";
INSERT INTO "tableName" ("col1", "col2", "col3") VALUES (val1, val2, val3) ON CONFLICT ("id") DO UPDATE SET "col1" = EXCLUDED."col1", "col2" = EXCLUDED."col2", "col3" = EXCLUDED."col3";

Overrides:

setBatchSize in class PostgreSqlInsertWriter

getKeyFieldNames

public List<String> getKeyFieldNames()

getConflictAction

public ConflictAction getConflictAction()

setConflictAction

public PostgreSqlUpsertWriter setConflictAction(ConflictAction conflictAction)

Indicates the action to perform on conflict (default ConflictAction.UPDATE).