com.northconcepts.datapipeline.sql.mysql

Class MySqlUpsertWriter

java.lang.Object

com.northconcepts.datapipeline.core.DataObject

com.northconcepts.datapipeline.core.Endpoint

com.northconcepts.datapipeline.core.DataEndpoint

com.northconcepts.datapipeline.core.DataWriter

com.northconcepts.datapipeline.core.AbstractWriter

com.northconcepts.datapipeline.core.TextWriter

com.northconcepts.datapipeline.sql.mysql.MySqlInsertWriter

com.northconcepts.datapipeline.sql.mysql.MySqlUpsertWriter

public class MySqlUpsertWriter
extends MySqlInsertWriter

Generates MySQL upsert statement from records.

Nested Class Summary

Nested classes/interfaces inherited from class com.northconcepts.datapipeline.core.DataEndpoint

DataEndpoint.State

Field Summary

Fields inherited from class com.northconcepts.datapipeline.sql.mysql.MySqlInsertWriter

batchedInserts, batchSize, pretty, tableName

Fields inherited from class com.northconcepts.datapipeline.core.AbstractWriter

currentRecord

Fields inherited from class com.northconcepts.datapipeline.core.DataEndpoint

lastRecord, PRODUCT, PRODUCT_VERSION, VENDOR, XML_INPUT_FACTORY_KEY

Fields inherited from class com.northconcepts.datapipeline.core.Endpoint

BUFFER_SIZE, captureElapsedTime, DEFAULT_READ_BUFFER_SIZE

Fields inherited from class com.northconcepts.datapipeline.core.DataObject

id, log, name, TIMESTAMP_FORMAT

Constructor Summary

Constructors

Constructor and Description
MySqlUpsertWriter(String table, File file)
MySqlUpsertWriter(String table, File file, boolean append)

Method Summary

Modifier and Type	Method and Description
void	close() Indicates that this endpoint has finished reading or writing.
MySqlUpsertWriter	setBatchSize(int batchSize) Sets the number of records to chunk together in batch upserts.
MySqlUpsertWriter	setPretty(boolean pretty) Indicates if line breaks and indentations should be added to output upsert statement (default false). Example output (pretty = true): INSERT INTO `tableName` (`col1`, `col2`) VALUES (val1, val2),
protected void	writeRecord(Record record)

Methods inherited from class com.northconcepts.datapipeline.sql.mysql.MySqlInsertWriter

addExceptionProperties, getBatchSize, getSqlValueLiteral, getTableName, isBatchMode, isPretty, setFieldNamesInFirstRow

Methods inherited from class com.northconcepts.datapipeline.core.TextWriter

getFile, getWriter, isAppend, isAutoCloseWriter, isFieldNamesInFirstRow, isFlushOnWrite, setAutoCloseWriter, setFlushOnWrite, write

Methods inherited from class com.northconcepts.datapipeline.core.AbstractWriter

open, writeImpl

Methods inherited from class com.northconcepts.datapipeline.core.DataWriter

available, getNestedEndpoint, getNestedWriter, getRootEndpoint, getRootWriter, getWriter

Methods inherited from class com.northconcepts.datapipeline.core.DataEndpoint

decrementRecordCount, enableJmx, getLastRecord, getRecordCount, getRecordCountAsBigInteger, getRecordCountAsString, incrementRecordCount, isRecordCountBigInteger, resetRecordCount, toString

Methods inherited from class com.northconcepts.datapipeline.core.Endpoint

addElapsedtime, assertClosed, assertNotOpened, assertOpened, finalize, getClosedOn, getDescription, getElapsedTime, getElapsedTimeAsString, getOpenedOn, getOpenElapsedTime, getOpenElapsedTimeAsString, getSelfTime, getSelfTimeAsString, getState, isCaptureElapsedTime, isClosed, isOpen, setCaptureElapsedTime, setDescription

Methods inherited from class com.northconcepts.datapipeline.core.DataObject

exception, exception, exception, getId, getName, resetID

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

MySqlUpsertWriter

public MySqlUpsertWriter(String table,
                         File file,
                         boolean append)

MySqlUpsertWriter

public MySqlUpsertWriter(String table,
                         File file)

Method Detail

writeRecord

protected void writeRecord(Record record)
                    throws Throwable

Overrides:

writeRecord in class MySqlInsertWriter

Throws:

Throwable

close

public void close()

Description copied from class: DataEndpoint

Indicates that this endpoint has finished reading or writing.

Overrides:

close in class MySqlInsertWriter

setPretty

public MySqlUpsertWriter setPretty(boolean pretty)

Indicates if line breaks and indentations should be added to output upsert statement (default false).

Example output (pretty = true):
INSERT INTO `tableName` (`col1`, `col2`)
VALUES (val1, val2),

       (val1, val2),

       (val1, val2)

ON DUPLICATE KEY UPDATE `col1` = VALUES(`col1`), `col2` = VALUES(`col2`);
Example output (pretty = false):
INSERT INTO `tableName` (`col1`, `col2`) VALUES (val1, val2), (val1, val2), (val1, val2) ON DUPLICATE KEY UPDATE `col1` = VALUES(`col1`), `col2` = VALUES(`col2`);

Overrides:

setPretty in class MySqlInsertWriter

setBatchSize

public MySqlUpsertWriter setBatchSize(int batchSize)

Sets the number of records to chunk together in batch upserts. Upserts are not batched if this value is 1 or less. MySqlUpsertWriter defaults to non batching inserts.

Example output (batchSize = 2):
INSERT INTO `tableName` (`col1`, `col2`)
VALUES (val1, val2),

       (val1, val2,

       (val1, val2)

ON DUPLICATE KEY UPDATE `col1` = VALUES(`col1`), `col2` = VALUES(`col2`);
Example output (batchSize = 0):
INSERT INTO `tableName` (`col1`, `col2`) VALUES (val1, val2) ON DUPLICATE KEY UPDATE `col1` = VALUES(`col1`), `col2` = VALUES(`col2`);
INSERT INTO `tableName` (`col1`, `col2`) VALUES (val1, val2) ON DUPLICATE KEY UPDATE `col1` = VALUES(`col1`), `col2` = VALUES(`col2`);

Overrides:

setBatchSize in class MySqlInsertWriter