com.northconcepts.datapipeline.jdbc

Class JdbcUpsertWriter

java.lang.Object

com.northconcepts.datapipeline.core.DataObject

com.northconcepts.datapipeline.core.Endpoint

com.northconcepts.datapipeline.core.DataEndpoint

com.northconcepts.datapipeline.core.DataWriter

com.northconcepts.datapipeline.jdbc.JdbcUpsertWriter

public class JdbcUpsertWriter
extends DataWriter

Writes records to a database table using the upsert idiom (attempt to insert, but update if duplicate key exists). The default strategy used is GenericUpsert, however several other are provided: MergeUpsert, VariableFieldsUpsert, MySqlUpsert.

Nested Class Summary

Nested classes/interfaces inherited from class com.northconcepts.datapipeline.core.DataEndpoint

DataEndpoint.State

Field Summary

Fields

Modifier and Type	Field and Description
protected Record	currentRecord

Fields inherited from class com.northconcepts.datapipeline.core.DataEndpoint

lastRecord, PRODUCT, PRODUCT_VERSION, VENDOR, XML_INPUT_FACTORY_KEY

Fields inherited from class com.northconcepts.datapipeline.core.Endpoint

BUFFER_SIZE, captureElapsedTime, DEFAULT_READ_BUFFER_SIZE

Fields inherited from class com.northconcepts.datapipeline.core.DataObject

id, log, name, TIMESTAMP_FORMAT

Constructor Summary

Constructors

Constructor and Description
JdbcUpsertWriter(Connection connection, String tableName, IUpsert upsert, String... keyFieldNames)
JdbcUpsertWriter(Connection connection, String tableName, String... keyFieldNames)
JdbcUpsertWriter(JdbcConnectionFactory factory, String tableName, IUpsert upsert, String... keyFieldNames)
JdbcUpsertWriter(JdbcConnectionFactory factory, String tableName, String... keyFieldNames)

Method Summary

Modifier and Type	Method and Description
DataException	addExceptionProperties(DataException exception) Adds this endpoint's current state to a DataException.
void	close() Indicates that this endpoint has finished reading or writing.
boolean	getAutoCloseConnection()
int	getBatchSize()
Connection	getConnection()
Integer	getJdbcType(Class<?> type) The Types constant (or a database specific value) assigned to the specified Java class type using setJdbcType(Class, int) or null if none was set.
Integer	getJdbcType(String fieldName) The Types constant (or a database specific value) assigned to the specified field using setJdbcType(String, int) or null if none was set.
FieldList	getKeyFields()
FieldList	getNonUpdateFields() Returns the list of fields that must not be updated if the recored already exists (default null).
String	getTableName()
IUpsert	getUpsert()
boolean	isBatchMode()
boolean	isCommitBatch() Returns true if this reader calls commit on the connection after every batch is sent.
boolean	isDebug() Indicates if the generated SQL should be logged (default false).
void	open() Makes this endpoint ready for reading or writing.
JdbcUpsertWriter	setAutoCloseConnection(boolean closeConnection)
JdbcUpsertWriter	setBatchSize(int batchSize) Sets the number of records to chunk together in batch updates.
JdbcUpsertWriter	setCommitBatch(boolean commitBatch) Indicates if this reader should call commit on the connection after every batch is sent, otherwise commit is called after the final batch update.
JdbcUpsertWriter	setDebug(boolean debug) Indicates if the generated SQL should be logged (default false).
JdbcUpsertWriter	setJdbcType(Class<?> type, int jdbcType) Forces field values of the specified Java class type to be sent to the database using PreparedStatement.setObject(parameterIndex, fieldValue, jdbcType) (instead of the appropriate PreparedStatement.setXXX() method for its field's type).
JdbcUpsertWriter	setJdbcType(String fieldName, int jdbcType) The given Java object will be converted to the given targetSqlType before being sent to the database.
JdbcUpsertWriter	setNonUpdateFields(FieldList nonUpdateFields) Assigns the list of fields that must not be updated if the recored already exists (default null).
JdbcUpsertWriter	setNonUpdateFields(String... nonUpdateFields)
void	setParameterValue(Field field, int fieldIndex, PreparedStatement statement)
void	setParameterValue(Field field, int fieldIndex, PreparedStatement statement, int parameterIndex)
void	write(Record record) Writes the specified record to this DataWriter and increases the record-count by 1.
protected void	writeImpl(Record record) Overridden by subclasses to write the specified record to this DataWriter.

Methods inherited from class com.northconcepts.datapipeline.core.DataWriter

available, getNestedEndpoint, getNestedWriter, getRootEndpoint, getRootWriter

Methods inherited from class com.northconcepts.datapipeline.core.DataEndpoint

decrementRecordCount, enableJmx, getLastRecord, getRecordCount, getRecordCountAsBigInteger, getRecordCountAsString, incrementRecordCount, isRecordCountBigInteger, resetRecordCount, toString

Methods inherited from class com.northconcepts.datapipeline.core.Endpoint

addElapsedtime, assertClosed, assertNotOpened, assertOpened, finalize, getClosedOn, getDescription, getElapsedTime, getElapsedTimeAsString, getOpenedOn, getOpenElapsedTime, getOpenElapsedTimeAsString, getSelfTime, getSelfTimeAsString, getState, isCaptureElapsedTime, isClosed, isOpen, setCaptureElapsedTime, setDescription

Methods inherited from class com.northconcepts.datapipeline.core.DataObject

exception, exception, exception, getId, getName, resetID

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

currentRecord

protected Record currentRecord

Constructor Detail

JdbcUpsertWriter

public JdbcUpsertWriter(Connection connection,
                        String tableName,
                        IUpsert upsert,
                        String... keyFieldNames)

JdbcUpsertWriter

public JdbcUpsertWriter(Connection connection,
                        String tableName,
                        String... keyFieldNames)

JdbcUpsertWriter

public JdbcUpsertWriter(JdbcConnectionFactory factory,
                        String tableName,
                        IUpsert upsert,
                        String... keyFieldNames)

JdbcUpsertWriter

public JdbcUpsertWriter(JdbcConnectionFactory factory,
                        String tableName,
                        String... keyFieldNames)

Method Detail

getConnection

public Connection getConnection()

getTableName

public String getTableName()

getAutoCloseConnection

public boolean getAutoCloseConnection()

setAutoCloseConnection

public JdbcUpsertWriter setAutoCloseConnection(boolean closeConnection)

getBatchSize

public int getBatchSize()

setBatchSize

public JdbcUpsertWriter setBatchSize(int batchSize)

Sets the number of records to chunk together in batch updates. Updates are not batched if this value is 1 or less. JdbcWriter defaults to non batching updates.

isBatchMode

public boolean isBatchMode()

isCommitBatch

public boolean isCommitBatch()

Returns true if this reader calls commit on the connection after every batch is sent.

setCommitBatch

public JdbcUpsertWriter setCommitBatch(boolean commitBatch)

Indicates if this reader should call commit on the connection after every batch is sent, otherwise commit is called after the final batch update.

getKeyFields

public FieldList getKeyFields()

getNonUpdateFields

public FieldList getNonUpdateFields()

Returns the list of fields that must not be updated if the recored already exists (default null). This method may return null or a list of empty fields.

setNonUpdateFields

public JdbcUpsertWriter setNonUpdateFields(FieldList nonUpdateFields)

Assigns the list of fields that must not be updated if the recored already exists (default null).

setNonUpdateFields

public JdbcUpsertWriter setNonUpdateFields(String... nonUpdateFields)

getUpsert

public IUpsert getUpsert()

setJdbcType

public JdbcUpsertWriter setJdbcType(String fieldName,
                                    int jdbcType)

The given Java object will be converted to the given targetSqlType before being sent to the database.

Note that this method may be used to pass database-specific abstract data types.

Parameters:

fieldName -

jdbcType - the object containing the JDBC Type parameter value

getJdbcType

public Integer getJdbcType(String fieldName)

The Types constant (or a database specific value) assigned to the specified field using setJdbcType(String, int) or null if none was set.

Returns:

one of the Types constants or a database specific value.

setJdbcType

public JdbcUpsertWriter setJdbcType(Class<?> type,
                                    int jdbcType)

Forces field values of the specified Java class type to be sent to the database using PreparedStatement.setObject(parameterIndex, fieldValue, jdbcType) (instead of the appropriate PreparedStatement.setXXX() method for its field's type). This method may be used to pass database-specific abstract data types.

Parameters:

type - the Java class type to override.

jdbcType - one of the Types constants or a database specific value.

getJdbcType

public Integer getJdbcType(Class<?> type)

The Types constant (or a database specific value) assigned to the specified Java class type using setJdbcType(Class, int) or null if none was set.

Returns:

one of the Types constants or a database specific value.

isDebug

public boolean isDebug()

Indicates if the generated SQL should be logged (default false).

setDebug

public JdbcUpsertWriter setDebug(boolean debug)

Indicates if the generated SQL should be logged (default false).

addExceptionProperties

public DataException addExceptionProperties(DataException exception)

Description copied from class: Endpoint

Adds this endpoint's current state to a DataException. Since this method is called whenever an exception is thrown, subclasses should override it to add their specific information.

Overrides:

addExceptionProperties in class DataWriter

setParameterValue

public void setParameterValue(Field field,
                              int fieldIndex,
                              PreparedStatement statement)
                       throws SQLException

Throws:

SQLException

setParameterValue

public void setParameterValue(Field field,
                              int fieldIndex,
                              PreparedStatement statement,
                              int parameterIndex)
                       throws SQLException

Throws:

SQLException

write

public void write(Record record)
           throws DataException

Description copied from class: DataWriter

Writes the specified record to this DataWriter and increases the record-count by 1. Any exception raised while writing will be converted to a DataException using DataObject.exception(Throwable).

Subclasses generally do not need to override this method, instead they should implement DataWriter.writeImpl(Record).

Overrides:

write in class DataWriter

Throws:

DataException

writeImpl

protected void writeImpl(Record record)
                  throws Throwable

Description copied from class: DataWriter

Overridden by subclasses to write the specified record to this DataWriter.

Specified by:

writeImpl in class DataWriter

Throws:

Throwable

open

public void open()

Description copied from class: DataEndpoint

Makes this endpoint ready for reading or writing.

Overrides:

open in class DataEndpoint

close

public void close()

Description copied from class: DataEndpoint

Indicates that this endpoint has finished reading or writing.

Overrides:

close in class DataEndpoint