Data Pipeline 2.2.5 Now Available

  • Added JavaBeanReader whice uses XPath expressions to identify field values and break records (see the Read from Java beans example)
  • AbstractReader’s setStartingRow and setLastRow no return this
  • Filter rule IsInstanceOfJavaType now returns false for null values
  • Added number-to-date methods to BasicFieldTransformer (numberToDate(), minutesToDate(), hoursToDate(), and daysToDate())
  • BasicFieldTransformer.Operation and BasicFieldTransformer.StringOperation are now public classes
  • BasicFieldTransformer.add(Operation … operation) is now public
  • ConditionalTransformer is now private (use TransformingReader.filter instead)
  • TransformingReader now contains an optional Filter, allowing any transformer to be conditionally applied
  • Removed TransformingReader.add(Filter filter, Transformer … transformer) method

How To Manage Your Application Properties using WebDAV

Managing the configuration of an application is a consistent pain-point for developers, administrators, and business analysts.

Often in production environments, configuration is isolated as files on the local disk, limiting easy access by all but administrators.  Another common approach is to store configuration in a database or LDAP.  While this have benefits of a local disk file, it lacks the ability to manage properties as naturally as a file.

The solution proposed uses a database and WebDAV to help resolve many negatives to do with application configuration in both a simple and lightweight way.
Continue reading

Use dynamic proxies to create a simple, powerful event bus (Part 2)

In part 1 of the event bus series we discussed implementing a simple and powerful event bus using just three classes. If you haven’t read it yet, I strongly recommend you read it first.

Use dynamic proxies to create a simple, powerful event bus (Part 1)

In this blog we’ll build on part 1 by adding several important features to the event bus to make it production ready.  Continue reading

Data Pipeline 2.2.1 Now Available

Data Pipeline 2.2.1 is now available with the following enhancements:

  • added batch execution to JdbcWriter (see JdbcWriter.setBatchSize)
  • added callback mechanism to track job progress (see JobTemplate.transfer(R reader, W writer, boolean async, JobCallback callback))
  • early access to DeMuxReader

Data Pipeline 2.2 Released

We are happy to announce that Data Pipeline 2.2 is now released. We have added the enhancements below:

  • added an XPath-based XmlReader
  • updates so that Excel now defaults to the Apache POI instead of JXL
  • updates so that the following classes use java.util.List instead of java.util.ArrayList in their public APIs: CompositeValue, FieldList, Lookup, LookupTransformer, Record, RecordList