All posts by The DataPipeline Team

About The DataPipeline Team

We make Data Pipeline — a lightweight ETL framework for Java. Use it to filter, transform, and aggregate data on-the-fly in your web, mobile, and desktop apps. Learn more about it at northconcepts.com.

Data Pipeline 2.2.1 Now Available

Data Pipeline 2.2.1 is now available with the following enhancements:

  • added batch execution to JdbcWriter (see JdbcWriter.setBatchSize)
  • added callback mechanism to track job progress (see JobTemplate.transfer(R reader, W writer, boolean async, JobCallback callback))
  • early access to DeMuxReader

Data Pipeline 2.2 Released

We are happy to announce that Data Pipeline 2.2 is now released. We have added the enhancements below:

  • added an XPath-based XmlReader
  • updates so that Excel now defaults to the Apache POI instead of JXL
  • updates so that the following classes use java.util.List instead of java.util.ArrayList in their public APIs: CompositeValue, FieldList, Lookup, LookupTransformer, Record, RecordList

Data Pipeline 2.1 Released

We are happy to announce that Data Pipeline 2.1 is now released. We have added a great deal of enhancements below:

  • support for Excel 2003 XLS files
  • support for Excel XLSX (XML format) files
  • added FixedWidthReader.setLastFieldConsumesRemaining(boolean lastFieldConsumesRemaining) functionality
  • added ExcelReader.setUseSheetColumnCount(boolean useSheetColumnCount) functionality
  • added more string utils to BasicFieldTransformer
  • added ConditionalTransformer class
  • added TransformingReader.add(Filter filter, Transformer ... transformer) functionality
  • setField now has type-specific constructors
  • added Eclipse project files
  • added Ant build project
  • bug-fix: whitespace (like tab) can now be used as the field separator in CSVReader
  • bug-fix: handle null variable names in expressions