Frequently Asked Questions

Data Pipeline makes it easy to add data conversion, processing, and transformation to Java applications.

The toolkit has readers and writers for physical files (CSV, Excel, Fixed-width), plus chainable secondary readers for processing and transforming data (filter, remove duplicates, lookups, validation).

Data Pipeline is actively being updated with new features to reflect our customer's needs. If you have an idea for a new feature, please send us a message through our contact page and we'll attempt to include it in a future release.

If your request is of an urgent nature, please email us, we would be happy to provide you a quote to enhance or customize Data Pipeline to your needs.

First, please use the search at the top of the page, perhaps your question is answered elsewhere on this site.

You can send us a question through our contact page and we will answer as soon as possible.

For priority items, we provide commercial support through email for an additional fee on an annual basis. Commercial support provides you with the most responsive support from our experts, it features:

• Priority treatment
• Dedicated Email Address from our Technical Support Team
• Less than 24 hour response time during business days

If you've purchased a Commercial Support Package from us, please send your questions to support@northconcepts.com. Please include the version # and as much detail about your issue as possible.

For priority handling, we provide a Commercial Support Package, which can be purchased on an annual basis. For other items, please send your questions using our contact page.

Yes. We are actively updating Data Pipeline with new features to reflect our customer's needs.

Data Pipeline is available under a Commercial License Agreement and can be bought directly on our purchase page.

If you are interested in an alternative licensing option (such as an OEM or educational license), please email us with your needs.

Please email us with your specific needs.

Our Getting Started Guide and Examples will have you up and running quickly.

Data Pipeline may stall if DataReader.read() is called before DataReader.open().

Java's default, maximum memory size of 64 MB may not be adequate for your application.  Try using the -Xmx Java option to set a higher maximum.

Data Pipeline supports Oracle Java Standard Edition SDK 1.8 or greater.

Yes. DebugReader can be layered onto another DataReader to log everything it reads.

Data Pipeline is distributed with all the necessary libraries (except for JDBC drivers) under the distributions lib folder.

Data can be cached in a RecordList in one of two ways:
1. Automatically, by writing to a MemoryWriter then calling getRecordList(). See the write to memory example.
2. Manually, by creating a RecordList and adding Records to it. See the read from memory example.

Yes. RecordLists can be searched by calling findAll(), findFirst(), or findLast() . Searches can also be performed by filtering incoming data.

Yes. FilteringReader provides a means of selecting records either programmatically or using the run-time expression language. See the filter records example.

Simply reading an MS Excel file with the evaluateExpressions property set to true (by default) will cause VLookups to be evaluated. See the read from an Excel file example.