Embeddable ETL engine for Java

  • ✔ Process large volumes of data inside your app.
  • ✔ Combine, filter, and aggregate without a database.
  • ✔ Operate on streaming data in-flight.
  • ✔ Create reusable data processing components.
  • ✔ Shorten your development time.

What you can do with Data Pipeline

  • Migrate data from one format or structure to another
  • Consume (or produce) large XML, CSV, and fixed-width files
  • Use the same data processing code in your desktop and web apps
  • Embed in your data management tools, apps, and services
  • Replace hourly batch jobs with real-time data
  • Prepare data for loading into a Big Data database
  • Convert data to/from your vendor or customer's format
  • Process data in-app as it arrives or leaves
  • Stream event data to customers as it occurs

Small Footprint

Data Pipeline takes up less than 20 MB on disk and runs in less than 20 MB of RAM.

Streaming Data

Processing data one piece at a time as it moves through the pipeline can be more than 100x faster than storing it to disk to query or process later.

Fully Customizable

Enhance the engine to fit your unique needs. Plug in your own logic or modify existing behavior to your specific requirements.

100% Java

Use the JVM languages and tools you already know: Java, Clojure, Groovy, Scala, JRuby, Jython, JavaScript, and more.

Explore the Code

Learn how Data Pipeline and stream processing can help your business.