Real-time data processing and aggregation in Java.
- ✔ Process large volumes of data inside your apps.
- ✔ Combine, filter, and aggregate without a database.
- ✔ Operate on streaming data in-flight.
- ✔ Create reusable data processing components.
Small Footprint
Data Pipeline takes up less than 20 MB on disk and runs in less than 20 MB of RAM.
Streaming Data
Processing data one piece at a time as it moves through the pipeline can be more than 100x faster than storing it to disk to query or process later.
Fully Customizable
Enhance the engine to fit your unique needs. Plug in your own logic or modify existing behavior to your specific requirements.
100% Java
Use the JVM languages and tools you already know: Java, Clojure, Groovy, Scala, JRuby, Jython, JavaScript, and more.
What you can do with Data Pipeline
- Migrate data from one format or structure to another
- Consume or produce large XML, CSV, or fixed-width files
- Use the same data processing code in your desktop and web apps
- Embed in your data management tool, app, or service
- Replace hourly batch jobs with real-time data
- Prepare data for loading into a Big Data database
- Convert data to/from your vendor or customer's format
- Process data in-app as it arrives or leaves
- Stream event data to customers as it occurs