Real-time data processing and aggregation in Java.
- ✔ Process large volumes of data inside your apps.
- ✔ Combine, filter, and aggregate without a database.
- ✔ Operate on streaming data in-flight.
- ✔ Create reusable data processing components.
Data Pipeline takes up less than 20 MB on disk and runs in less than 20 MB of RAM.
Processing data one piece at a time as it moves through the pipeline can be more than 100x faster than
storing it to disk to query or process later.
Enhance the engine to fit your unique needs. Plug in your own logic or modify existing behavior to your
Use the JVM languages and tools you already know: Java, Clojure, Groovy, Scala, JRuby, Jython,
What you can do with Data Pipeline
- Migrate data from one format or structure to another
- Consume or produce large XML, CSV, or fixed-width files
- Use the same data processing code in your desktop and web apps
- Embed in your data management tool, app, or service
- Replace hourly batch jobs with real-time data
- Prepare data for loading into a Big Data database
- Convert data to/from your vendor or customer's format
- Process data in-app as it arrives or leaves
- Stream event data to customers as it occurs
Learn how Data Pipeline and stream processing can help your business.