Powering in-app data processing
- ✔ Process large volumes of data inside your app.
- ✔ Combine, filter, and aggregate without a database.
- ✔ Operate on streaming data in-flight.
- ✔ Create reusable data processing components.
- ✔ Shorten your development time.
What you can do with Data Pipeline
- Migrate data from one format or structure to another
- Consume (or produce) large XML, CSV, and fixed-width files
- Use the same data processing code in your desktop and web apps
- Embed in your data management tools, apps, and services
- Replace hourly batch jobs with real-time data
- Prepare data for loading into a Big Data database
- Convert data to/from your vendor or customer's format
- Process data in-app as it arrives or leaves
- Stream event data to customers as it occurs
Data Pipeline takes up less than 20 MB on disk and runs in less than 20 MB of RAM.
Processing data one piece at a time as it moves through the pipeline can be more than 100x faster than
storing it to disk to query or process later.
Enhance the engine to fit your unique needs. Plug in your own logic or modify existing behavior to your
Use the JVM languages and tools you already know: Java, Clojure, Groovy, Scala, JRuby, Jython,
Learn how Data Pipeline and stream processing can help your business.