Welcome to the DataPipeline 7.0 release. Since our last update, the DataPipeline team has been hard at work adding more declarative components, new integrations, new transformations, and generally making the framework easier to use. Our goal is to make simple use-cases easy and complex ones less difficult to implement.
We’re pleased to announce the release of DataPipeline version 6.0. This release includes our new DataPipeline Foundations addon that brings decisioning, source-target data mapping, and other cool features to your software.
Updated: May 2023
With data being produced from many sources in a variety of formats businesses must have a sane way to gain useful insight. Data integration is the process of transforming data from one or more sources into a form that can be loaded into a target system or used for analysis and business intelligence.
Data integration libraries take some programming burden from the shoulders of developers by abstracting data processing and transformation tasks and allowing the developer to focus on tasks that are directly related to the application logic.
Machine learning and artificial intelligence in general are two of today’s hottest skills. AI and ML conferences provide a place for you to improve your skills, discuss trends, and exchange ideas with other data scientists, developers, and entrepreneurs. Whether you’re new to the world of machine learning, trying to stay up-to-date, or just looking to network, there’s a conference happening for you. This article lists over 50 conferences taking place around the world for you to consider attending.
Updated: May 2023
ETL is a process for performing data extraction, transformation and loading. The process extracts data from a variety of sources and formats, transforms it into a standard structure, and loads it into a database, file, web service, or other system for analysis, visualization, machine learning, etc.
ETL tools come in a wide variety of shapes. Some run on your desktop or on-premises servers, while others run as SaaS in the cloud. Some are code-based, built on standard programming languages that many developers already know. Others are built on a custom DSL (domain specific language) in an attempt to be more intentional and require less code. Others still are completely graphical, only offering programming interfaces for complex transformations.
What follows is a list of ETL tools for developers already familiar with Java and the JVM (Java Virtual Machine) to clean, validate, filter, and prepare your data for use.
Updated: June 2022
Being a data scientist means dedication to continuous learning. One great way to keep learning, improve your network, and get exposed to different views is to attend conferences.
Since 2020 organizers have been opting for online virtual conferences instead of in-person conferences. In 2021 and 2022 the same trend continues although some conferences are also being scheduled to be attended in person since the second half of the year 2021.
Data science conferences are one of the best ways to learn, develop new skills, meet and discuss ideas and discover how others are applying AI, analytics and machine learning in their work.
Here are several conferences for data scientists you should consider attending.
We’re excited to introduce Data Pipeline version 4.1, the second update on our 2016 roadmap.
This release features MongoDB integration, expression language additions, and improved transformations and joins. We’ve also thrown in a ton of examples for all the new 4.1 and 4.0 features. Enjoy. Continue reading
Data Pipeline v3.1.4 is now available for download. This release includes support for MySQL upserts, lower JSON and XML memory usage, bug fixes, and more.
Continue reading
Data Pipeline 3.1 is now available for download. This is a milestone release that adds native support for hierarchical data (nested records and multidimensional arrays).
Data Pipeline makes it easy to read, transform, and write XML and Excel files. This post shows you how you too can load data from an on-disk XML file, apply transformations on the fly, and save the result to an Excel file.
