Conditionally Map Data Field from Source to Target

This example shows how you can use Data Pipeline to perform data mapping operations from a source to a target using conditions. It allows users to define conditional rules that determine how data in each field should be mapped based on specific criteria, enabling flexible and dynamic data transformation.

The example can be used to selectively map data based on certain conditions, allowing users to filter and segment data during the mapping process. This can be useful for creating subsets of data based on specific criteria, such as mapping only sales data for a particular region or time period.

Java Code Listing

package com.northconcepts.datapipeline.foundations.examples.datamapping;

import com.northconcepts.datapipeline.core.Record;
import com.northconcepts.datapipeline.foundations.datamapping.DataMapping;
import com.northconcepts.datapipeline.foundations.datamapping.DataMappingResult;
import com.northconcepts.datapipeline.foundations.datamapping.FieldMapping;
import com.northconcepts.datapipeline.internal.expression.DefaultExpressionContext;

public class ConditionallyMapDataFieldFromSourceToTarget {

    public static void main(String[] args) {
        DefaultExpressionContext input = new DefaultExpressionContext();
        input.setValue("fname", "John");
        input.setValue("lname", "Smith");

        DataMapping mapping = new DataMapping()
                .addFieldMapping(new FieldMapping("first_name", "source.fname").setCondition("length(source.fname) > 3")) // don't map this field if fname is not greater than 3 characters
                .addFieldMapping("last_name", "toUpperCase(source.lname)");
        
        DataMappingResult result = mapping.map(input);
        Record target = result.getTarget();

        System.out.println("target = " + target);
        
        result.logExceptions();
    }

}

Code Walkthrough

1. DefaultExpressionContext instance is used to hold the source/input data with the key-value pattern. Values are entered with setValue() method.
2. DataMapping is created to enable the field mapping rules.
3. addFieldMapping() method accepts two String arguments:
   1. targetFieldName - the name of the field in the target record.
   2. sourceExpression - a custom expression to hold the formatting logic for a target field.
4. Fields in the source and target records can be accessed with the dot (.) operator: source.fname.
5. Checking for a condition in a field is applied with setCondition() method. The expression in the example checks whether the length of source.fname field is greater than 3. Only if this condition is met, the mapping for the field is implemented.
6. Different methods for formatting strings can be used in sourceExpression. In the given example, toUpperCase() is called.
7. DataMappingResult instance is created to apply mapping for the input.
8. The target record is obtained from result object and printed in the console with System.out.println().
9. The exceptions from the result, if exist, are also logged.

Console output

target = Record (MODIFIED) {
    0:[first_name]:STRING=[John]:String
    1:[last_name]:STRING=[SMITH]:String
}