Read a Fixed-width File / Fixed-length Record File

Updated: Jun 24, 2023

View on GitHub

This example is specifically designed to read data from fixed-width or fixed-length files, providing an efficient and reliable way to extract structured data from such file formats. Users can define the exact widths for each field, allowing for precise parsing and extraction of data from the file.

Real-life use cases for this example can include reading data from legacy systems or working with data files that follow a fixed-width format. It is particularly useful in scenarios where data is organized in columns with predetermined widths, such as mainframe systems, financial records, or log files. By leveraging this library, users can effectively extract and process data from fixed-length files, enabling further analysis, transformation, or integration with other data sources.

 

Fixed Width Input File

Account LastName        FirstName       Balance     CreditLimit   AccountCreated  Rating 
101     Reeves          Keanu           9315.45     10000.00      1/17/1998       A      
312     Butler          Gerard          90.00       1000.00       8/6/2003        B      
868     Hewitt          Jennifer Love   0           17000.00      5/25/1985       B      
761     Pinkett-Smith   Jada            49654.87    100000.00     12/5/2006       A      
317     Murray          Bill            789.65      5000.00       2/5/2007        C      

 

Java Code

package com.northconcepts.datapipeline.examples.cookbook;

import java.io.File;

import org.apache.log4j.Logger;

import com.northconcepts.datapipeline.core.DataEndpoint;
import com.northconcepts.datapipeline.core.Record;
import com.northconcepts.datapipeline.fixedwidth.FixedWidthReader;

public class ReadAFixedWidthFile {
    
    public static final Logger log = DataEndpoint.log; 

    public static void main(String[] args) {
        FixedWidthReader reader = new FixedWidthReader(new File("example/data/input/credit-balance-01.fw"));

        reader.setFieldNamesInFirstRow(true);
        
        reader.addFields(8);
        reader.addFields(16);
        reader.addFields(16);
        reader.skipField(12);  // ignore field 3 - Balance
        reader.addFields(14);
        reader.addFields(16);
        reader.addFields(7);

        reader.open();
        try {
            Record record;
            while ((record = reader.read()) != null) {
                log.info(record);
            }
        } finally {
            reader.close();
        }
    }
}

 

Code Walkthrough

1. A FixedWidthReader instance is created corresponding to the file credit-balance-01.fw.
2. The first row in the dataset is declared as field names with setFieldNamesInFirstRow(true) method.
3. addFields() method is used to specify each column's width/length for the reader. The first column is 8 chars long, the second one is 16 chars long, and so on.
4. skipField() method is used to skip and ignore the column data. In the given example, the Balance field is not added to the reader. 
5. The reader is opened.
6. Each row in the file is deserialized into a Record object and printed to the console.
7. The reader is closed.

 

Console Output

22:53:25,229  INFO [main] datapipeline:39 - Record {
    0:[Account]:STRING=[101]:String
    1:[LastName]:STRING=[Reeves]:String
    2:[FirstName]:STRING=[Keanu]:String
    3:[CreditLimit]:STRING=[10000.00]:String
    4:[AccountCreated]:STRING=[1/17/1998]:String
    5:[Rating]:STRING=[A]:String
}

22:53:25,238  INFO [main] datapipeline:39 - Record {
    0:[Account]:STRING=[312]:String
    1:[LastName]:STRING=[Butler]:String
    2:[FirstName]:STRING=[Gerard]:String
    3:[CreditLimit]:STRING=[1000.00]:String
    4:[AccountCreated]:STRING=[8/6/2003]:String
    5:[Rating]:STRING=[B]:String
}

22:53:25,240  INFO [main] datapipeline:39 - Record {
    0:[Account]:STRING=[868]:String
    1:[LastName]:STRING=[Hewitt]:String
    2:[FirstName]:STRING=[Jennifer Love]:String
    3:[CreditLimit]:STRING=[17000.00]:String
    4:[AccountCreated]:STRING=[5/25/1985]:String
    5:[Rating]:STRING=[B]:String
}

22:53:25,241  INFO [main] datapipeline:39 - Record {
    0:[Account]:STRING=[761]:String
    1:[LastName]:STRING=[Pinkett-Smith]:String
    2:[FirstName]:STRING=[Jada]:String
    3:[CreditLimit]:STRING=[100000.00]:String
    4:[AccountCreated]:STRING=[12/5/2006]:String
    5:[Rating]:STRING=[A]:String
}

22:53:25,242  INFO [main] datapipeline:39 - Record {
    0:[Account]:STRING=[317]:String
    1:[LastName]:STRING=[Murray]:String
    2:[FirstName]:STRING=[Bill]:String
    3:[CreditLimit]:STRING=[5000.00]:String
    4:[AccountCreated]:STRING=[2/5/2007]:String
    5:[Rating]:STRING=[C]:String
}