Read from Memory

Updated: Feb 21, 2022

View on GitHub

This example shows you how to read data from memory using the MemoryReader class. The demo code creates some data in memory and writes it to a CSV file. However, data from memory can be written to other types of output sources like MS Excel, XML,Database, etc as well.

This example can also be modified to write to memory.

Java Code Listing

package com.northconcepts.datapipeline.examples.cookbook;

import java.io.File;

import com.northconcepts.datapipeline.core.DataReader;
import com.northconcepts.datapipeline.core.DataWriter;
import com.northconcepts.datapipeline.core.Record;
import com.northconcepts.datapipeline.core.RecordList;
import com.northconcepts.datapipeline.csv.CSVWriter;
import com.northconcepts.datapipeline.job.Job;
import com.northconcepts.datapipeline.memory.MemoryReader;

public class ReadFromMemory {

    public static void main(String[] args) {
        Record record1 = new Record();
        record1.setField("name", "John Wayne");
        record1.setField("balance", 156.35);

        Record record2 = new Record();
        record2.setField("name", "Peter Parker");
        record2.setField("balance", 0.96);
        
        RecordList recordList = new RecordList(record1, record2);
        
        DataReader reader = new MemoryReader(recordList);
        DataWriter writer = new CSVWriter(new File("example/data/output/credit-balance-02.csv"));
    
        Job.run(reader, writer);
    }
    
}

Code walkthrough

1. First two Record objects are created with fields called name and balance set to some test values.
2. A new RecordList is created and these objects are added to it.
3. A MemoryReader is created corresponding to the RecordList.
4. A CSVWriter is created corresponding to the output CSV file credit-balance-02.csv.
5. Data is transferred from memory to the output CSV file via JobTemplate.DEFAULT.transfer.

MemoryReader

A MemoryReader is an input reader that reads data from memory. It is a sub-class of DataReader and is created using a RecordList. It has a method called getRecordList() which can be used to retrieve the underlying RecordList.
RecordList is a data structure for storing Record objects. Record encapsulates a single line of information and is the basic unit of data transfer. Data transfers of all types occur as Record objects i.e. data from an input source is read as a Record object and written to the output source as a Record object.

Output CSV file

name,balance
John Wayne,156.35
Peter Parker,0.96