Read JSON Records From File

Updated: Sep 12, 2023

View on GitHub

In this example, you are going to learn how you can read JSON records from a file using DataPipeline. You can employ this example for applications such as log analysis, data migration, configuration file processing, and web scraping, where the ability to read and manipulate JSON data within Java-based projects is essential for effective data handling and processing.

 

Input JSON File

[
  {
    "country":"Philippines"
    ,"capital":"Manila"
    ,"language":"Filipino"
    ,"x-coodrinates" : 14.583331
    ,"y-coodrinates" : 121.0
  }
  ,{
    "country":"Japan"
    ,"capital":"Tokyo"
    ,"language":"Japanese"
    ,"x-coodrinates" : 35.652832
    ,"y-coodrinates" : 139.839478
  }
]

 

Java code listing

package com.northconcepts.datapipeline.examples.cookbook;

import java.io.File;

import com.northconcepts.datapipeline.core.DataReader;
import com.northconcepts.datapipeline.core.DataWriter;
import com.northconcepts.datapipeline.core.StreamWriter;
import com.northconcepts.datapipeline.job.Job;
import com.northconcepts.datapipeline.json.JsonRecordReader;

public class ReadJsonRecordsFromFile {

    public static void main(String[] args) {
        DataReader reader = new JsonRecordReader(new File("example/data/input/simple-json-to-file-with-bigdecimal.json"))
                .addRecordBreak("/array/object");
        
        DataWriter writer = StreamWriter.newSystemOutWriter();
        
        Job.run(reader, writer);
    }
}

 

Code walkthrough

1. JsonRecordReader is created corresponding to the input file simple-json-to-file-with-bigdecimal.json.
2. .addRecordBreak("/array/object") adds a separator/divider between each object in the array.
3. StreamWriter is used to write the data to the console.newSystemOutWriter() creates a new instance of StreamWriter writing to System.out.
4. Data is transferred from the reader to the writer via Job.run() method. See how to compile and run data pipeline jobs.

 

Console Output

-----------------------------------------------
0 - Record (MODIFIED) {
    0:[country]:STRING=[Philippines]:String
    1:[capital]:STRING=[Manila]:String
    2:[language]:STRING=[Filipino]:String
    3:[x-coodrinates]:DOUBLE=[14.583331]:Double
    4:[y-coodrinates]:DOUBLE=[121.0]:Double
}

-----------------------------------------------
1 - Record (MODIFIED) {
    0:[country]:STRING=[Japan]:String
    1:[capital]:STRING=[Tokyo]:String
    2:[language]:STRING=[Japanese]:String
    3:[x-coodrinates]:DOUBLE=[35.652832]:Double
    4:[y-coodrinates]:DOUBLE=[139.839478]:Double
}

-----------------------------------------------
2 records