Read Json Lines As Record

Updated: Jan 22, 2023

View on GitHub

This example shows you how to read data from a JSON Lines file in Java using the JsonRecordReader class.

This demo code reads objects from a JSON Lines file via JsonRecordReader and then writes an array of objects to the console via StreamWriter class.

This example can easily be modified to show how to read Json Lines using JsonReader class.

Input JSON file

{"Employee": [{"empMID": "mock:1", "comments": [], "col1": "something", "contact": [{"address":"2400 waterview", "freetext":true}], "gender": "male"}, {"empMID": "mock:2", "comments": [], "col1": "something", "contact": [{"address":"2200 waterview", "freetext":true}], "gender": "female"}], "cola": false, "colb": false}
{"Employee": [{"empMID": "mock:10", "comments": [], "col1": "something", "contact": [{"address":"2410 waterview", "freetext":true}], "gender": "female"}, {"empMID": "mock:11", "comments": [], "col1": "something", "contact": [{"address":"2210 waterview", "freetext":true}], "gender": "female"}], "cola": false, "colb": false}

Java Code Listing

package com.northconcepts.datapipeline.examples.cookbook;

import com.northconcepts.datapipeline.core.DataReader;
import com.northconcepts.datapipeline.core.DataWriter;
import com.northconcepts.datapipeline.core.StreamWriter;
import com.northconcepts.datapipeline.job.Job;
import com.northconcepts.datapipeline.json.JsonRecordReader;

import java.io.File;

public class ReadJsonLinesAsRecord {
    public static void main(String[] args) {
        DataReader reader = new JsonRecordReader(new File("data/input/json-input-with-lines.jsonl"))
                .addRecordBreak("/object");

        DataWriter writer = StreamWriter.newSystemOutWriter();

        Job.run(reader, writer);
    }
}

Code Walkthrough

1. First, JsonRecordReader object is created corresponding to the input file json-input-with-lines.jsonl.
2. Unlike with JsonReader where you need to specify the fields that are to be included, using JsonRecordReader does not require you to specify the fields.
3. .addRecordBreak() method is used to separate the records. /object specify the path to the record you want to read. To see the paths to the record run the code in debug mode i.e .setDebug(true), it is false by default.
4. Data is extracted from the JSON file via Job.run().
5. Array of objects are then printed on the console via StreamWriter class.

Console Output

-----------------------------------------------
0 - Record (MODIFIED) (has child records) {
    0:[Employee]:ARRAY of RECORD=[[
        Record (MODIFIED) (is child record) (has child records) {
            0:[empMID]:STRING=[mock:1]:String
            1:[comments]:ARRAY of UNDEFINED=[[]]:ArrayValue
            2:[col1]:STRING=[something]:String
            3:[contact]:ARRAY of RECORD=[[
                Record (MODIFIED) (is child record) {
                    0:[address]:STRING=[2400 waterview]:String
                    1:[freetext]:BOOLEAN=[true]:Boolean
                }]]:ArrayValue
            4:[gender]:STRING=[male]:String
        }, 
        Record (MODIFIED) (is child record) (has child records) {
            0:[empMID]:STRING=[mock:2]:String
            1:[comments]:ARRAY of UNDEFINED=[[]]:ArrayValue
            2:[col1]:STRING=[something]:String
            3:[contact]:ARRAY of RECORD=[[
                Record (MODIFIED) (is child record) {
                    0:[address]:STRING=[2200 waterview]:String
                    1:[freetext]:BOOLEAN=[true]:Boolean
                }]]:ArrayValue
            4:[gender]:STRING=[female]:String
        }]]:ArrayValue
    1:[cola]:BOOLEAN=[false]:Boolean
    2:[colb]:BOOLEAN=[false]:Boolean
}

-----------------------------------------------
1 - Record (MODIFIED) (has child records) {
    0:[Employee]:ARRAY of RECORD=[[
        Record (MODIFIED) (is child record) (has child records) {
            0:[empMID]:STRING=[mock:10]:String
            1:[comments]:ARRAY of UNDEFINED=[[]]:ArrayValue
            2:[col1]:STRING=[something]:String
            3:[contact]:ARRAY of RECORD=[[
                Record (MODIFIED) (is child record) {
                    0:[address]:STRING=[2410 waterview]:String
                    1:[freetext]:BOOLEAN=[true]:Boolean
                }]]:ArrayValue
            4:[gender]:STRING=[female]:String
        }, 
        Record (MODIFIED) (is child record) (has child records) {
            0:[empMID]:STRING=[mock:11]:String
            1:[comments]:ARRAY of UNDEFINED=[[]]:ArrayValue
            2:[col1]:STRING=[something]:String
            3:[contact]:ARRAY of RECORD=[[
                Record (MODIFIED) (is child record) {
                    0:[address]:STRING=[2210 waterview]:String
                    1:[freetext]:BOOLEAN=[true]:Boolean
                }]]:ArrayValue
            4:[gender]:STRING=[female]:String
        }]]:ArrayValue
    1:[cola]:BOOLEAN=[false]:Boolean
    2:[colb]:BOOLEAN=[false]:Boolean
}

-----------------------------------------------
2 records