Write Concatenated JSON

Updated: Sep 26, 2022

This example shows you how to write concatenated JSON using JsonLinesWriter.

Several Records are used to create the sample input data. You can also use a file as input instead as seen in the Write Concatenated JSON With Nested Data example.

Java Code Listing

package com.northconcepts.datapipeline.examples.cookbook;

import com.northconcepts.datapipeline.core.DataWriter;
import com.northconcepts.datapipeline.core.Record;
import com.northconcepts.datapipeline.core.RecordList;
import com.northconcepts.datapipeline.job.Job;
import com.northconcepts.datapipeline.json.JsonLinesWriter;
import com.northconcepts.datapipeline.memory.MemoryReader;


import java.io.File;

public class WriteConcatenatedJson {
    public static void main(String[] args) {
        Record record1 = new Record();
        record1.setField("stageName", "John Wayne");
        record1.setField("realName", "Marion Robert Morrison");
        record1.setField("gender", "male");
        record1.setField("city", "Winterset");
        record1.setField("balance", 156.35);

        Record record2 = new Record();
        record2.setField("stageName", "Spiderman");
        record2.setField("realName", "Peter Parker");
        record2.setField("gender","male");
        record2.setField("city", "New York");
        record2.setField("balance", -0.96);

        MemoryReader reader = new MemoryReader(new RecordList(record1, record2));
        
        DataWriter writer = new JsonLinesWriter(new File("data/output/concatenated-json.jsonl")).setNewLine("");

        Job.run(reader, writer);
    }
}

Code Walkthrough

  1. MemoryReader is used to obtain the records from RecordList.
  2. A JsonLinesWriter object is created to write to the concatenated-json.jsonl output file. By default JsonLinesWriter will print each record on a new line so .setNewLine("") is added in order to concatenate the JSON data without a line break.
  3. Data is then transferred from the reader to the output file via Job.run().

Output

{"stageName":"John Wayne","realName":"Marion Robert Morrison","gender":"male","city":"Winterset","balance":156.35}{"stageName":"Spiderman","realName":"Peter Parker","gender":"male","city":"New York","balance":-0.96}

All Examples

Mobile Analytics