Write a Simple JSON File

Updated: Jul 8, 2023

View on GitHub

This example writes records to a JSON (JavaScript Object Notation) file. It offers a simple and efficient way to serialize data into a widely-used and human-readable format, allowing users to store, exchange, and share structured data in JSON files.

DataPipeline can write records from various sources, such as databases, APIs, or in-memory data structures, into JSON files. This enables data persistence for a variety of use-cases to a lightweight and portable format.

This example can be modified to show how to read records from JSON file.

 

Java Code Listing

package com.northconcepts.datapipeline.examples.cookbook;

import java.io.File;

import com.northconcepts.datapipeline.core.DataReader;
import com.northconcepts.datapipeline.core.DataWriter;
import com.northconcepts.datapipeline.core.Record;
import com.northconcepts.datapipeline.core.RecordList;
import com.northconcepts.datapipeline.job.Job;
import com.northconcepts.datapipeline.json.SimpleJsonWriter;
import com.northconcepts.datapipeline.memory.MemoryReader;

public class WriteSimpleJsonFile {

    public static void main(String[] args) {
        Record record1 = new Record();
        record1.setField("country", "Philippines");
        record1.setField("capital", "Manila");
        record1.setField("language", "Filipino");
 
        Record record2 = new Record();
        record2.setField("country", "Japan");
        record2.setField("capital", "Tokyo");
        record2.setField("language", "Japanese");
                
        DataReader reader = new MemoryReader(new RecordList(record1, record2));
         
        DataWriter writer = new SimpleJsonWriter(new File("example/data/output/simple-json-to-file.json"))
                .setPretty(true);
         
        Job.run(reader, writer);
    }
}

 

Code Walkthrough

1. Record is created to persist data in a key-value field.
2. Record.setField() method is used to add new fields with the specified field name and value. For this example, you are going to write a Record with three fields country, capital and language and value can be added to each field as record1.setField("country", "Philippines");.
3. MemoryReader is created to obtain records from an in-memory RecordList.
4. SimpleJsonWriter is created to write records to a simple-json-to-file.json file.
5. setPretty(true) method enables line breaks and indentations to be automatically added to the output XML file(disabled by default). Notice how the output JSON file is nicely formatted with indentations and line breaks.
6. Data are transferred from MemoryReader to SimpleJsonWriter via Job.run() method. See how to compile and run data pipeline jobs.

 

Record

Record class holds persistent data in key-value fields as it flows through the pipeline. A method setField() in this class creates a new field as key-value pair by taking the field name and a value as a parameter.

 

MemoryReader

Obtains records from an in-memory RecordList. You can get all records added to this object via getRecordList() method.

 

RecordList

As the name suggests it is used to store a list of Record objects in memory. It implements Java's Iterable interface so you can perform operations similar to Java Collections classes to this object.

 

SimpleJsonWriter

Writes JSON in the following simple format. It extends TextWriter class and can be created using File or Writer object. A method setPretty() in this class indicates if line breaks and indentations should be added to output (default false).

 

Output JSON file

[
  {
    "country":"Philippines"
    ,"capital":"Manila"
    ,"language":"Filipino"
  }
  ,{
    "country":"Japan"
    ,"capital":"Tokyo"
    ,"language":"Japanese"
  }
]