Write to JSON Stream (Simple)

This example shows how to write to a JSON stream in Java using the SimpleJsonWriter class. The demo code creates test data in memory and writes it to a JSON stream. However, the JSON stream can be created from other input sources like CSV, Excel, etc.

This example can easily be modified to show how to read from a JSON stream.

Java Code Listing

package com.northconcepts.datapipeline.examples.cookbook;

import java.io.OutputStreamWriter;

import com.northconcepts.datapipeline.core.Record;
import com.northconcepts.datapipeline.core.RecordList;
import com.northconcepts.datapipeline.job.Job;
import com.northconcepts.datapipeline.json.SimpleJsonWriter;
import com.northconcepts.datapipeline.memory.MemoryReader;

public class WriteToJsonStream {

    /*
     * Produces the following JSON (line breaks added for clarity)
     * 
     * [ { "stageName":"John Wayne", "realName":"Marion Robert Morrison", "gender":"male", "city":"Winterset", "balance":156.35 }, {
     * "stageName":"Spiderman", "realName":"Peter Parker", "gender":"male", "city":"New York", "balance":-0.96 } ]
     */

    public static void main(String[] args) throws Throwable {

        Record record1 = new Record();
        record1.setField("stageName", "John Wayne");
        record1.setField("realName", "Marion Robert Morrison");
        record1.setField("gender", "male");
        record1.setField("city", "Winterset");
        record1.setField("balance", 156.35);

        Record record2 = new Record();
        record2.setField("stageName", "Spiderman");
        record2.setField("realName", "Peter Parker");
        record2.setField("gender","male");
        record2.setField("city", "New York");
        record2.setField("balance", -0.96);

        MemoryReader reader = new MemoryReader(new RecordList(record1, record2));

        SimpleJsonWriter writer = new SimpleJsonWriter(new OutputStreamWriter(System.out));

        Job.run(reader, writer);
    }

}

Code Walkthrough

1. Two Record objects are created in memory with the fields stageName, realName,gender,city and balance set to some test values.
2. These Record objects are added to a RecordList.
3. A MemoryReader is created to read this RecordList.
4. A new SimpleJsonWriter is created corresponding to the file path of the output JSON file credit-balance-04.json.
5. Data are transferred from MemoryReader to SimpleJsonWriter via Job.run() method. See how to compile and run data pipeline jobs.

SimpleJsonWriter

A SimpleJsonWriter is an output writer that can be used to write to a JSON stream. It is a sub-class of TextWriter.html and overrides the open and close among other methods. It encapsulates all the details of writing data to a JSON stream, so the user need not write this code.
A JSON stream may also be created programmatically (refer create a JSON stream programmatically). This approach should be used when one needs more control over the JSON stream creation process.

Record

Record class holds persistent data in key-value fields as it flows through the pipeline. A method setField() in this class creates a new field as key-value pair by taking field name and a value as a parameter.

RecordList

As the name suggests it is used to store a list of Record objects in memory. It implements Iterable so you can perform operations similar to Java Collection to this object.

MemoryReader

Obtains records from an in-memory RecordList. You can obtain records added to this object via getRecordList() method.

Output JSON file

[
   {
      "stageName":"John Wayne",
      "realName":"Marion Robert Morrison",
      "gender":"male",
      "city":"Winterset",
      "balance":156.35
   },
   {
      "stageName":"Spiderman",
      "realName":"Peter Parker",
      "gender":"male",
      "city":"New York",
      "balance":-0.96
   }
]