Write to JSON Stream Programmatically

Updated: Feb 21, 2022

View on GitHub

This example shows how to create a JSON stream in Java using the JsonTemplate and JsonWriter classes. The demo code creates some test data in memory and writes it to a JSON stream. However, a JSON stream can be created from other input sources like CSV, Excel, etc.

This example can easily be modified to show how to read from a JSON stream or create a JSON stream simply.

Java Code Listing

package com.northconcepts.datapipeline.examples.cookbook;

import java.io.FileWriter;

import com.northconcepts.datapipeline.core.Record;
import com.northconcepts.datapipeline.core.RecordList;
import com.northconcepts.datapipeline.job.Job;
import com.northconcepts.datapipeline.json.JsonWriter;
import com.northconcepts.datapipeline.json.builder.JsonDetailMarker;
import com.northconcepts.datapipeline.json.builder.JsonObject;
import com.northconcepts.datapipeline.json.builder.JsonTemplate;
import com.northconcepts.datapipeline.memory.MemoryReader;

public class WriteToJsonStreamProgrammatically {
    
    
/* Produces the following JSON (line breaks added for clarity)
  
{
  "actors":{
    "north-america":[
      {
        "stage-name":"John Wayne",
        "real-name":"Marion Robert Morrison",
        "gender":"male",
        "city":"Winterset",
        "balance":156.35
      },
      {
        "stage-name":"Spiderman",
        "real-name":"Peter Parker",
        "gender":"male",
        "city":"New York"
      }
    ]
  }
}

*/

    public static void main(String[] args) throws Throwable {

        Record record1 = new Record();
        record1.setField("stageName", "John Wayne");
        record1.setField("realName", "Marion Robert Morrison");
        record1.setField("gender", "male");
        record1.setField("city", "Winterset");
        record1.setField("balance", 156.35);

        Record record2 = new Record();
        record2.getField("stageName", true).setValue("Spiderman");
        record2.getField("realName", true).setValue("Peter Parker");
        record2.getField("gender", true).setValue("male");
        record2.getField("city", true).setValue("New York");
        record2.getField("balance", true).setValue(-0.96);
               
        MemoryReader reader = new MemoryReader(new RecordList(record1, record2));
        
        
        // create the JSON template
        // arguments to field(), objectField(), arrayField(), and when() are expressions
        // literal strings must be quoted 
        JsonTemplate template = new JsonTemplate();
        
        // detail() marks the node where records are added
        JsonDetailMarker detailElement = template.object().object("'actors'").array("'north-america'").detail();
        
        JsonObject actor = detailElement.object();

        actor.field("'stage-name'", "stageName");
        actor.field("'real-name'", "realName");
        actor.field("'gender'", "gender");
        actor.field("'city'", "city");
        actor.when("balance >= 0").field("'balance'", "balance");  // add this field when balance is not negative

        // add the template to the writer
        JsonWriter writer = new JsonWriter(template, new FileWriter("example/data/output/credit-balance-03.json"));
        
        Job.run(reader, writer);
    }

}

Code Walkthrough

1. Two Record objects are created in memory with the fields stageName, realName,gender,city and balance set to some test values.
2. A RecordList is created with these Records.
3. A MemoryReader is created to read this RecordList.
4. The JSON stream structure is created using the JsonTemplate, JSonDetailMarker and JsonObject instances.
5. A JsonWriter is created using the JsonTemplate and the path of the output file credit-balance-03.json.
6. Data is transferred from memory to the output JSON file via JobTemplate.DEFAULT.transfer method.

JSON stream creation

The following steps eleborate how the JSON stream is created (Step 4 above):

1. First a JsonTemplate is created.
2. The template.object method is invoked to obtain the JsonObject corresponding to the template.
3. A JsonObject is then created corresponding to the actors field via JsonObject.object("'actors'").
4. A JsonArray is created corresponding to the north-america field via the object.array method.
5. The array.detail method is invoked to indicate that this element has sub-elements which returns a JSonDetailMarker.
6. A new JsonObject is created corresponding to the actor element via JsonDetailMarker.object.
7. The stageName, realName,gender,city and balance fields are set via actor.field method. This method accepts two String parameters, the first String specifies the field name to be used in the output JSON stream and the second String specifies the field name in the input Record object that should be used to get the value of this field name.
8. The actor.when method is used to create and add a balance field only when the balance is non negative.

JsonTemplate and JsonObject

JsonTemplate and JsonObject are the main classes used to create the output JSON stream. JsonTemplate is a high level abstraction which represents the entire JSON stream. JsonObject represents an individual JSON element.

Output JSON stream

 {
  "actors":{
    "north-america":[
      {
        "stage-name":"John Wayne",
        "real-name":"Marion Robert Morrison",
        "gender":"male",
        "city":"Winterset",
        "balance":156.35
      },
      {
        "stage-name":"Spiderman",
        "real-name":"Peter Parker",
        "gender":"male",
        "city":"New York"
      }
    ]
  }
}