Write JSON Lines

This example shows you how to write JSON Lines using JsonLinesWriter.

The input source is created from a MemoryReader with programmatically added records.

This example can be easily modified to show how to Write Concatenated JSON.

Java Code Listing

package com.northconcepts.datapipeline.examples.cookbook;

import java.io.File;

import com.northconcepts.datapipeline.core.DataWriter;
import com.northconcepts.datapipeline.core.Record;
import com.northconcepts.datapipeline.core.RecordList;
import com.northconcepts.datapipeline.job.Job;
import com.northconcepts.datapipeline.json.JsonLinesWriter;
import com.northconcepts.datapipeline.memory.MemoryReader;

public class WriteJsonLines {
	public static void main(String[] args) {
		Record record1 = new Record();
		record1.setField("stageName", "John Wayne");
		record1.setField("realName", "Marion Robert Morrison");
		record1.setField("gender", "male");
		record1.setField("city", "Winterset");
		record1.setField("balance", 156.35);

		Record record2 = new Record();
		record2.setField("stageName", "Spiderman");
		record2.setField("realName", "Peter Parker");
		record2.setField("gender","male");
		record2.setField("city", "New York");
		record2.setField("balance", -0.96);

		MemoryReader reader = new MemoryReader(new RecordList(record1, record2));
		
		DataWriter writer = new JsonLinesWriter(new File("data/output/json-lines.jsonl"));

		Job.run(reader, writer);
	}
}

Code Walkthrough

1. MemoryReader is used to obtain the records from RecordList
2. The JsonLinesWriter object is created to produce the json-lines.jsonl output file.
3. Data is then transferred from the reader to the output file via Job.run().

Output

{"stageName":"John Wayne","realName":"Marion Robert Morrison","gender":"male","city":"Winterset","balance":156.35}
{"stageName":"Spiderman","realName":"Peter Parker","gender":"male","city":"New York","balance":-0.96}