Read Big Decimal And Big Integer In JSON

This example shows you how to read big decimals and/or big integers from a JSON file using DataPipeline.

In this demo code, you are going to use JsonRecordReader to read records that contain productId (BigInteger), productName, price (BigDecimal), availableStock (BigInteger), and rating (BigDecimal) fields from simple-json-to-file-with-bigdecimal.json file and print them on the console via StreamWriter.newSystemOutWriter().

BigDecimal consists of an arbitrary precision integer unscaled value and a 32-bit integer scale, and BigInteger consists of an arbitrary precision integer value with no fractional component or scale.

Input JSON file

[
  {
    "productId": 100000000000000000000000000001,
    "productName": "UltraWidget Pro",
    "price": 1499.99,
    "availableStock": 5000000000000000,
    "rating": 4.75
  },
  {
    "productId": 100000000000000000000000000002,
    "productName": "MegaGadget X",
    "price": 2999.49,
    "availableStock": 3000000000000000,
    "rating": 4.9
  },
  {
    "productId": 100000000000000000000000000003,
    "productName": "SmartDevice Z",
    "price": 799.95,
    "availableStock": 750000000000000,
    "rating": 4.6
  }
]

Java Code Listing

package com.northconcepts.datapipeline.examples.cookbook;

import java.io.File;

import com.northconcepts.datapipeline.core.DataReader;
import com.northconcepts.datapipeline.core.DataWriter;
import com.northconcepts.datapipeline.core.StreamWriter;
import com.northconcepts.datapipeline.job.Job;
import com.northconcepts.datapipeline.json.JsonRecordReader;

public class ReadBigDecimalAndBigIntegerInJson {

    public static void main(String[] args) {
        DataReader reader = new JsonRecordReader(new File("example/data/input/simple-json-to-file-with-bigdecimal-and_biginteger.json"))
                .addRecordBreak("/array/object")
                .setUseBigDecimal(true)
                .setUseBigInteger(true);
        
        DataWriter writer = StreamWriter.newSystemOutWriter();
        
        Job.run(reader, writer);
    }
}

Code Walkthrough

1. First, JsonRecordReader is created corresponding to the input JSON file simple-json-to-file-with-bigdecimal-and_biginteger.json.
2. By passing true to setUseBigDecimal() method, you are enabling JsonRecordReader to use BigDecimal which is disabled by default.
3. By passing true to setUseBigInteger() method, you are enabling JsonRecordReader to use BigInteger which is disabled by default.
4. In this example JsonRecordReader is used to read the productId, availableStock fields as BigInteger, and the price & rating fields as BigDecimal.
5. Data are transferred from JsonRecordReader to the console via Job.run() method. See how to compile and run data pipeline jobs

JsonRecordReader

JsonRecordReader is an input reader that can read records from an input JSON stream. A method JsonRecordReader.addRecordBreak tells the reader to return a new record using whatever fields have been assigned. This method is basically used to demarcate records.

Console output

-----------------------------------------------
0 - Record (MODIFIED) {
    0:[productId]:BIG_INTEGER=[100000000000000000000000000001]:BigInteger
    1:[productName]:STRING=[UltraWidget Pro]:String
    2:[price]:BIG_DECIMAL=[1499.99]:BigDecimal
    3:[availableStock]:BIG_INTEGER=[5000000000000000]:BigInteger
    4:[rating]:BIG_DECIMAL=[4.75]:BigDecimal
}

-----------------------------------------------
1 - Record (MODIFIED) {
    0:[productId]:BIG_INTEGER=[100000000000000000000000000002]:BigInteger
    1:[productName]:STRING=[MegaGadget X]:String
    2:[price]:BIG_DECIMAL=[2999.49]:BigDecimal
    3:[availableStock]:BIG_INTEGER=[3000000000000000]:BigInteger
    4:[rating]:BIG_DECIMAL=[4.9]:BigDecimal
}

-----------------------------------------------
2 - Record (MODIFIED) {
    0:[productId]:BIG_INTEGER=[100000000000000000000000000003]:BigInteger
    1:[productName]:STRING=[SmartDevice Z]:String
    2:[price]:BIG_DECIMAL=[799.95]:BigDecimal
    3:[availableStock]:BIG_INTEGER=[750000000000000]:BigInteger
    4:[rating]:BIG_DECIMAL=[4.6]:BigDecimal
}

-----------------------------------------------
3 records