Serialize and Deserialize Records

This example shows how to convert DataPipeline records into byte streams and vice versa. This can be used to backup and restore data, transmit sensor data from IoT devices, or stage incoming financial transactions regardless of format. 

Java Code Listing

package com.northconcepts.datapipeline.examples.cookbook;

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.math.BigInteger;

import com.northconcepts.datapipeline.core.Record;

public class SerializeAndDeserializeRecords {

    public static void main(String[] args) throws Throwable {
        Record record1 = new Record();
        record1.setField("name", "John Wayne");
        record1.setField("balance", "12345678901234567890");

        Record record2 = new Record();
        record2.setField("name", "Peter Parker");
        record2.setField("balance", new BigInteger("98765432109876543210"));

        System.out.println(record1);
        System.out.println(record2);

        ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
        ObjectOutputStream outputStream = new ObjectOutputStream(byteArrayOutputStream);
        outputStream.writeObject(record1);
        outputStream.writeObject(record2);
        outputStream.close();

        byte[] bytes = byteArrayOutputStream.toByteArray();
        System.out.println("Bytes written: " + bytes.length);

        System.out.println("Serialization of records completed");

        ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes);
        ObjectInputStream inputStream = new ObjectInputStream(byteArrayInputStream);
        record1 = (Record) inputStream.readObject();
        record2 = (Record) inputStream.readObject();
        inputStream.close();

        System.out.println(record1);
        System.out.println(record2);

        System.out.println("Deserialization of records completed");
    }

}

Code Walkthrough

1. First, a Record is created to store fields as a key-value pair.
2. record1.setField() creates a field with the name specified in the first parameter and the value specified in the specified second parameter .
3. ByteArrayOutputStream is created to stream the bytes of serialized objects.
4. ObjectOutputStream is created to write Java objects to Java's OutputStream.
5. outputStream.writeObject() converts records into a stream of bytes.
6. The bytes are then stored into byte[] bytes array by invoking toByteArray() method on the byteArrayOutputStream object.
7. ByteArrayInputStream is created to stream bytes which are stored in bytes array.
8. ObjectInputStream is created to read a stream of bytes from byteArrayInputStream object and convert them back to an actual Java object using inputStream.readObject().
9. inputStream.readObject() returns a value of type Object and should be cast to Record type.

Console output

Record (MODIFIED) {
    0:[name]:STRING=[John Wayne]:String
    1:[balance]:STRING=[12345678901234567890]:String
}

Record (MODIFIED) {
    0:[name]:STRING=[Peter Parker]:String
    1:[balance]:BIG_INTEGER=[98765432109876543210]:BigInteger
}

Bytes written: 674
Serialization of records completed
Record (MODIFIED) {
    0:[name]:STRING=[John Wayne]:String
    1:[balance]:STRING=[12345678901234567890]:String
}

Record (MODIFIED) {
    0:[name]:STRING=[Peter Parker]:String
    1:[balance]:BIG_INTEGER=[98765432109876543210]:BigInteger
}

Deserialization of records completed