Handle Exceptions

Updated: Jun 20, 2023

View on GitHub

This example specializes in catching exceptions and errors that occur during data processing, ensuring smooth and reliable data operations. It provides a robust mechanism for identifying and handling various types of errors, including those related to data formatting, dataset issues, or other unexpected errors. When used within applications that deal with data processing, the example can help prevent application crashes or unexpected behavior, providing users with informative error messages for troubleshooting and resolving issues.

Input CSV File

A badly formatted CSV file is intentionally used so as to produce the exception. i.e. it has an opening quote " before Butler without a closing one.

Account,LastName,FirstName,Balance,CreditLimit,AccountCreated,Rating
101,Reeves,Keanu,9315.45,10000.00,1/17/1998,A
312,"Butler,Gerard,90.00,1000.00,8/6/2003,B
868,Hewitt,Jennifer Love,0,17000.00,5/25/1985,B
761,Pinkett-Smith,Jada,49654.87,100000.00,12/5/2006,A
317,Murray,Bill,789.65,5000.00,2/5/2007,C

 

Java Code Listing

package com.northconcepts.datapipeline.examples.cookbook;

import java.io.File;

import org.apache.log4j.Logger;

import com.northconcepts.datapipeline.core.DataEndpoint;
import com.northconcepts.datapipeline.core.DataException;
import com.northconcepts.datapipeline.core.DataReader;
import com.northconcepts.datapipeline.core.NullWriter;
import com.northconcepts.datapipeline.csv.CSVReader;
import com.northconcepts.datapipeline.job.Job;

public class HandleExceptions {
    
    public static final Logger log = DataEndpoint.log; 

    public static void main(String[] args) throws Throwable {
        try {
            DataReader reader = new CSVReader(new File("example/data/input/bad-credit-balance-01.csv"))
                .setFieldNamesInFirstRow(true);
            
            Job.run(reader, new NullWriter());
        } catch (DataException e) {
            log.error(e, e);
//            log.info(e.get("CSVReader.lineText"));
//            log.info(e.get("CSVReader.column"));
//            log.info(e.get("TextReader.line"));
//            log.info(e.getRecord());
//            throw e;
        }
    }

}

 

Code Walkthrough

1. Logger object is declared from the log variable of DataEndpoint.
2. CSVReader instance is created corresponding to the input file bad-credit-balance-01.csv.
3. setFieldNamesInFirstRow(true) is used to specify that the first row of the dataset is the field names, not the actual data.
4. Job.run() is used to transfer the data from the reader to the NullWriter().
5. A DataException is captured in the catch block.
6. The Logger displays the exception message on the console using error() method.

 

Console Output

00:26:45.876 [main] ERROR com.northconcepts.datapipeline - com.northconcepts.datapipeline.core.DataException: unterminated string: expected double quote ("), found end-of-line (\n)
-------------------------------
CSVReader-1.allowMultiLineText=[false]
CSVReader-1.allowQuoteInField=[false]
CSVReader-1.bufferSize=[0]
CSVReader-1.closedOn=[null]
CSVReader-1.column=[43]
CSVReader-1.description=[null]
CSVReader-1.endingQuote=["]
CSVReader-1.exhausted=[false]
CSVReader-1.fieldNames=[Account,LastName,FirstName,Balance,CreditLimit,AccountCreated,Rating]
CSVReader-1.fieldNamesInFirstRow=[true]
CSVReader-1.fieldSeparator=[,]
CSVReader-1.file=[example\data\input\bad-credit-balance-01.csv]
CSVReader-1.firstRow=[false]
CSVReader-1.id=[1]
CSVReader-1.lastRecord=[Record {
    0:[Account]:STRING=[101]:String
    1:[LastName]:STRING=[Reeves]:String
    2:[FirstName]:STRING=[Keanu]:String
    3:[Balance]:STRING=[9315.45]:String
    4:[CreditLimit]:STRING=[10000.00]:String
    5:[AccountCreated]:STRING=[1/17/1998...]
CSVReader-1.lastRow=[-1]
CSVReader-1.line=[3]
CSVReader-1.lineSeparators=[\n, \r\n, \r]
CSVReader-1.lineageSupported=[true]
CSVReader-1.name=[CSVReader-1]
CSVReader-1.openedOn=[2022.05.25-14:09:53.018]
CSVReader-1.parser.cache=[312,"Butler,Gerard,90.00,1000.00,8/6/2003,B]
CSVReader-1.recordCount=[1]
CSVReader-1.saveLineage=[false]
CSVReader-1.skipEmptyRows=[false]
CSVReader-1.startingQuote=["]
CSVReader-1.startingRow=[0]
CSVReader-1.state=[OPENED]
CSVReader-1.thread=[main]
CSVReader-1.timestamp=[2022.05.25-14:09:53.155]
CSVReader-1.trimFields=[true]
LineParser.cache=[312,"Butler,Gerard,90.00,1000.00,8/6/2003,B]
LineParser.cacheIndex=[43]
LineParser.column=[43]
LineParser.line=[2]
char=[end-of-line (\n)]
printable char=[end-of-line (\n)]
record=[Record (MODIFIED) {
    0:[Account]:STRING=[312]:String
    1:[LastName]:STRING=[null]
    2:[FirstName]:STRING=[null]
    3:[Balance]:STRING=[null]
    4:[CreditLimit]:STRING=[null]
    5:[AccountCreated]:STRING=[null]
    6:[Rating]:STRING=[null]
}
]
-------------------------------
com.northconcepts.datapipeline.core.DataException: unterminated string: expected double quote ("), found end-of-line (\n)
	at com.northconcepts.datapipeline.core.LineParser.match(LineParser.java:393) ~[northconcepts-datapipeline-small-business-7.2.0-SNAPSHOT.jar:?]
	at com.northconcepts.datapipeline.csv.CSVReader.matchString(CSVReader.java:478) ~[northconcepts-datapipeline-small-business-7.2.0-SNAPSHOT.jar:?]
	at com.northconcepts.datapipeline.csv.CSVReader.matchValue(CSVReader.java:430) ~[northconcepts-datapipeline-small-business-7.2.0-SNAPSHOT.jar:?]
	at com.northconcepts.datapipeline.csv.CSVReader.fillRecord(CSVReader.java:394) ~[northconcepts-datapipeline-small-business-7.2.0-SNAPSHOT.jar:?]
	at com.northconcepts.datapipeline.core.AbstractReader.readImpl(AbstractReader.java:144) ~[northconcepts-datapipeline-small-business-7.2.0-SNAPSHOT.jar:?]
	at com.northconcepts.datapipeline.core.TextReader.readImpl(TextReader.java:103) ~[northconcepts-datapipeline-small-business-7.2.0-SNAPSHOT.jar:?]
	at com.northconcepts.datapipeline.core.DataReader.read(DataReader.java:172) ~[northconcepts-datapipeline-small-business-7.2.0-SNAPSHOT.jar:?]
	at com.northconcepts.datapipeline.core.AbstractReader.read(AbstractReader.java:121) ~[northconcepts-datapipeline-small-business-7.2.0-SNAPSHOT.jar:?]
	at com.northconcepts.datapipeline.job.Job.run(Job.java:623) ~[northconcepts-datapipeline-small-business-7.2.0-SNAPSHOT.jar:?]
	at com.northconcepts.datapipeline.job.Job.run(Job.java:52) ~[northconcepts-datapipeline-small-business-7.2.0-SNAPSHOT.jar:?]
	at com.northconcepts.datapipeline.examples.cookbook.HandleExceptions.main(HandleExceptions.java:29) [main/:?]