Examples

All examples can be found on GitHub (https://github.com/NorthConcepts/DataPipeline-Examples).

  1. Buffer Records by Time Period or Count
  2. Capture Data Not Joined in a Lookup
  3. Compile and Run a Job
  4. Continue After an Error
  5. Convert a Single Source DataReader into Many
  6. Convert JSON to CSV
  7. Debug my Code
  8. Extract Bigrams, Trigrams, and Ngrams
  9. Generate a PDF
  10. Generate a Word Document
  11. Group Records by TimePeriod or Count
  12. Handle Exceptions
  13. Log Diagnostic Information
  14. Measure Data being Read and Written
  15. Measure Data Lineage Performance
  16. Measure Performance of Reader and Writer
  17. Obtain Statistics
  18. Open and Close Several Data Readers and Data Writers at Once
  19. Pipe a Writer to a Reader
  20. Profile Performance
  21. Read BigDecimal and BigInteger from an Excel file
  22. Read a CSV File
  23. Read a Fixed-width File / Fixed-length Record File
  24. Read a JSON Stream
  25. Read a Patient File
  26. Read a Simple JSON File
  27. Read a Simple XML File
  28. Read a Web Server Log
  29. Read an Avro File
  30. Read an XML File
  31. Read an XML File (2)
  32. Read and Write to an EventBus
  33. Read Big Decimal In JSON
  34. Read Emails
  35. Read From Amazon S3
  36. Read from a Database
  37. Read from an Excel File
  38. Read from Gmail
  39. Read from Java beans
  40. Read from JMS Queue
  41. Read from JMS Topic
  42. Read from Memory
  43. Read from MongoDB
  44. Read JSON Records From File
  45. Read using TimedReader
  46. Read XML Records From File
  47. Search for a Record
  48. Search Twitter for Tweets
  49. Serialize and Deserialize Data
  50. Serialize and Deserialize Records
  51. Throttle Data being Read
  52. Throttle Data being Written
  53. Use Multi Threading in a Single Job
  54. Use a Retrying Reader
  55. Use a Retrying Writer
  56. Use Data Lineage
  57. Use Data Lineage with ExcelReader
  58. Use Data Lineage with FixedWidthReader
  59. Use Data Lineage with JdbcReader
  60. Use Data Lineage with Lookup
  61. Use Streaming Excel for Reading
  62. Use Streaming Excel for Writing
  63. Upsert Records to a Database Using Insert and Update
  64. Upsert Records to a Database Using Merge
  65. Upsert Records to MySql or MariaDB
  66. Upsert Records to Oracle
  67. Upsert Records to PostgreSql
  68. Upsert Records to Sybase
  69. Upsert Variable Field Records
  70. Write a CSV File to Database (1)
  71. Write a CSV File to Database (2)
  72. Write a CSV File to Fixed Width
  73. Write a Simple JSON File
  74. Write a Simple XML File
  75. Write a Sequence of Files by Record Count
  76. Write a Sequence of Files by Elapsed Time
  77. Write an Avro File
  78. Write an XML File Programmatically
  79. Write an XML File using FreeMarker Templates
  80. Write CSV To XML Using FreeMarker Templates
  81. Write HTML using FreeMarker Templates
  82. Write Key-Value Fields to MapWriter
  83. Write to Amazon S3 Using Multipart Streaming
  84. Write to a Database Using Custom Jdbc Insert Strategy
  85. Write to a Database Using Generic Upsert Strategy
  86. Write to a Database Using Merge Upsert Strategy
  87. Write to a Database Using Merge Upsert Strategy with Batch
  88. Write to a Database Using Multiple Connections
  89. Write to a Database Using Multi Row Prepared Statement Insert Strategy
  90. Write to a Database Using Multi Row Statement Insert Strategy
  91. Write to Excel
  92. Write to JMS Queue
  93. Write to JMS Topic
  94. Write to JSON Stream (simple)
  95. Write to JSON Stream Programmatically
  96. Write to Memory
  97. Write to MongoDB
  98. Write to Several Data Writers at Once
  99. Write to the Console
  100. Write to XML Stream (Simple)
Mobile Analytics