Examples

All examples can be found on GitHub (https://github.com/NorthConcepts/DataPipeline-Examples).

  1. Buffer Records by Time Period or Count
  2. Capture Data Not Joined in a Lookup
  3. Compile and Run a Job
  4. Continue After an Error
  5. Convert a Single Source DataReader into Many
  6. Convert JSON to CSV
  7. Debug my Code
  8. Extract Bigrams, Trigrams, and Ngrams
  9. Generate a PDF
  10. Generate a Word Document
  11. Group Records by TimePeriod or Count
  12. Handle Exceptions
  13. Log Diagnostic Information
  14. Measure Data being Read and Written
  15. Measure Data Lineage Performance
  16. Measure Performance of Reader and Writer
  17. Obtain Statistics
  18. Open and Close Several Data Readers and Data Writers at Once
  19. Pipe a Writer to a Reader
  20. Profile Performance
  21. Read BigDecimal and BigInteger from an Excel file
  22. Read a Bloomberg Message File
  23. Read a CSV File
  24. Read a Fixed-width File / Fixed-length Record File
  25. Read a JSON Stream
  26. Read a Parquet File
  27. Read a Patient File
  28. Read a Simple JSON File
  29. Read a Simple XML File
  30. Read a Web Server Log
  31. Read an Avro File
  32. Read an XML File
  33. Read an XML File (2)
  34. Read and Write to an EventBus
  35. Read Big Decimal In JSON
  36. Read Emails
  37. Read From Amazon S3
  38. Read from a Database
  39. Read from an Excel File
  40. Read from Gmail
  41. Read from Java beans
  42. Read from JMS Queue
  43. Read from JMS Topic
  44. Read from Memory
  45. Read from MongoDB
  46. Read Google Analytics goal conversions
  47. Read Google Analytics Social Interactions
  48. Read Google Analytics views
  49. Read Google Calendar
  50. Read Google Contacts
  51. Read Google Gmail Messages
  52. Read JSON Records From File
  53. Read Selected Fields from a Parquet File
  54. Read using TimedReader
  55. Read XML Records From File
  56. Search for a Record
  57. Search Twitter for Tweets
  58. Serialize and Deserialize Data
  59. Serialize and Deserialize Records
  60. Throttle Data being Read
  61. Throttle Data being Written
  62. Use Multi Threading in a Single Job
  63. Use a Retrying Reader
  64. Use a Retrying Writer
  65. Use Data Lineage with CsvReader
  66. Use Data Lineage with ExcelReader
  67. Use Data Lineage with FixedWidthReader
  68. Use Data Lineage with JdbcReader
  69. Use Data Lineage with Lookup
  70. Use Data Lineage with ParquetReader
  71. Use Streaming Excel for Reading
  72. Use Streaming Excel for Writing
  73. Upsert Records to a Database Using Insert and Update
  74. Upsert Records to a Database Using Merge
  75. Upsert Records to MySql or MariaDB
  76. Upsert Records to Oracle
  77. Upsert Records to PostgreSql
  78. Upsert Records to Sybase
  79. Upsert Variable Field Records
  80. Write a CSV File to Database (1)
  81. Write a CSV File to Database (2)
  82. Write a CSV File to Fixed Width
  83. Write a Parquet File
  84. Write a Simple JSON File
  85. Write a Simple XML File
  86. Write a Sequence of Files by Record Count
  87. Write a Sequence of Files by Elapsed Time
  88. Write an Avro File
  89. Write an XML File Programmatically
  90. Write an XML File using FreeMarker Templates
  91. Write CSV To XML Using FreeMarker Templates
  92. Write HTML using FreeMarker Templates
  93. Write Key-Value Fields to MapWriter
  94. Write to Amazon S3 Using Multipart Streaming
  95. Write to a Database Using Custom Jdbc Insert Strategy
  96. Write to a Database Using Generic Upsert Strategy
  97. Write to a Database Using Merge Upsert Strategy
  98. Write to a Database Using Merge Upsert Strategy with Batch
  99. Write to a Database Using Multiple Connections
  100. Write to a Database Using Multi Row Prepared Statement Insert Strategy
  101. Write to a Database Using Multi Row Statement Insert Strategy
  102. Write to Excel
  103. Write to JMS Queue
  104. Write to JMS Topic
  105. Write to JSON Stream (simple)
  106. Write to JSON Stream Programmatically
  107. Write to Memory
  108. Write to MongoDB
  109. Write to Several Data Writers at Once
  110. Write to the Console
  111. Write to XML Stream (Simple)
Mobile Analytics