Examples

All examples can be found in the examples/src folder of the download or on GitHub (https://github.com/NorthConcepts/DataPipeline-Examples).

Get DataPipeline 5.0.0

Basic Input/Output

  1. Compile and Run a Job
  2. Continue After an Error
  3. Convert a Single Source DataReader into Many
  4. Debug my Code
  5. Generate a PDF
  6. Generate a Word Document
  7. Handle Exceptions
  8. Measure Data being Read and Written
  9. Obtain Statistics
  10. Open and Close Several Data Readers and Data Writers at Once
  11. Pipe a Writer to a Reader
  12. Read a CSV File
  13. Read a Fixed-width File / Fixed-length Record File
  14. Read a JSON Stream
  15. Read a Simple JSON File
  16. Read a Simple XML File
  17. Read a Web Server Log
  18. Read an Avro File
  19. Read an XML File
  20. Read an XML File (2)
  21. Read and Write to an EventBus
  22. Read Emails
  23. Read from a Database
  24. Read from an Excel File
  25. Read from Gmail
  26. Read from Java beans
  27. Read from JMS Queue
  28. Read from JMS Topic
  29. Read from Memory
  30. Read from MongoDB
  31. Search for a Record
  32. Search Twitter for Tweets
  33. Serialize and Deserialize Data
  34. Throttle Data being Read
  35. Throttle Data being Written
  36. Use Multi Threading in a Single Job
  37. Use a Retrying Reader
  38. Use a Retrying Writer
  39. Use Streaming Excel for Reading
  40. Use Streaming Excel for Writing
  41. Write a CSV File to Database (1)
  42. Write a CSV File to Database (2)
  43. Write a CSV File to Fixed Width
  44. Write a Simple JSON File
  45. Write a Simple XML File
  46. Write an Avro File
  47. Write an XML File Programmatically
  48. Write an XML File using FreeMarker Templates
  49. Write CSV To XML Using FreeMarker Templates
  50. Write HTML using FreeMarker Templates
  51. Write to a Database Using Multiple Connections
  52. Write to Excel
  53. Write to JMS Queue
  54. Write to JMS Topic
  55. Write to JSON Stream (simple)
  56. Write to JSON Stream Programmatically
  57. Write to Memory
  58. Write to MongoDB
  59. Write to Several Data Writers at Once
  60. Write to the Console
  61. Write to XML Stream (Simple)
  62. Buffer Records by Time Period or Count
  63. Extract Bigrams, Trigrams, and Ngrams
  64. Group Records by TimePeriod or Count
  65. Read From Amazon S3
  66. Read JSON Records From File
  67. Read XML Records From File
  68. Write to Amazon S3 Using Multipart Streaming

Data Manipulation

  1. Add Nonpersistent Data to Records and Fields
  2. Cache Lookup Values
  3. Conditionally Transform Data
  4. Copy a Field
  5. Filter Records
  6. Filter Twitter in Real Time
  7. Find The Minimum Maximum Average Sum Count
  8. Handle Filtered Records
  9. Handle Transforming Reader Failures
  10. Handle Transforming Writer Failures
  11. Handle Validation Failures
  12. Lookup Data in a Database
  13. Lookup Data in any Data Reader
  14. Manipulate Fields
  15. Remove Duplicates
  16. Remove Fields
  17. Rename a Field
  18. Select and Arrange Fields
  19. Set a Calculated Field at Runtime
  20. Set a Calculated Field Programmatically
  21. Set a Field
  22. Sort Records
  23. Specify Lookup Values Programmatically
  24. Validate Incoming Data
  25. Add a Sequence Number Column
  26. Add a Sequence Number Column when Values Change
  27. Add a Sequence Number Column for Repeat Values
  28. Get a Query Param from a Url
  29. Conditionally Write to Multiple Files
  30. Read a Set of Files
  31. Log Job Progress

Customization

  1. Add my own Functions
  2. My Data Reader
  3. My Data Writer
  4. My Filter
  5. My Lookup
  6. My Proxy Reader
  7. My Proxy Writer
  8. My Transformer
  9. Write my own Data Reader
  10. Write my own Data Writer
  11. Write my own Filter or Validator
  12. Write my own Lookup
  13. Write my own Transformer

Tags

Avro (5)
Cache (1)
CSV (15)
Debug (1)
Email (2)
Filter (10)
HTML (1)
JDBC (8)
JMS (4)
JSON (9)
Lookup (5)
Meter (2)
Native (1)
PDF (1)
Pipe (1)
Push (1)
RTF (1)
Rules (2)
Search (1)
Simple (10)
Sort (1)
Stats (2)
STDOUT (10)
XML (13)
XPath (5)
Get DataPipeline 5.0.0
Mobile Analytics