1. Home
  2. Documentation
  3. Examples
  4. All Examples

All Examples

  1. Add Array Fields to Entities DataPipeline Foundations Schema
  2. Add Attributes and Tags to Schemas DataPipeline Foundations Schema
  3. Add Calculated Fields to a Decision Table DataPipeline Foundations Decision Table Calculated Field
  4. Add Calculated Fields to a Decision Tree DataPipeline Foundations Calculated Field Decision Tree
  5. Add Indexes to Entities DataPipeline Foundations Schema
  6. Add Jira Comments Jira
  7. Add Nested Entities as Fields DataPipeline Foundations Schema
  8. Add Non-Persistent Data to Records and Fields In-Memory Out-of-band data
  9. Add Primary Keys to Entities DataPipeline Foundations Schema
  10. Add Relationship Between Entities DataPipeline Foundations Schema
  11. Add a Decision Table to a Pipeline DataPipeline Foundations Decision Table
  12. Add a Decision Tree to a Pipeline DataPipeline Foundations Decision Tree
  13. Add a Sequence Number Column Transform Calculated Field Assign Field
  14. Add a Sequence Number Column for Repeat Values Transform Calculated Field Assign Field
  15. Add a Sequence Number Column when Values Change Transform Calculated Field Assign Field
  16. Add my own Functions Customize Expression Language Assign Field Functions
  17. Automatically Map Data And Exclude Fields Declaratively Version 8.1.0 DataPipeline Foundations XML Declarative Data Mapping
  18. Blacklist and Whitelist Functions in DP Expression Language Customize Expression Language Functions
  19. Buffer Records by Time Period or Count Streaming data
  20. Build DataMappingPipeline Declaratively from Json DataPipeline Foundations Declarative Data Mapping JSON
  21. Build DataMappingPipeline Declaratively from XML DataPipeline Foundations XML Declarative Data Mapping
  22. Build DataMappingPipeline Programmatically DataPipeline Foundations Data Mapping
  23. Cache Lookup Values Lookup Transform JDBC Cache
  24. Capture Data Not Joined in a Lookup Lookup
  25. Capture Data that Failed Data Mapping Validation DataPipeline Foundations Data Mapping
  26. Compare Records Using Diff DataPipeline Foundations Diff
  27. Compare Schemas Using Diff DataPipeline Foundations Diff Schema
  28. Compile and Run a Job Job management CSV Fixed Width
  29. Compress a Parquet File Parquet
  30. Compress an Orc File ORC
  31. Conditionally Map Data Field from Source to Target DataPipeline Foundations Data Mapping
  32. Conditionally Transform Data Simple Transform Filter JSON Assign Field
  33. Conditionally Write to Multiple Files Transform Multi-writer Threading
  34. Conditionally map Data from Source to Target DataPipeline Foundations Data Mapping
  35. Configure AmazonS3FileSystem Using Client Amazon S3
  36. Configure the Insert Strategy used in JdbcMultiWriter Version 8.1.0 JDBC Multi-Writer JDBC Insert Strategy JDBC
  37. Continue After an Error Exceptions
  38. Convert JSON to CSV Simple CSV Conversion JSON
  39. Convert JSON to Parquet Parquet Conversion JSON
  40. Convert Unstructured JSON to Parquet Parquet Conversion JSON
  41. Convert a Single Source DataReader into Many XML Simple Conversion Avro Threading JSON
  42. Copy a Field Copy Field Transform
  43. Create Custom Pipeline Action Customize DataPipeline Foundations Pipeline
  44. Create Custom Pipeline Input DataPipeline Foundations Pipeline
  45. Create Custom Pipeline Output DataPipeline Foundations Pipeline
  46. Create Jira Issue Jira
  47. Create a JsonPipelineInput Declaratively From Xml DataPipeline Foundations XML Declarative Pipeline
  48. Create a JsonPipelineInput Declaratively from JSON DataPipeline Foundations Declarative Pipeline JSON
  49. Create a JsonPipelineInput Programmatically DataPipeline Foundations Pipeline
  50. Create an XmlPipelineInput Declaratively From Xml DataPipeline Foundations XML Declarative Pipeline
  51. Create an XmlPipelineInput Declaratively from Json DataPipeline Foundations Declarative Pipeline JSON
  52. Create an XmlPipelineInput Programmatically DataPipeline Foundations Pipeline
  53. Debug my Code Debug
  54. Declaratively Map Data DataPipeline Foundations Declarative Data Mapping
  55. Declaratively Map Data Using Positions DataPipeline Foundations Declarative Data Mapping
  56. Declaratively Map Data with Source and Target Schema DataPipeline Foundations Declarative Data Mapping Schema
  57. Declaratively Map XML Files DataPipeline Foundations XML Declarative Data Mapping
  58. Declaratively Set Default Values for Missing Data DataPipeline Foundations Declarative Schema
  59. Declaratively Transform Records using Schema DataPipeline Foundations Declarative Schema
  60. Delete Jira Issue Jira
  61. Detect Date and Time Patterns in Strings DataPipeline Foundations Analyze Data
  62. Detect Numbers in Strings DataPipeline Foundations Analyze Data
  63. Detect Primary Keys In a Dataset DataPipeline Foundations Schema Dataset
  64. Evaluate a Decision Table DataPipeline Foundations Decision Table
  65. Evaluate a Decision Table with Lookup DataPipeline Foundations Lookup Decision Table
  66. Evaluate a Decision Tree DataPipeline Foundations Decision Tree
  67. Evaluate a Decision Tree with Lookup DataPipeline Foundations Lookup Decision Tree
  68. Execute an Action in a Decision Table DataPipeline Foundations Decision Table
  69. Execute an Action in a Decision Tree DataPipeline Foundations Decision Tree
  70. Extract Bigrams, Trigrams, and Ngrams XML Transform Analyze Data
  71. Filter Columns with All Null Values DataPipeline Foundations Pipeline
  72. Filter Json Records in a Pipeline DataPipeline Foundations Pipeline Filter
  73. Filter Json Records in a Pipeline Declaratively from Json DataPipeline Foundations Declarative Pipeline Filter JSON
  74. Filter Json Records in a Pipeline Declaratively from Xml DataPipeline Foundations XML Declarative Pipeline Filter
  75. Filter Records Rules Transform Expression Language Filter
  76. Filter Twitter in Real Time Streaming data Filter Twitter
  77. Filter Twitter in Real Time Using v2 API Twitter2 Streaming data Filter
  78. Find The Minimum Maximum Average Sum Count Stats Transform CSV Conversion Aggregate STDOUT
  79. Generate Java Beans from a Database DataPipeline Foundations JDBC Code Generation
  80. Generate MySQL DDL Programmatically MySQL Code Generation
  81. Generate MySQL INSERT Statements Programmatically MySQL Database Code Generation
  82. Generate MySql DDL From CSV MySQL DataPipeline Foundations CSV Code Generation Schema
  83. Generate MySql INSERT Statements From CSV MySQL CSV Database Code Generation
  84. Generate Parquet Schema By Analyzing All Records Parquet CSV
  85. Generate Parquet Schema By Analyzing Initial Records Parquet CSV
  86. Generate PostgreSQL DDL From Excel DataPipeline Foundations MS Excel PostgreSQL Code Generation Schema
  87. Generate PostgreSQL DDL Programmatically PostgreSQL Code Generation
  88. Generate PostgreSql DDL From CSV DataPipeline Foundations CSV PostgreSQL Code Generation Schema
  89. Generate SQL Queries Programmatically MySQL Microsoft SQL Server JDBC PostgreSQL
  90. Generate Schema From JDBC DataPipeline Foundations JDBC Schema
  91. Generate Schema Java Code DataPipeline Foundations Code Generation Schema
  92. Generate Union and Sub-Select SQL Queries Programmatically MySQL Microsoft SQL Server JDBC PostgreSQL
  93. Generate a PDF XML Simple PDF
  94. Generate a Word Document Simple RTF JSON
  95. Generate toRecord And fromRecord Using The GenerateRecordSerializers Tool Code Generation Serialization
  96. Get a Query Param from a URL Transform Assign Field
  97. Group Records by TimePeriod or Count Aggregate
  98. Handle Exceptions Exceptions
  99. Handle Filtered Records Transform Fixed Width Filter Calculated Field Assign Field
  100. Handle Transforming Reader Failures Exceptions XML Transform Rename Field Filter
  101. Handle Transforming Writer Failures Exceptions Transform Conversion Fixed Width Filter Calculated Field
  102. Handle Validation Failures XML Validate Filter
  103. How To Read An XML File With A Byte Order Mark (BOM) XML
  104. How to Convert Tabular Data to Trees Using Aggregation Transform CSV Sort
  105. Infer Date Time from a CSV File DataPipeline Foundations Analyze Data Pipeline Dataset
  106. Insert Multiple Rows to Oracle Using Insert All Oracle Database Version 8.1.0 JDBC Insert Strategy JDBC
  107. Insert Multiple Rows to Oracle Using Select Union Oracle Database Version 8.1.0 JDBC Insert Strategy JDBC
  108. Join CSV Files CSV
  109. Load Snapshot of Dataset DataPipeline Foundations Pipeline
  110. Log Diagnostic Information Debug
  111. Log Job Progress Job management STDOUT
  112. Look Up Jira Transition IDs Jira
  113. Lookup Data in a Database Lookup Transform JDBC
  114. Lookup Data in any Data Reader Lookup Transform
  115. Lookup in a Flat File Lookup Transform
  116. Manipulate Fields Transform Conversion In-Memory
  117. Map Data Using Automatic Mapping And Exclude Fields Version 8.1.0 DataPipeline Foundations CSV Data Mapping
  118. Map Data from Source to Target DataPipeline Foundations Data Mapping
  119. Map Data from Source to Target in a Pipeline DataPipeline Foundations Data Mapping
  120. Map Data from Source to Target in a Pipeline with Validation DataPipeline Foundations Data Mapping
  121. Map Data from Source to Target with Lookup DataPipeline Foundations Data Mapping
  122. Map Data with Rule Based Validation DataPipeline Foundations Data Mapping
  123. Map Data with Schema Based Validation DataPipeline Foundations Data Mapping
  124. Map Records in Pipeline using Schema DataPipeline Foundations Pipeline Schema
  125. Map Records using Schema DataPipeline Foundations Schema
  126. Measure Data Lineage Performance Debug Data Lineage Performance
  127. Measure Data being Read and Written Meter
  128. Measure Performance of Reader and Writer Performance
  129. My Data Reader Customize
  130. My Data Writer Customize
  131. My Filter Customize Filter
  132. My Lookup Customize Lookup
  133. My Proxy Reader Customize Transform
  134. My Proxy Writer Customize Transform
  135. My Transformer Customize Transform
  136. Obtain Statistics Stats MS Excel Meter
  137. Open and Close Several Data Readers and Data Writers at Once JDBC
  138. Pipe a Writer to a Reader Pipe Push Threading
  139. Profile Performance Performance
  140. Read A CSV File using Multithreading CSV Threading
  141. Read An ORC file from Amazon S3 Amazon S3 ORC
  142. Read An ORC file from Amazon S3 using a Temporary File Amazon S3 ORC
  143. Read Big Decimal In JSON JSON
  144. Read BigDecimal and BigInteger from an Excel file MS Excel
  145. Read Concatenated JSON JSON
  146. Read Emails Email STDOUT
  147. Read Google Analytics Social Interactions Google
  148. Read Google Analytics goal conversions Google
  149. Read Google Analytics views Google
  150. Read Google Calendar Google
  151. Read Google Contacts Google
  152. Read Google Gmail Messages Email Google
  153. Read JSON Records From File JSON
  154. Read Jira Comments Jira
  155. Read Jira issues Jira
  156. Read Json Lines JSON Lines JSON
  157. Read Json Lines As Record JSON Lines JSON
  158. Read MailChimp Contacts MailChimp
  159. Read MailChimp Contacts Using the RESTful Client API MailChimp
  160. Read Parquet from Amazon S3 Parquet Amazon S3
  161. Read Parquet from Amazon S3 using a Temporary File Parquet Amazon S3
  162. Read Selected Fields from a Parquet File Parquet
  163. Read Selected Fields from an Orc File ORC
  164. Read Shopify Inventory Items E-commerce Shopify
  165. Read Shopify Inventory Levels E-commerce Shopify
  166. Read Shopify Orders E-commerce Shopify
  167. Read Shopify Store Locations E-commerce Shopify
  168. Read Trello Cards Trello
  169. Read Tweets from a User's Timeline Using v2 API Twitter2 STDOUT Twitter
  170. Read UUID from PostgreSQL Job management UUID JDBC PostgreSQL
  171. Read XML Records From File XML
  172. Read a Bloomberg Message File Bloomberg
  173. Read a CSV File CSV
  174. Read a Fixed-width File / Fixed-length Record File Fixed Width
  175. Read a JSON Stream Conversion XPath STDOUT JSON
  176. Read a Parquet File Parquet
  177. Read a Patient File Simple CSV
  178. Read a Set of Files CSV Batch
  179. Read a Simple JSON File Simple JSON
  180. Read a Simple XML File XML Simple
  181. Read a Tab Separated Values File CSV
  182. Read a Web Server Log Web Server Logs STDOUT
  183. Read an Avro File Avro
  184. Read an Orc File ORC
  185. Read an XML File XML XPath
  186. Read an XML File (2) XML Transform Conversion XPath JDBC
  187. Read and Transform Data in Parallel Transform Threading
  188. Read and Write CSV as String CSV
  189. Read and Write Kafka Events Kafka
  190. Read and Write Unsigned Numbers in Parquet Files Version 8.1.0 Parquet
  191. Read and Write to an EventBus EventBus Threading
  192. Read from Amazon S3 Amazon S3
  193. Read from CSV And Writer to Excel DataPipeline Foundations Pipeline
  194. Read from Gmail Email STDOUT
  195. Read from JMS Queue JMS In-Memory
  196. Read from JMS Topic JMS In-Memory
  197. Read from Java beans XPath Java Beans
  198. Read from Memory CSV In-Memory
  199. Read from MongoDB CSV STDOUT MongoDB
  200. Read from a Database JDBC
  201. Read from an Excel File MS Excel
  202. Read from database using OPINIONATED type mapping JDBC
  203. Read from database using ORIGINAL type mapping JDBC
  204. Read from database using STRICT type mapping JDBC
  205. Read using TimedReader Performance
  206. Remove Duplicates Transform Remove Duplicates (Dedup)
  207. Remove Fields Transform Exclude Fields
  208. Remove/Retain Duplicate Fields Transform Remove Duplicates (Dedup)
  209. Rename Duplicate Fields Transform Rename Field
  210. Rename Fields Containing Spaces and Symbols Transform Rename Field
  211. Rename Fields Quickly in Flat And Tabular Data Version 8.1.0 Fast Transform Rename Field Tabular Data Performance Flat Data
  212. Rename a Field Transform Rename Field
  213. Save and Load DataMapping to XML DataPipeline Foundations XML Data Mapping
  214. Save and Load DecisionTable to XML DataPipeline Foundations XML Decision Table
  215. Save and Load DecisionTree to XML DataPipeline Foundations XML Decision Tree
  216. Save and Load Diff to Record DataPipeline Foundations Diff Dataset
  217. Save and Restore Pipeline from JSON DataPipeline Foundations Pipeline
  218. Search Followers of a Twitter User Using v2 API Twitter2 STDOUT
  219. Search Twitter for Tweets Copy Field MS Excel Transform STDOUT Multi-writer Twitter
  220. Search Twitter for Tweets Using v2 API Twitter2 STDOUT
  221. Search for Jira Issues by Multiple IDs Jira
  222. Search for a Record Expression Language In-Memory Filter Search
  223. Select and Arrange Fields Include Fields Transform
  224. Serialize DataMapping containing special characters to XML DataPipeline Foundations XML Data Mapping Serialization
  225. Serialize RecordList to and from JSON JSON Serialization
  226. Serialize RecordList to and from XML XML Serialization Dataset
  227. Serialize and Deserialize Data Native CSV STDOUT Serialization
  228. Serialize and Deserialize Records STDOUT Serialization
  229. Set Default Values for Missing Data DataPipeline Foundations Schema
  230. Set a Calculated Field Programmatically Transform Calculated Field Assign Field
  231. Set a Calculated Field at Runtime Transform Expression Language Calculated Field Assign Field
  232. Set a Field Transform Assign Field
  233. Show Column Statistics DataPipeline Foundations Analyze Data Pipeline Dataset
  234. Show Data Mapping Model Problems DataPipeline Foundations Data Mapping
  235. Show Schema Model Problems DataPipeline Foundations Schema
  236. Show Unique Values in Column DataPipeline Foundations Analyze Data Pipeline
  237. Show the Columns and Tables of a Schema DataPipeline Foundations JDBC
  238. Skip First Line Using Limit Reader CSV
  239. Sort Records Transform Sort
  240. Specify Lookup Values Programmatically Lookup Transform
  241. Store Dataset In Memory DataPipeline Foundations Dataset
  242. Store Dataset on Disk DataPipeline Foundations Dataset
  243. Stream Record Fields Java Streams Streaming data
  244. Throttle Data being Read Throttle
  245. Throttle Data being Written Throttle
  246. Transform Records using Schema DataPipeline Foundations Schema
  247. Transition Jira Issue Jira
  248. Update Jira Issue Jira
  249. Upsert Records With Multiple JDBC Connections Version 8.1.0 Upsert JDBC Multi-Writer JDBC
  250. Upsert Records to MySql or MariaDB MySQL Upsert JDBC
  251. Upsert Records to Oracle Oracle Database Upsert JDBC
  252. Upsert Records to PostgreSql Upsert JDBC PostgreSQL
  253. Upsert Records to Sybase Microsoft SQL Server Upsert JDBC Sybase
  254. Upsert Records to a Database Using Insert and Update Upsert JDBC
  255. Upsert Records to a Database Using Merge Upsert JDBC
  256. Upsert Variable Field Records Upsert JDBC
  257. Use Data Lineage with CsvReader Debug Data Lineage CSV
  258. Use Data Lineage with ExcelReader MS Excel Debug Data Lineage
  259. Use Data Lineage with FixedWidthReader Debug Data Lineage Fixed Width
  260. Use Data Lineage with JdbcReader Debug Data Lineage JDBC
  261. Use Data Lineage with Lookup Lookup Debug Data Lineage
  262. Use Data Lineage with OrcReader Debug Data Lineage ORC
  263. Use Data Lineage with ParquetReader Parquet Debug Data Lineage
  264. Use Multi Threading in a Single Job Threading
  265. Use SchemaFilter to Validate Records in a Pipeline DataPipeline Foundations Schema
  266. Use Streaming Excel for Reading MS Excel
  267. Use Streaming Excel for Writing MS Excel CSV
  268. Use a Custom AWS S3 Endpoint with AmazonS3FileSystem Version 8.2.0 Amazon S3
  269. Use a Retrying Reader Retrying Avro
  270. Use a Retrying Writer Retrying Avro
  271. Use an EventBus DataPipeline Foundations EventBus
  272. Use an EventBus in a Pipeline EventBus Threading
  273. Validate Data using Entity Inheritance DataPipeline Foundations Validate Schema
  274. Validate Data using Nested Entities DataPipeline Foundations Validate Schema
  275. Validate Incoming Data Rules Transform Validate Expression Language
  276. Validate Record Fields DataPipeline Foundations Schema
  277. Validate Records using Fields and Rules DataPipeline Foundations Schema
  278. Validate Records using Rules DataPipeline Foundations Schema
  279. Validate Root Record Using Schema Validate Filter Schema
  280. Validate a Field DataPipeline Foundations Schema
  281. Validate a Value DataPipeline Foundations Schema
  282. Write A Parquet File Containing Spaces And Symbols In Column Names Parquet Transform CSV
  283. Write A Parquet File Using Database Table Schema Version 8.1.0 Parquet Database JDBC
  284. Write An ORC file to Amazon S3 using a Temporary File Amazon S3 ORC
  285. Write An Orc file to Amazon S3 Amazon S3 ORC
  286. Write Arrays and Nested Records to XML using FreeMarker Templates XML Freemarker XPath In-Memory Template
  287. Write CSV To XML Using FreeMarker Templates XML CSV Template
  288. Write Concatenated JSON
  289. Write Concatenated JSON With Nested Data Streaming data JSON
  290. Write Database View To Parquet File Version 8.1.0 Parquet Database JDBC
  291. Write Excel to Amazon S3 MS Excel Amazon S3
  292. Write HTML using FreeMarker Templates HTML In-Memory Template
  293. Write JSON Lines JSON Lines In-Memory JSON
  294. Write JSON Lines With Nested Data JSON Lines JSON
  295. Write Key-Value Fields to MapWriter In-Memory
  296. Write Parquet to Amazon S3 Parquet Amazon S3
  297. Write Parquet to Amazon S3 Using a Temporary File Parquet Amazon S3
  298. Write UUID to JSON file UUID JSON
  299. Write a CSV File to Database (1) CSV JDBC
  300. Write a CSV File to Database (2) CSV JDBC
  301. Write a CSV File to a Fixed Width / Fixed Length Record File CSV Fixed Width
  302. Write a JSON File using FreeMarker Templates Freemarker XPath In-Memory Template JSON
  303. Write a Parquet File Parquet
  304. Write a Parquet file using Schema Parquet
  305. Write a Parquet file using schema from EntityDef Parquet Schema
  306. Write a Parquet file using the schema derived from any dataset Parquet
  307. Write a Parquet file using the schema from a database query Version 8.1.0 Parquet JDBC
  308. Write a Sequence of Files by Elapsed Time Simple CSV
  309. Write a Sequence of Files by Record Count Simple CSV
  310. Write a Simple JSON File Simple JSON
  311. Write a Simple XML File XML Simple
  312. Write an Avro File Avro
  313. Write an Orc File ORC
  314. Write an XML File Programmatically XML In-Memory Template
  315. Write an XML File using FreeMarker Templates XML Freemarker XPath In-Memory Template
  316. Write my own Data Reader Customize
  317. Write my own Data Writer Customize
  318. Write my own Filter or Validator Customize Transform Validate Filter
  319. Write my own Lookup Customize Lookup Transform
  320. Write my own Transformer Customize Transform
  321. Write to A Data Writer using Tee Reader STDOUT Multi-writer
  322. Write to Amazon S3 Using Multipart Streaming Amazon S3
  323. Write to Excel MS Excel CSV
  324. Write to JMS Queue JMS In-Memory
  325. Write to JMS Topic JMS In-Memory
  326. Write to JSON Stream (Simple) Simple In-Memory JSON
  327. Write to JSON Stream Programmatically In-Memory Template JSON
  328. Write to Memory CSV In-Memory
  329. Write to MongoDB CSV MongoDB
  330. Write to Several Data Writers at Once MS Excel In-Memory Multi-writer
  331. Write to XML Stream (Simple) XML Simple In-Memory
  332. Write to a Database Using Custom Jdbc Insert Strategy JDBC Insert Strategy JDBC
  333. Write to a Database Using Generic Upsert Strategy Upsert JDBC
  334. Write to a Database Using Merge Upsert Strategy Upsert JDBC
  335. Write to a Database Using Merge Upsert Strategy with Batch Upsert JDBC
  336. Write to a Database Using Multi Row Prepared Statement Insert Strategy JDBC Insert Strategy JDBC
  337. Write to a Database Using Multi Row Statement Insert Strategy JDBC Insert Strategy JDBC
  338. Write to a Database Using Multiple Connections MS Excel JDBC Multi-Writer JDBC
  339. Write to the Console STDOUT

Tags

All Examples

Mobile Analytics