1. Home
  2. Documentation
  3. DataPipeline Manager
  4. How to Save Hourly Stock Prices to a Daily File

How to Save Hourly Stock Prices to a Daily File

The example will show you how to create an hourly job to load stock prices from Google finance and save them to a daily file. 

You should read about how to create a simple job before continuing with this.

You'll need to download a copy of the Data Pipeline framework to compile and run this job.

1. Create the job factory class

In the how to create a simple job example, you created a class implementing java.lang.Runnable and registered it as a job in your project.  This example will go a step higher and implement a com.northconcepts.datapipeline.manager.client.JobFactory.  The factory approach lets you return different instances of Runnable depending on the situation.  In this case the Runnable returned will be a DataPipeline Job instance ― which also happens to implement Runnable.

You can read about the details of this code in the Read a JSON Stream example.  The important things to know are:

  1. The factory's createJob() method is called each time the job is run.
  2. The factory uses the symbols and file parameters to build the job instance.

2. Create a project

The next step is to define your project.xml file.


Job parameters are expressions that get evaluated just before each job is run.  See the Data Pipeline Expression Language page to understand which types of expressions and functions are supported.  For now, just keep in mind:
  1. String values must be quoted with double or single quotes or they will be treated as variables.
  2. Slashes must be escaped the same as in Java string literals.

3. Deploy the project

  1. Compile and jar the job class you created in the first step
  2. Create a folder for your new project inside <workspace-folder>/projects/.  For example: c:\workspace\projects\hello-world-project\.
  3. Place your project.xml file inside your new project folder.
  4. Create a lib folder under your new project folder.
  5. Place your project's jar inside the new lib folder.
  6. Place the Data Pipeline jar (NorthConcepts-DataPipeline-4.1.0.jar) inside the new lib folder.
Mobile Analytics