Service Library and Reusability

ActiveBatch promotes the concept of reusability. The idea is that engineers write code in many different forms. It would be great to share that code for different uses without having to modify it and place the code in a different form. For example, if a SQL Stored Procedure contains logic that can be reused, it seems crazy to have to recode the logic in another format just to be usable.

This chapter will examine the reusability aspects of code and logic using a number of example technologies. To begin we’ll examine the Service Library object. The Service Library object is the main implementation for gathering logic (or methods) from a number of discrete sources and then exporting those methods into reusable job steps.

 

The first technology we will examine is Web Services. Web Services like most programmatic interfaces can be simple or quite complex. In this example, we want to look up a company’s stock price using its ticker symbol.

 

Creating a Service Library Object – Web Services

 

To create a Service Library object, right click on the Objects folder and select New… Service Library.

The name of the Service Library object will be StockLookup.  Next click on Properties to declare the Web Service.

 

 

Every Web Service provides a WSDL document that describes the various interfaces of the Web Service (in fact that’s what WSDL stands for “Web Services Description Language”). For this web service, we’ll use the following URL that points to the WSDL document we want: http://ws.cdyne.com/delayedstockquote/delayedstockquote.asmx?wsdl. That data is entered into the WSDL Location property. With a valid specification entered, please press the Load button to load and validate the WSDL information. If the data is entered properly you should see Binding information appear in that part of the window. In this case, “DelayedStockQuote” is the service and “DelayedStockQuoteSoap” is the port. To see the actual methods (and their provided descriptions) that this web service exports, press the Refresh button. The three (3) methods you see above are provided by this particular web service.

 

Press the Save and Close button to define the Service Library object.

 

Note: The Web Service mentioned above, at the time of this writing, offered a free limited stock ticker price facility. The free but limited Web Service does have a counter as to how many times you can look up a stock price.

 

Using the Service Library

 

Next, we’ll create our customary plan to house our Web Services jobs. Right click on UsersGuideCaseStudies and select New… Plan. The name of our new plan will be ServiceLibraryPlan. Next, click on the Service Libraries category. This is where we will associate our StockLookup service library object.

 

 

Clicking the Associate button causes a Browse dialog to appear. Look for the Objects folder and expand the folder to reveal the StockLookup Service Library object. Click on the checkbox to enable the object for inclusion. The image above shows the service library object now associated with the plan. This means that any underlying plans or jobs can use this defined web service.

 

Please click the Save and Close button to create the plan. The navigation tree should look similar to this:

 

 

Creating a Web Services Job

 

Now we’re ready to create our first Web Services job. The initial portion is the same as usual. Right click on plan ServiceLibraryPlan and select New… Job. Please name the job QuoteLookup. We’ll create a constant variable, “symbol”, that represents the symbol of the company we want to look up the stock price for. In our example we’ll use MSFT for Microsoft as the ticker symbol. Note: Just because the variable is defined as “Constant” doesn’t mean that it can’t be changed. “Constant” simply means that the data source is entered manually or programmatically.

 

 

Now we’re ready to start defining the Job Properties portion of our job. Please click on Job Properties and set the Job Type property to Jobs Library.

 

 

If you scroll to the bottom of the left side portion of the editor you’ll see the new Job Library addition. Service Libraries appear as additional Job Library steps and more specifically the methods each Service Library object exports. Jobs only see those Service Libraries that have been associated with their job or parent plans. If you remember we associated the service library object “StockLookup” with our ServiceLibraryPlan plan (of which this job is a part).

 

The method “GetQuote” allows us to specify a ticker symbol and retrieve the latest stock price. To place that method or job step in our newly created job, please drag GetQuote from the left side tool bar and drop it in the “steps” bubble.

 

 

As with any Web Service or Job Step it is important to know the properties of the job step. In this case, you will need to click on the “request” property and select GetQuoteRequest. Please expand the request property to reveal two subproperties. LicenseKey should be set to zero (0) and StockSymbol should be an ActiveBatch string substitution sequence for our company ticker symbol. ${symbol} will represent the company symbol.

 

You’ll notice an icon at the bottom of this step. This icon indicates that this step is expected to return a status and/or data. Many job steps do not return data so the icon may not always be present. Since we’re invoking a Web Service that returns data we expect to use the leftmost icon. Please click on the icon. You should see something similar to the following:

 

 

This figure shows what is returned by the Web Service. This structure is returned in GetQuoteResult. The fully qualified execution variable name is GetQuote.ReturnValue.GetQuoteResult.field where field is one of the above values.

For example, %{GetQuote.ReturnValue.GetQuoteResult.DayHigh} would return today’s high price for this company’s stock; %{GetQuote.ReturnValue.GetQuoteResult.CompanyName}. The Log step lets us see the data when we run the job.

 

 

By dragging General/Log step we will be able to write the data to our job’s log file. The Log step takes a single property: the data we want to write. We’ve entered “Company: ${symbol} Today’s high is:”. As we start to enter %{ a variable helper will pop-up to help us complete a potentially long variable name. Clicking on the dropdown shows the entire text that’s present.

 

Please click the OK button to save the job steps we’ve created.

 

Now click the OK button to save the job we’re creating.

 

To test the new Job, right-click on QuoteLookup and select Trigger.

 

 

Assuming the instance runs successfully, it will appear in the Instances pane, right-click on the job instance (QuoteLookup) and select View Log. The portion we’re interested in appears below.

 

 

Note the string substitution for the full company name and stock price.

 

Creating a Service Library – REST

 

RESTful web services are a different type of web service. REST based web services do not have a WSDL to describe the data contract for the methods deployed by a particular service. REST services are described largely by documentation. In other words, the developer describes the methods that make up the REST service and then describes the requirements for each method in terms of web operation constructs. Every REST service consists of the following optional areas: Headers and Body. The Response area is required to determine the status of the method that was invoked. Each method uses those areas as well as a web verb (i.e. GET, PUT, POST) and an Endpoint for the method. Any of these areas or properties can be described and used as required by the developer. This also includes authentication.

 

Let’s look at a REST example. For this example, we’ll use Google’s Maps REST service. Google Maps provides many methods or operations but for this example we’ll describe and use just one: GetDirections.

 

To describe a REST service to ActiveBatch we’ll create a new Service Library object. Right-click on the Objects folder within UsersGuideCaseStudies and create the new Service Library object named GoogleMaps. After naming the object click on Properties.

 

 

Before we describe what you’re looking at, it’s important to understand that all the information being presented came from Google’s Map API documentation. There is no external metadata file for ActiveBatch to read (unlike the WSDL of the Web Services).

 

First, we select the REST Adapter. Once selected, the display will change to the REST areas and properties. The Common URL and Category Name are common to all related REST operations. The Category Name is used to classify the operations of this REST service under a single category which is displayed in the Jobs Library canvas. In this example, the Category Name will be “Google Maps”. The Common URL (if specified) is appended to the Endpoint property to form a full URL specification. For Google Maps different endpoints can be specified however they all begin with https://maps.googleapis.com. So that value is specified in the Common URL. We’re now ready to add our operation. Under the Operations section, the first icon is a plus sign which when clicked allows you to add an operation. In this example, we’ll name the operation “GetDirections”.

 

Each operation requires a web verb, endpoint and one or more of the web areas noted. For GetDirections, a GET verb is to be used. The Endpoint is the rest (or whole, if Common URL is not specified) of the URL specification in addition to any parameters required by the Operation. In this example, the endpoint is:

 

/maps/api/directions/xml?origin=%(origin)&destination=%(destination)&key=%(apiKey.Password)

 

This means the resultant endpoint will be:

 

https://maps.googleapis.com/maps/api/directions/xml?origin=%(origin)&destination=%(destination)&key=%(apiKey.Password)

 

The parameters are those required by the GetDirections operation. In this case, origin, destination and a Google API key that is used for authentication.

 

This particular operation does not require any Headers or a Body. The variables tab is used to collect the information required by the operation. Since this operation will ultimately become a job step, the variables specified will become properties of the job step for this operation.

 

 

To add a variable, click the “Add Variable” button. Note that currently the control does not allow any reordering so please create the variables in the order you would like to see them listed in the job step. For this operation we will create three (3) variables; Origin, Destination and APIKey. The data type for Origin and Destination are strings. We’ve also given each variable a default value (which is optional). The data type for APIkey is credential. This means that a User Account object is expected to hold any confidential information and is not to be disclosed.

 

The next section describes the Response area. REST services support two (2) formats: JSON and XML. Rather than build what could be a complex schema by hand, ActiveBatch provides an “Import…” button. You simply copy-and-paste a sample Response block provided by the developer of the REST method’s documentation into the Import block. For example:

 

 

The example above shows an expected Response block described in XML. ActiveBatch can auto-detect the format of the block entered. The Service Library REST importer will analyze the Response block provided. If the import is successful, a Response block schema will be created.

 

 

This is a partial schema that is shown due to the overall length of the structure that is passed back from this operation. Click OK to save the Service Library. Now we’ll proceed to use it.

 

User Account for REST usage

 

Now that we’ve created the REST Service Library object we need to create our Plan and Jobs that will be using the REST method. As mentioned previously, REST allows each developer to decide on their own authentication scheme. For Google Maps, an API key is needed. The API key is considered confidential so the use of a User Account object makes the most sense.

 

So please right-click on the Objects folder (within the folder UsersGuideCaseStudies) and select New…User Account. We’ll name this object “Google API Key”. Select Properties.

 

 

Since the API key is all text, we’ll select the Username/Password credential type. Since Authentication in REST is left to the developer there is a possibility that a password provided by ActiveBatch could be sent in the clear. As a result, you must set the sensitivity of the credential to “Low”. This allows ActiveBatch to provide this credential to the REST job step at the time of use. The API key will be stored in the Password property so the Username property is basically of comment value. So let’s pick a descriptive Username of “APIKey”.

 

You can now click the OK button to create this object. Now, let’s move on to the Plan and Jobs that will use this REST method.

 

REST Plan and Jobs

 

Please create a new Plan named GetShippingDirections under the Plan ServiceLibraryPlan. This Plan will contain several variables used by the underlying jobs. Two (2) Jobs will be used: GetCustomerAddress and GetDirections.

GetCustomerAddress will perform a database lookup of the desired customer and obtain the address. That address will then be passed to GetDirections which will perform the Google Maps “GetDirections” method and return the driving directions from the “origin” to the customer’s address. We could shorten this example by statically coding the necessary variables but we think this longer example continues to illustrate variable data passing.

 

So let’s first create the “GetShippingDirections” plan as indicated. The following variables should be specified.

 

 

The Database Server and Database will be same as we created in CaseStudy4. “Thomas Jones” is one of the entries in that database. We’ve left the “DeliveryDriverEmail” blank. If you intend to use Email to send the directions, please make sure to enter a valid e-mail specification. CustomerName and DeliveryDriverEmail should be marked as Required. The others can be left with their default.

 

Next, select Service Libraries. We will now associate our GoogleMaps Service Library with this Plan. This means that any Job placed within the Plan will have implicit access to that REST service method. Click the Associate button and look for the GoogleMaps object within the Objects folder.

 

 

Next, using either Map View or the Object Navigation tree, please create a Job within the GetShippingDirections Plan named GetCustomerAddress. The Job will use the Job Type of Jobs Library Job. Please run the Jobs Library Editor and create the following job steps.

 

 

 

These are all the steps for this job. For clarity the “Connection String” is Provider=SQLOLEDB.1;Initial

Catalog=${CustomerDatabase};Data Source=${DatabaseServer};Integrated Security=SSPI;Persist Security Info=False;

 

This job will look up the desired customer’s name in the database and retrieve the address (which is contained in separate fields). In “Expression_2” job step, the ConCat function is used to concatenate the various address fields into one variable (FullAddress). That variable is set into an active variable named “Destination”.

 

Next, using either Map View or the Object Navigation tree, please create another Job within the GetShippingDirections Plan named GetDirections. Click on the Variables tab and ensure these variables are defined.

 

 

Note: Destination is included only to alert the Job Author that this variable will be used. If this variable is omitted, it would be created as a result of the “SetVariable” job step in GetCustomerAddress. The Job will use the Job Type of Jobs Library Job. Please run the Jobs Library Editor and create the following job steps.

 

 

 

1. The Google Map GetDirections method is the first job step used. You should notice that the parameters of the job step are the same “variables” as specified in the REST Service Library for that operation. The API Key is a credential so the … helper will enumerate User Account objects. Select the “Google API Key” objects within the Objects folder. The Origin and Destination parameters are set by active variables of the same name. Note that the job step, as expected, returns data on a successful invocation.

2. The Expression job step initializes two (2) new Execution Variables, Count and Directions, that will be used in later steps.

3. The data that is passed back by the GetDirections job step is a collection of direction elements. Note the equal sign preceding the execution variable GetDirections.ReturnValue.DirectionsResponse.route.leg. This means that we need to access the collection represented by that variable. The ForEachItem step allows you to obtain each element of the collection and iterate with that data. The individual element is stored in “step” (context name).

4. The IF branch checks that the element actually contains driving instructions (Step.html_instructions != null). If so, the embedded Expression step contains two (2) assignment statements. The first uses the ActiveBatch Function ConCat to concatenate the different parts of the instructions. The second increments Count (or the count of driving instructions). You may also notice the <br> (in addition to the carriage-return/line feed combination). That’s because we will be sending an HTML formatted e-mail.

5. Once the iteration is complete, you have a choice. You can enable or disable the Log step to see the directions OR you can enable or disable the E-Mail step. The Message body contains an example of sending an HTML formatted e-mail with the directions.

   Once you’ve completed the job steps, click OK to create the GetDirections job step.

    

   

   
   Next, using Map View, draw a successful completion trigger from GetCustomerAddress to GetDirections.

   You’re now ready to Trigger the GetShippingDirections Plan, so right-click on that object and select Trigger.

 

 

Remember that we marked the above variables as required. This means we have an opportunity to change them. We will need to change the DeliveryDriverEmail variable to an acceptable value. Click OK to trigger the Plan (and the underlying Jobs).

 

 

All the instances were successful.

 

Here’s an extract of the e-mail.

 

 

Note: We could have added mileages and other data that is returned by the REST service.

 

Creating a Service Library – API (.NET Assembly)

 

Just as with the previous Web Services and REST examples, ActiveBatch is able to leverage your existing .NET assemblies. The advantage is that without refactoring your code, you’re able to use that logic as part of an ActiveBatch workflow.

 

Let’s create a new Service Library named MathDotNetLibrary within the Objects folder. Select Properties and then specify the API Adapter from the dropdown. Click the Configure button and select NetAssemblyAdapterEndpoint.

 

 

In this example, we’ll be using a .NET assembly that contains two (2) functions: SolvePythagoreanTheorem and GetLeg.

GetLeg will return a random value between 1 and 20. GetLeg will be called twice to obtain 2 legs of the triangle. SolvePythagoreanTheorem will accept the 2 legs as input and solve for the third leg.

 

 

In reality every .NET assembly contains other functions that may be of no use. To select the functions/methods you want, select the function in the large pane and click the Add button to add them to the Selected Operations area. That is what we did with the two named functions. Click the Save and Close button to save the new Service Library.

 

Create a Plan – API (.NET Assembly)

 

With the API Service Library in place, we can create new Jobs that will use that .NET Assembly methods. First, let’s create a Plan for the Service Library association. The Plan should be named MathAPI within the ServiceLibraryPlan. Create the Plan and select Service Libraries. Click the Associate button and select MathDotNetLibrary within the Objects folder.

 

 

When we create our Jobs the Service Library will already be associated and appear in our Jobs Library canvas.

 

Create Jobs – API (.NET Assembly)

 

Now we’re ready to create our Jobs; one for the GetLeg method and one for the SolvePythagoreanTheorem method. Both Jobs will have their Job Type set to “Jobs Library”.

 

Right-click on the MathAPI plan and select New… Job. Name the Job GetLegs. Select Job Properties and change the Job Type as previously indicated. Enter the following steps:

 

 

The most important aspect of this job is the FOR loop (for 2 legs) and the GetLeg step which returns a random number. The last step assigns the 2 legs to Leg1 and Leg2 for later use by the next job SolvePythagoreanTheorem. Once the job steps have been entered, click Save and Close to save and create the new job.

 

Right-click on the MathAPI Plan and create another new job named SolveForHypotenuse. This job will also be a Jobs Library type job with the following steps:

 

 

The two (2) Log steps display the 2 legs of the triangle. The SolvePythagoreanTheorem passes the 2 leg values as input to the method. The step returns the hypotenuse value which is displayed in the final Log step.

 

Click OK to save the job. Use Map View to create a completion trigger between GetLegs and SolveForHypotenuse.

 

 

Triggering the MathAPI Plan causes the 2 jobs to run as indicated. The log file for SolveForHypotenuse displays the results.

 

 

Creating a Service Library – API (SQL Server/Oracle Stored Procedure/Functions)

 

ActiveBatch supports the leveraging of logic within a SQL Server Stored Procedure or Function. Again we use the power of the Service Library to determine the methods to be exported to the Integrated Jobs Library. In this example, we will use a SQL Server Function that encodes a Base64 string and another that decodes a Base64 string. The functions were originally written for database and stored procedure use. With the Service Library we can reuse the logic as part of our workflow. Note: Other than connection differences Oracle Stored Procedures/Functions are implemented in the same manner.

 

Create a Service Library object named SQLServerLibrary within the Objects folder. Select Properties and select API Adapter. Click the Configure button and select SqlServerAdapterEndpoint.

 

 

The values shown will vary for the system you choose. Click the OK button to continue.

 

 

Select Functions(Scalar-valued) and the two (2) functions will appear. Select them and click the Add button to mark those functions as selected operations. Click Save and Close to save the Service Library object.

 

Create a Plan – SQL Server Functions

 

Create a new Plan under ServiceLibraryPlan named SQLServerServiceLib. Select Service Libraries and click the Associate button. Look for SQLServerLibrary under the Objects folder.

 

 

Click Save and Close to save the Plan.

 

Create a Job – SQL Server Functions – EncodeBase64

 

Now we’ll create the first of two jobs to use the functions as job steps.

 

Create a new job named EncodeBase64String. Create a constant variable named EncoderString. The job type should be set to Jobs Library. Enter the Job Step Editor and at the bottom of the Jobs Category list the new SQL Server function/job steps will appear.

 

 

The key job step is the EncodeTextToBase64. Text data is passed to the job step and the return value is passed back from the step in Base64 string format. We set an active variable named Base64String so a later job can decode the Base64 string back to text.

 

Create a Job – SQL Server Functions – DecodeBase64

 

The second job is to decode the Base64 string back to text. Create a new job and name this job DecodeBase64String. The job type should be Jobs Library.

 

We use the active variable that was set in the previous job; Base64String. We invoke the DecodeBase64ToText step and the original text will be displayed.