Terms and Concepts

This section provides a high-level description of ActiveBatch objects, and also defines key terms and concepts. Other sections provide significantly more detail regarding some the topics briefly described here. Links are provided when that is the case.

 

When you connect to an ActiveBatch Job Scheduler, you are actually connecting to a Job Scheduler Namespace, where all ActiveBatch objects reside in the ActiveBatch backend database. A namespace is where all ActiveBatch objects reside for a given Job Scheduler. Similar to a file system, the ActiveBatch namespace contains objects that can be end-nodes or containers. A container may contain zero or more other objects including other containers. Continuing with the file system analogy, you specify the container name(s) when searching for a specific object. If the context is such where the current container (i.e. current directory) has been set, ActiveBatch will use that context and search that container. The purpose of a namespace is to catalog objects for later retrieval.

ActiveBatch is an object-oriented Job scheduling solution.

  • There are 14 object types an ActiveBatch developer (sometimes called a Job author or designer) can create.

  • The objects are primarily accessed and created in AbatConsole's Object Navigation Pane.

  • All objects have property settings. The properties describe the object.

  • Objects have actions that they support. For example, you can enable or disable an object, or trigger a Job.

  • The Job Scheduler manages the objects, as they are stored in the ActiveBatch backend database, when a Job author saves them.

  • All objects are securable (e.g., read, write, modify, delete, etc.).

  • The majority of the object types are reusable, that is, they can be shared among other objects (primarily Jobs and Plans) that support their use.

  • Some shared object types are optional - you can use them if you would like to take advantage of a specific feature. Other shared object types are required to get a basic Job up and running (every Job is associated with a User Account object and a Queue object).

  • The purpose of shared objects is to minimize the maintenance of your ActiveBatch environment. Rather than hard-coding key Job properties, such as the system the Job should run on, or the dates and times it should run - Jobs that share the same information can use the same shared object. Should any aspect of the shared object change, the change will affect all the Jobs using it.

 

Every object has both common and unique properties that pertain to that object. Common Object Properties include the Name, Label, Description, and the State of the object. An object’s Name is how you see the object identified in AbatConsole. If you wish to display the Label of the object instead, go to Tools > Settings > General > General > Display Mode - and select Label from the dropdown list.

 

While it recommended to uniquely Name objects to avoid confusion, ActiveBatch makes no such requirement. An object’s Label, however, must be unique within the namespace. An object’s Label is typically used when scripting ActiveBatch operations. For example, if you write a script to manipulate ActiveBatch objects it will most likely be the Label that you reference, not the Name.

 

When a shared object is used by another object that supports it, it is said to be "associated" to the shared object. It is mainly Job and Plan objects that are associated with shared objects. For example, the Schedule object is a shared object. The Schedule object determines when triggerable objects should run. A Schedule object is configured with date(s) and optionally time(s) - the time can be set on the Schedule object or embedded on the Job itself. When a Schedule object is associated with multiple Jobs and/or Plans, the Jobs and Plans will all run on the same dates (and possibly times) set in the Schedule. Should the Schedule change, all the Jobs and Plans associated with it will be affected (they will run on the modified Schedule). This type of behavior is true for all shared objects. That is, when you modify the shared object, all objects associated with the shared object will be automatically updated to reflect the change.

 

Shared objects include the User Account, Alert, Schedule, Calendar, Resource, Service Library, Object List and Queues (Execution and Generic).

 

Note: There are few scenarios in which a shared object is associated to another shared object (as opposed to a Job or Plan). For example, you can optionally associate an Alert object to a Queue object.

 

Non-shared objects include the Folder, Plan, Job, Job Reference, and Plan Reference.

 

Note: The Folder object does not support any shared object associations.

 

Virtual Root is an optional feature that allows you to connect to a Job Scheduler Folder (best practice) or Plan object, as opposed to connecting to the root of the Job Scheduler. When connected to a Virtual Root, the user's scope is limited. They will only see and have access to objects and instances that are child objects within the connection point. This feature can be used for security purposes, or simply to de-clutter a user's view, if they don't need to see or have access to containers that are not their concern.

Trigger is an ActiveBatch action that causes a Job or Plan to become eligible for execution. A trigger can occur in several ways, such as using a Schedule object, configuring an event-based trigger, configuring a completion trigger, or via a manual trigger, and so on.

 

When a trigger occurs, the Job Scheduler clones the triggerable object, whereby an Instance of the object is created, using all the pertinent details in the triggerable object's definition. It is the Job instance that runs. It is the Job definition or Job template that is used to create the instance. Whether or not the instance runs immediately depends on other environmental conditions and/or object properties set on the definition. For example, a condition may be set on a Job object where the Job cannot run until a certain file is present in a specified directory. This is called a File Constraint. Or the instance may not run immediately because the Execution Agent system it is configured to run on is already running its maximum amount of allowed Jobs; therefore the instance must wait its turn.

 

Triggerable objects include: Job, Job Reference, Plan, and Plan Reference. When these objects are triggered, an Instance is typically created unless something is preventing this (e.g., the object is disabled). Briefly, a Job Reference points to an existing Job, and a Plan Reference points to an existing Plan. Only Job and Job references are sent to Execution Agents to run. Plan and Plan references are not configured with a payload, and therefore they are not sent to Agents to run. Plans and Plan references do go into an "executing" state, but they are only in this state while the underlying Jobs within the Plan or Plan Reference are running.

 

The objects listed below are containers (Folder and Plan) and triggerable objects (Job/Job references and Plan/Plan references).

 

The Folder Object is used to organize ActiveBatch objects. It is similar to a folder in the Windows file system. While its primary use is for organizational purposes, it also has other benefits. For example, you can define variables on the Folder level, where the variables are visible (in scope) to the child objects nested within the Folder. Child objects can also inherit security from a parent Folder. Folders also support policies. You can create a default policyClosed A default policy allows you to preset a new object's properties to override factory default property values, and/or preset properties that are blank by default. See Default Policy for more details.  for each type of ActiveBatch object. Folders also support the creation of audit policiesClosed An audit policy provides a way for a user to add additional audit information when working with ActiveBatch objects and/or instances. For example, an audit policy can be created that prompts an ActiveBatch user to enter a reason why they are modifying an object, or disabling an object, or manually triggering a Job or Plan. The reason entered is stored in the audit trail of the object or instance. For more details see Audit Policy.. Folders are not triggerable.

The Job Object consists of all the details required to run to your Process, Script, or Jobs Library Job. This includes the payload of the Job (i.e., the work to be done), where the Job will run (i.e. the Execution AgentClosed The Execution Agent ("Agent") is the component that runs ActiveBatch jobs. It must be installed and configured on a system that is specified in the Machine property of an Execution Queue object. The Job Scheduler uses the Queue's Machine property to determine what system to connect to in order to dispatch jobs. Most ActiveBatch environments have more than one system with an Agent installation. Which Agent to send the Job to is based on the Submission Queue that is assigned (associated) to each Job, a required Job property. system), and the user credentials it will run under. How the payload is specified will depend on the Job type selected.

 

Job Type Description

Jobs Library

This type of Job allows you to use production-ready Job steps which are built-in building blocks for commonly scripted actions and functions.

Process

This type of Job consists of either a script file or an executable to run.

Script

This type of Job is similar to a Process Job except that the script information is embedded within the Job and not referenced through an external script.

 

Note: When running a script as either a Process or Script Job type, the Execution Agent system (where the Jobs run) will need to have the required software installed to successfully run the script. For example, when running a PowerShell script, the Execution Agent system would need to have PowerShell installed.

 

Every Job needs to be associated with a Queue object (determines where the Job will run) and every Job needs to be associated with a User Account object (specifies the user credentials to run the Job with). Therefore, you must create at least one Execution Queue object and one User Account object to get a basic Job up and running, in addition to creating the Job object itself. At a minimum, the Job object (sometimes referred to as simply the Job, Job definition or Job template) needs the following properties set: Name and Label, Submission Queue, User Account, Job Type and its associated details.

 

Note: ActiveBatch comes with two built-in Queues named "OnDemand" and "RuntimeQueue". It also comes with a built-in User Account named "RuntimeUserAccount". These objects have a special purpose when associated to Jobs, but you still must create your own Execution Queue and User Account before you can run any Job in ActiveBatch.

A Job Reference Object points to an existing Job. Its purpose is to eliminate the creation of Jobs that do the same work. For example, assume JobA runs in several different workflows (for example, in three different Plans). Rather than create copies of JobA and place them in the Plans, you can create one Job, where one Plan will contain the Job, and the remaining two Plans will contain a Job Reference that points to JobA. Job references inherit some properties from the Job it is pointing to, and some properties are configurable on the Job Reference itself. A key benefit in using the Job Reference is it inherits the payload from the Job it is pointing to, so if anything changes regarding the payload, the change only needs to be made on one Job. Finally, if there are subtle differences between the Job and Job Reference, often times the use of variables can manage the difference(s).

A Plan Object is one of the three supported container types (the Job Scheduler and Folder are the remaining two). A Plan is a triggerable object. Other objects are added to Plans. Any other object type (shared or non-shared) can be added to a Plan, including nested Plans. A key purpose of the Plan is to place related Jobs within it, and optionally add other related objects - e.g., the Plan's Schedule, Alert, etc. Note: There are other options where to place related shared objects - see Organizing Objects. In addition, you can optionally set the order the Jobs should run in (e.g., sequentially), then trigger (start) the Plan. Like the Folder object, you can also create variables on the Plan-level, and child objects can inherit security from the Plan. Plans also support policies. You can create a default policyClosed A default policy allows you to preset a new object's properties to override factory default property values, and/or preset properties that are blank by default. See Default Policy for more details.  for each type of ActiveBatch object. Plans also support the creation of an auditing policyClosed An audit policy provides a way for a user to add additional audit information when working with ActiveBatch objects and/or instances. For example, an audit policy can be created that prompts an ActiveBatch user to enter a reason why they are modifying an object, or disabling an object, or manually triggering a Job or Plan. The reason entered is stored in the audit trail of the object or instance. For more details see Audit Policy. in which you can specify which operations require additional auditing information.

A Plan Reference Object points to an existing Plan. Its purpose is to eliminate the creation of Plans that do the same work of an already existing Plan. For example, assume PlanA runs in several different workflows. Rather than create three copies of PlanA, you can create one Plan and its associated Jobs. When you wish to run the Plan in other workflows, create a Plan Reference.  A key benefit in using the Plan Reference is if the payload of Job(s) within the Plan changes, the change only needs to be made on one Job, not multiple Jobs. Finally, if there are subtle differences between the Plan and its references, often times the use of variables can manage the difference(s).

The objects listed below are shared (associated with) primarily Jobs and Plans (except Queue objects are not associated with Plans). In a few cases, some shared objects are associated with other shared objects.

 

Note: Every Job must be associated with a Queue.

 

The Queue object determines which Execution Agent system the Job will run on.  There are 2 types of Queues a user can create - execution and generic. In addition, users can also create a Managed Queue, which is a type of Generic Queue, but an extra license is required.

  • Execution Queue Object - The Execution Queue is a direct path to the Execution Agent where a Job instance will run. The Execution Agent is the cross platform ActiveBatch component that runs your Jobs. The Execution Agent system is identified in the Execution Queue's Machine property. Any Job associated with an Execution Queue will always run on the same Execution Agent system the Execution Queue points to, unless a manual override is performed.

  • Generic Queue Object - The Generic Queue consists of two or more Execution Queues. When a Job is associated with a Generic Queue, it will be dispatched to one of the Execution Queues associated to the Generic Queue. A key benefit of the Generic Queue is a Job is not tied to a single Execution Agent system - it can be dispatched to any of the Execution Queues that are members of the Generic Queue.

    The Generic Queue is ActiveBatch's high availability solution (active-active) for the Execution Agent. If an Execution Queue within the Generic Queue becomes unavailable (for example, the Execution Agent system is down for maintenance), the other Execution Queues are still available to run Jobs.

    When a Job is associated to a Generic Queue, the assumption is it can run on any of the Execution Queues that are members of the Generic Queue. However, on a per Job basis, there is a way, using Queue Characteristic properties, to pare down the number of eligible Execution Queues within the Generic Queue that the Job can run on. This optional feature, combined with additional Generic Queue properties, are used by the Job Scheduler to determine which Execution Queue to dispatch the Job to.

     

    Note: Technically, a Generic Queue can consist of just one Execution Queue. But if configured this way, all Jobs associated to the Generic Queue would always run on the same system - the only Execution Queue associated to the Generic Queue.  In this scenario, it would make more sense to assign the Job to an Execution Queue, unless the intention is to add more Execution Queues to the Generic Queue at a later time.

  • A Generic Queue can also consist of systems in which ActiveBatch itself can be responsible for determining the proper number of systems based on historical demand as well as real-time scheduling. These facilities are known as Managed and Smart Queues.

    A Managed Queue is a special type of Generic Queue that supports the use of Amazon EC2 (for Cloud-based instances), Microsoft Azure (for Cloud-based instances) or automated provisioning of virtual systems using Microsoft’s System Center Virtual Machine Manager (SC-VMM) as the controlling facility. SC-VMM supports Hyper-V, VMware and Cytrix Xen. A Smart Queue builds on the facilities of a Managed Queue by increasing or decreasing systems based on real and historical demand.

    • The providers for Managed Queue are separately licensed. VMware requires the VMware Extensions license; SCVMM requires the System Center Extensions license, Amazon EC2 requires the Amazon EC2 license and Microsoft Azure requires the Microsoft Azure license.

  • Built in Queues - Built into the product are two Queues - the OnDemand Queue and the RuntimeQueue. They offer specific functionality described in the Queue section of the documentation.

     

    Note: The VariableQueue was deprecated and replaced by the RuntimeQueue.

     

 

Note: Every Job must be associated with a User Account object, where the Type of User Account (a property setting) is Username/Password.

 

The User Account Object represents a set of security credentials. Several credential types are supported. The types are listed in the Properties tab of the User Account object, in the Credential Type dropdown list. The Username/Password Credential Type is what you select and configure for credentials used for Job execution. A User Account object can also contain a security certificate that can be used for SSL authentication requirements when using FTPS or Web Services. A User Account object can also contain information for connecting to an SAP system, or be configured to store security token information when connecting to various systems (for example, Twitter).

 

The advantage of the User Account object (like all other shared objects) is that a change to the object (for example, a new password) automatically affects every Job associated with the object. The User Account object is a securable entity, like all ActiveBatch objects. By securable, we mean you can set security properties for each and every object, and specify the Active Directory users and/or groups that have access to the objects, and what their permissions are (e.g. read, write, modify, delete, etc.).

 

The RuntimeUserAccount is a special built-in object that allows a Job author to defer the association of the User Account object to execution time by using a built-in variable named #UserAccount. The variable would represent the full path of the User Account object you want associated with the Job. The User Account object (that the variable will resolve to) must be listed on the Job's Associations tab.

An Alert Object represents one or more alert actions that may be performed when an ActiveBatch condition requiring an alert occurs (e.g., a Job fails). You may assign an Alert object to one or more Queues, Jobs or Plans. Alerts are supported at the Scheduler level, but only embedded alerts (you cannot associate an Alert object). A common alert action is sending an email to one or more recipients when an ActiveBatch condition occurs.

A Schedule Object consists of date(s) and optionally time(s) that triggerable objects should run on. The time can be obtained from the Schedule object, or it can be set (embedded) directly on the triggerable object. At the very least, a Schedule object associated with triggerable objects will result in recurring trigger dates.

 

Schedules can operate on a Calendar, fiscal and business date basis. When the scheduled date/time arrives, the triggerable object will trigger, providing there are no Date/Time exclusions established on the triggerable object and the triggerable object and Schedule are enabled (a disabled object of any type means the object will not perform the action it is designed to do). One or more Schedules can be associated with a single triggerable object. For example, you may have a weekly Schedule and a weekend Schedule associated to a Job.

A Calendar Object consists of dates that are either non-business days (e.g., Saturday and Sunday) and/or holidays. When the Calendar object is associated to Plan/Job on the Constraints property sheet, the dates in the Calendar are days the Plan/Job should not run.  When a holiday date occurs (that is specified in the Calendar), the triggerable object will not run, but it can (optionally) be set to run on the previous or next business day.

 

Note: The Trigger (Advanced) feature can override a Calendar object preventing a Plan/Job running when the "Ignore Calendar" option is checked.

 

Another use of the Calendar object is to associate it to a Schedule object's Exclusion or Inclusion list.

 

When the Calendar object is added to the Exclusion List, the combination of the Schedule object's trigger dates minus the Calendar's holiday dates results in "business days". For example, assume you want to run a Job on the 5th business day of every month. Using the Schedule object, combined with an Exclusion Calendar object, would be your best option.

 

Note: When the Calendar is used to identify business days (as described above), you cannot indicate you wish to run the Job on the previous or next business day - when a holiday date occurs. This is only an option when associating the Calendar object on the triggerable object's Constraints property sheet.

 

While the Calendar object is most frequently used to prevent triggers (a type of constraint) and to generate "business days" (when associated with the Schedule object's Exclusion list), it can also be used to do just the opposite - trigger Jobs. This is accomplished when the calender object is associated to a Schedule object's Inclusion List. In this scenario, the holidays configured in the Calendar result in trigger dates (in addition to the dates specified in the Schedule object - if any).

A Resource Object is used when configuring a Resource Constraint, where a constraint is a feature that prevents a triggered instance from running until a condition is met. There are four General Constraints, and the Resource Constraint is one type. When configuring a Resource Constraint, you must associate a Resource object to the constraint. The Resource object is configured with either a set (static) numerical value, or a dynamic numerical value obtained from an ActiveBatch active variable. The Resource Constraint has a Units Needed property which indicates how many units are required before the triggered object can run. If the Resource object has enough units (the static or dynamic numeric value described above), the Job will run. If the Resource object does not have enough units, what happens next depends on other constraint property settings. For example, the Job may "wait" for the number of units to become available, or the Job may fail due to the constraint not being satisfied.

A Service Library Object represents various implementations of external methods or APIs. These include REST, Web Services, .NET Assemblies, and SQL Server and Oracle stored procedures or functions. When using the Service Library object, the implementation you choose (e.g., accessing a REST API) will result in Job steps being added to the Jobs Library (one of the ActiveBatch Job types). The Service Library object provides a simple yet powerful approach to adding your own methods and operations to the Jobs library.

An Object List represents one or more ActiveBatch objects. In other words, it is a list of objects. Currently, an Object List object is used for object association purposes. Rather than embedded the list of associated objects within a Plan or Job, you can create an Object List object with those associations and then share the Object List object. This type of usage is similar to the difference between embedding alerts within a specific Job or using an Alert object that can be shared by many Jobs.

Variables are supported in ActiveBatch and have several key uses. First, variables allow a Job author to pass data to downstream workflows (e.g., from JobA to JobB). Second, variables can be used to soft-code object properties, which is particularly useful when you have multiple objects that have the same property value. If the property value should change, it is easier to update a single variable value, as opposed to updating several objects that share the same value for the given property. Third, variables can also be used as a constraining precondition (called a variable constraint) for Job execution. Fourth, you can designate variables to be exported to the running Job by marking it as an environment variable.

Scheduled Triggers are a type of automated trigger configured on a triggerable object's Triggers property sheet. There are 3 options when establishing a Scheduled trigger.  

  • Set a recurring time interval (e.g. every 30 minutes)

  • Use trigger dates and optionally trigger times from an associated Schedule Object (you can use the Schedule object's time or set the time on the triggerable object)

  • Use Constraint Based Scheduling (the triggerable object will run based on dates set on the associated Schedule object, but only after all the General Constraints have been met)

Event Triggers are another type of automated trigger that causes Jobs and Plans to be eligible for execution. When an event trigger is configured, ActiveBatch monitors for an occurrence of the event. If the event happens, a trigger occurs. Oftentimes event triggers are not predictable - that is, you can't be sure when the event will occur. This is very different from the Schedule object (mentioned above), where the triggers are recurring based on set dates and times. One type of event trigger is email. When configured, ActiveBatch will monitor an email mailbox for incoming emails, and trigger the Job when an email arrives. Another popular event trigger is the file trigger (or FTP file trigger). ActiveBatch monitors a directory, and when a change occurs (e.g., a file is added), it results in a trigger. ActiveBatch supports a wide variety of events and event sources that are described in detail in the event triggers section

A Completion Trigger is an automated trigger that occurs when an upstream Job or Plan triggers a downstream (following) Job or Plan upon its completion. For example, you may have 3 Jobs that need to run sequentially, one after another. To accomplish this, configure a completion trigger on each Job that needs to trigger the next Job. The next Job is triggered based on the upstream Job's completion status or exit code.  For example, JobA's completion trigger can be configured to trigger JobB only if JobA succeeds, or only if JobA exits with a code of 12. You can also optionally have a different Job that is triggered if JobA fails. Lastly, an upstream Job can trigger more than one Job after it completes, not just one.

 

A constraint is used to establish one or more preconditions that must be true to allow a Job or Plan to execute. Constraints are sometimes referred to as “dependencies”. ActiveBatch supports four types of General Constraints and two types of Date/Time constraints. Both general and date/time constraints are not triggers. They are evaluated after a trigger occurs, to see if the precondition(s) are satisfied, allowing the instance to run.

 

Note: There is a feature named Constraint-Based Scheduling (CBS) which is configured on a triggerable object's Triggers property sheet. CBS is a trigger mechanism that uses (evaluates) general constraints before the triggerable object can run. The day(s) that the CBS configured Jobs will run on is based on an associated Schedule object. The time the triggerable object will run is based on other CBS properties set by the Job author.

By default, ActiveBatch uses a normal 24-hour clock starting at midnight and ending at 11:59pm. This is called a Calendar day. ActiveBatch also supports the concept of a Business Day that by default, also begins at midnight and ends at 11:59pm. However, the business day is configurable. When configuring a business day, you specify a starting time period and the system will then add 24 hours (minus one minute) to make it a day.

 

Business days are particularly useful when implementing: Job/Plan constraints, Constraint-Based Scheduling (CBS), and business date variables.

A batch or batch run begins with the first object that is instantiated and continues to the last object that is instantiated, where the other objects (in the same batch run) were instantiated when: a Plan was triggered, or completion triggers were used, or Alert triggers were used - or any combination thereof. Instantiated means that the object is capable of being cloned into an instance. As a reminder, only Plans, Jobs and reference objects can be instantiated. Object instantiation begins with some aspect of a trigger being invoked, for example, a date/time trigger, event trigger, completion trigger, CBS trigger or manual trigger operation.

 

For example, assuming that PlanA contains Jobs A and B, and PlanA is the object configured with the initial trigger; PlanA will be the first object to be instantiated. That object is assigned a Batch Run ID and begins the batch. The underlying Jobs will also be instantiated based on their presence in PlanA (and have their own Instance ID's). However, they will also inherit the Batch Run ID of their parent, and all instances will be considered to be part of the same batch run. For more details see View Batch Run.

Workflows consists of one or more related batches. The related aspect between batches is typically comprised of Job or Plan constraints. For example, an afternoon batch may be constrained by a morning batch. In other words, the morning batch must complete successfully before the afternoon batch can begin executing.

 

Likewise, an end-of-day batch (or series of batches) will most likely contain constraints that enforce an order in which the batches must execute. While each batch is separate, they are still related and that relationship is important to the entire set of workflows. The Forecast specifically looks at workflows and their underlying batches and provides insight to the overall health of the workflows.