Grid Dispatcher Configuration

The dvce-app-config.xml file located in the platform/setup folder contains the GridDispatcherConfiguration used to configure the Computational Grid.

Any changes to the file will require a server restart to take effect.

<GridDispatcherConfiguration xmlns="http://www.onenetwork.com/Platform" active="true">
	<!-- Task performer configurations go here! -->
</GridDispatcherConfiguration>

There are two settings to configure, beyond the task performers themselves.

The first is whether or not the Computational Grid is active. Setting the active attribute to true activates the Computational Grid while a setting of false deactivates it.

TaskPerformer and TaskPerformerConfig

These two classes are designed to work together. As you may guess, TaskPerformerConfig is designed to give you a convenient way to configure TaskPerformers independently of the TaskPerformer itself. A TaskPerformerConfig has three main elements:

Type: This corresponds to the type field on a GridTask.
NumberOfThreads: This sets the number of threads available to execute this task type.
TaskPerformerClassName: This is the fully-qualified class name of the class responsible for execution. The class must implement the TaskPerformer interface.

To illustrate this, let's take a very simple example. The following code listing is designed to create a simple TaskPerformer and its related TaskPerformerConfig class. To keep it simple we're going to demonstrate a common pattern in grid-based execution code - the NoOp task.

A NoOp task, at first glance, may not appear to be terribly useful since it doesn't really do anything. However, this simple configuration can be very useful in the real world.

package com.example.taskperformer;      
public class NoOp extends AbstractTaskPerformer {
	public String getTaskType() {          
		return "NoOp";
	}        
	public TaskResult performTask(String taskParams, String taskInputParams) {          
		return TaskResult.SUCCEED;        
	}      
}

TaskPerformer classes extend the One Network Platform class AbstractTaskPerformer. The getTaskType( ) returns the string NoOp and the performTask( ) method simply returns TaskResult.SUCCEED. In short, this task doesn't do anything except succeed. There is nothing magical about the term NoOp within the context of this class. It could return any text value since the task type on a TaskPerformer is really only used for identification and documentation purposes. It has no intrinsic meaning to the Platform SDK.

This may lead you to wonder why they are useful at all. In fact, this class is used in production all the time. The original use for this class involved integration with legacy systems. There are times when an application needs to create a Job and tasks that depended on that Job before the system knew if any tasks would actually be added to the job. If tasks were never added to the Job, any tasks that depended on the job would never run since the job dependency is only removed when a task in that job runs.

In these situations, the simplest solution is to a place NoOp task in the job thus ensuring that the Job has at least one task in it.

Now that we have a TaskPerformer, let's see an example of configuration. There are two ways to do this. You can either use the dvce-app-config.xml file to configure the task schedule, or you can create a schedule programatically. The former is most common for grid tasks that execute on a set schedule while the latter allows you to arbitrarily create and place tasks on the Grid at any time.

First, let's look at an XML based example:

<TaskPerformerConfig>
	<Type>NoOp</Type>
	<NumberOfThreads>10</NumberOfThreads>
	<TaskPerformerClassName>com.example.taskperformer.NoOp</TaskPerformerClassName>
</TaskPerformerConfig>

This entry placed in the dvce-app-config.xml file will create a task which will run on a maximum of 10 threads.

Next, let's take a look at the same configuration implemented as a Java class. This approach allows you to submit tasks to the Grid from your code giving you full control over when they execute.

GridService gridService = Services.get(GridService.class);
TaskPerformerConfig cfg = gridService.newTaskPerformerConfig();
cfg.setType("NoOp");
cfg.setNumberOfThreads(10);
cfg.setTaskPerformerClassName("com.example.taskperformer.NoOp");
Collection<TaskPerformerThread> threads = gridService.startTaskPerformerThreads(cfg);

After starting a TaskPerformer, you may want to halt the TaskPerformer threads once the task has completed. ** is this done automatically at any stage ** The following code sample shows how this may be done.

//kill off threads
for(TaskPerformerThread thread : threads) {
	thread.halt();
}

Simply iterate through the TaskPerformer threads and halt them. This is especially useful when unit testing.

Next, let's take a look at a more complete example based on the book store tutorial. The book store tutorial steps you through the creation of a working business example where you can create and modify inventory (books), place orders, etc.

For our example we will extend the order and fulfillment processor to perform these tasks:

Send the order information to a third party payment processor
Fulfill the order
Send a notification to the user that processing is complete

Each of these tasks can be implemented as a TaskPerformer.

public class ProcessPayment extends AbstractTaskPerformer {
	public static final String TASK_TYPE = "ProcessPayment";
 
	@Override
	public String getTaskType() {
		return TASK_TYPE;
	}
 
	@Override
	public TaskResult performTask(String taskParams, String taskInput) {
		JSONObject params = new JSONObject(taskParams);
		String orderId = params.getString("orderId");
		String userId = params.getString("userId");
			//1)Contact the payment processor
			//2)Submit the payment information
			//3)Verify receipt 
			//If steps 1 or 2 fail due to some intermittent failure, return TaskResult.RETRY
			//If step 3 fails (Payment method declined, etc), return TaskResult.createFail("failure reason")
		return TaskResult.SUCCEED;
	}
}

This example is really a stub. The comment lines show where you would implement a vendor specific communications session to a payment gateway. Since each payment gateway is different in capability and interface methods, we only present the stub here.

Note that performTask takes two arguments, a string listing the parameters, and a string containing any input data from preceding tasks.

If you know this task might be long running, you can increase the maximum time allotted before it fails due to time-out:

paymentTask.setMaxRunTimeMillis(1000 * 30 * 60);

If the process is error prone, perhaps due to an overly crowded payment gateway or some external process that is beyond your control, you can change the maximum attempt count. Likewise, if its a task that can only run once, you can set the maximum to one. This example sets it to twenty.

paymentTask.setMaxAttempts(20);

Based on the results from the payment gateway, you can set the TaskResult to one of the statically enumerated result codes.

Next, let's suppose a different system is responsible for processing fulfillment. Perhaps its a call to a legacy ERP system like Oracle or SAP. Again, we'll show you the stub since all systems implement the actual integration a little differently. On the surface, this stub could be identical to the last, however we'll show you a different twist on the same idea. This time, we'll include some data from the task along with the success message.

public class FulfillOrder extends AbstractTaskPerformer {
	public static final String TASK_TYPE = "FulfillOrder";
 
	@Override
	public String getTaskType() {
		return TASK_TYPE;
	}
 
	@Override
	public TaskResult performTask(String taskParams, String taskInput) {
		String orderId = new JSONObject(taskParams).getString("orderId");
			//1. Check and update inventory
			//2. Return TaskResult.SUCCEED with the task output of the fulfillment identifier created by (1)  
		String fulfillmentId = "example-fulfillment-id";
		return TaskResult.createSucceed(fulfillmentId);
	}
}

For the most part, this is the same code we saw in listing 1.6. The last two lines of the performTask method are different in that instead of simply returning the TaskResult.SUCCEED message, it creates the success message and inserts a string within it. This allows succeeding tasks to receive input from their predecessors. In this case we are passing a fulfillment id, presumably coming from your fulfillment system of choice, to the next task which is designed to send a notification email to the user giving them the details of the fulfillment.

public class OrderNotification extends AbstractTaskPerformer {
	public static final String TASK_TYPE = "OrderNotification";
 
	@Override
	public String getTaskType() {
		return TASK_TYPE;
	}
 
		@Override
		public TaskResult performTask(String taskParams, String taskInput) {
			String userEmail = new JSONObject(taskParams).getString("email");
			String fulfillmentId = taskInput;
				//1. Get the user information from the taskParams 
				//2. taskInput will be the fulfillment ID returned from the previous task. If it is null, the previous task failed
				//3. Send an email notification ("order failed" if the taskInput was null) and return success 
			return TaskResult.SUCCEED;
		}
}

Take a look at how the taskInput argument is used here. We're taking the result from the task performer shown in listing 1.7 and we're sinning it in this task which follows in order of execution. In other words, if the task performer in listing 1.7 is successful in communicating with the fulfillment server, it will send the fulfillment ID assigned by that system to this task (listing 1.8) where it can be used by the code in the performTask method.

Now let's glue these prices together with the next code listing. As usual, we leave the implementation details to you, but you could put this in a workflow as a custom Java activity for complex ordering scenarios, or perhaps in a REST class if the ordering process is otherwise fairly simple.

public function processOrder(Order o){
	String userEmail=o.getUserEmail();  
	String userId=o.getUserID():
 
	JSONObject userInfo = new JSONObject().put("email", userEmail).put("userId", userId);
		
	GridService gridService = Services.get(GridService.class);
	GridTask paymentTask = gridService.newTask(ProcessPayment.TASK_TYPE);
 
	//In GridService.insertTasks, propagateFail defaults to true
	//In GridService.queueTasks, propagateFail defaults to false
	paymentTask.setPropagateFail(true); 
	paymentTask.setParams(userInfo.toString())
		
	GridTask fulfillmentTask = gridService.newTask(FulfillOrder.TASK_TYPE);
	fulfillmentTask.setTaskDependency(paymentTask);
	fulfillmentTask.setParams(userInfo.toString());
	fulfillmentTask.setPropagateFail(false);
		
	GridTask notifyTask = gridService.newTask(OrderNotification.TASK_TYPE);
	notifyTask.setTaskDependency(fulfillmentTask);
	notifyTask.setParams(userInfo.toString());
		
	gridService.insertTasks(Arrays.asList(paymentTask, fulfillmentTask, notifyTask));
	//Alternatively, since each task depends on the previous task, 
	//you can use a queue rather than setting the task dependencies:
	//gridService.queueTasks(uniqueQueueName, Arrays.asList(paymentTask, fulfillmentTask, notifyTask));
}

Notice that if the payment task fails, the fulfillment task will not run, but the notification task will. Instead of using the notify task to send both positive and negative notifications, you can use a more advanced feature of the TaskPerformer, the addToFailTransaction method. Here is an example of addToFailTransaction on the ProcessPayment TaskPerformer.

@Override
	public void addToFailTransaction(String taskParams, String taskInput, String failureReason) {
		JSONObject params = new JSONObject(taskParams);
		String orderId = params.getString("orderId");
		String email = params.getString("email");
		//1. Get the user information from the taskParams
		//2. Send an e-mail notification to the user immediately, giving them the reason for failure 
	}

Then we can modify the fulfillment GridTask to propagate its failure.

fulfillmentTask.setPropagateFail(true);

There may be cases where each task can handle failure in the addToFailTransaction method independently. However, since the TaskPerformer class lookup is not done until just before the task is executed, this method will normally only run if the task failed as part of an execution attempt. With task dependencies, a task can fail without ever running; in the previous example, if paymentTask failed, fulfillmentTask would never run. If you know the class name of the TaskPerformer at task creation time (rather than just the task type) and want addToFailTransaction to be executed when the task fails without ever running, you can call GridTask.setTaskPerfClass as in the following example.

fulfillmentTask.setTaskPerfClass("com.example.taskperformer.FulfillOrder");

You can use the current state of the GridTask to determine if the task has failed out of band (e.g. via a propagated failure) or as part of execution. In the latter case, the task will be in the RUNNING state.

TaskPerformer has a similar method, addToRetryTransaction, that will be executed before a task is retried.

This extended example has shown a potential real-world example of using Grid tasks to process the steps in order creation beyond simply just recording the order data as a set of persisted objects in the database. The beauty of setting these up as Grid tasks is the system is then inherently designed to scale. If a site receives a large boost in orders during , its possible to add additional grid nodes to the Platform instance to boost its capability.

For details on the classes we've presented so far, check out the javadocs.