4. Add queuing and transaction support with PHP

Owned by Kris Jin (Unlicensed)
Last updated: May 11, 2013Version comment

Description

SmartClient has advanced features for queuing multiple requests in one single request. This provides a mechanism for sending multiple requests to the server in a single HTTP turnaround, thus minimizing network traffic as well as allowing the server to treat multiple requests as a single transaction (if the server is able to do so). In this sample the previous sample will be refactored to add support for queuing and transaction support.

In order to work with this transaction request,  the code created in DSRequest in the previous article needs to be used. The existing code in RPCManager will need to be refactored. As the transaction request is actually a list of DSRequest objects wrapped with additional information, it is necessary to parse and store the list of DSRequest objects. Then, for each DSRequest object, the execute() method is called (as shown in the previous sample) to get the DSResponse object which will be stored in a list for later use. Once all requests are processed, the DSResponse objects will be used to build and send back the response to the front-end. As a side note, a single DSRequest will also be handled by the same code.

Prerequisites

In order to be able to run this sample,the latest build of SmartClient (at least version 8.3p) is required.

Adding additional classes

Reviewing the RestDataSource Documenatation, explains that the transaction request is actually a list of DSRequest objects with some additional information,that looks like this:

{ "transaction":
   { "transactionNum": 2,
       "operations": [
           ..list of dsrequest objects
       ]
   }
}

There is no major difference between a transaction with a single request and a single request, just some properties wrapped around it. This will be considered when constructing the RPCManager object,so if the payload is a simple request it will be patched, so it will appear to have a a transaction with a single operation. This way both cases can be handled in the same way:

RPCManager.php
function __construct($request)
{
	$this->_request = $request;	
	
	// if is not wrapped in a transaction then we'll wrap it to make unified handling
	// of the request
	if (!$this->checkTransaction())
	{
		$this->_request = '{ "transaction": { "transactionNum": "-1", "operations": ['.$this->_request.']} }';
	}
	
}

In the case of single DSRequest, make transactionNum -1, so later it can be checked as a transaction-less request. In order to decide if the request has transactions or not,  a helper function is required. This will simply check for various strings in the request body and if  found will assume there is a transaction request:

RPCManger.php
private function checkTransaction()
{
	$request_str = $this->_request;
	if( strpos($request_str,'transaction') &&
		strpos($request_str,'operations') &&
		strpos($request_str,'transactionNum') )
	{
		return true;
	}
	return false;            
}

Most of the activity is occurring in the processTransaction() method (which is the processARequest() method from previous article refactored to process a transaction instead of a single request):

RPCManager.php
private function processTransaction()
{
	// retrieve the requests with data in form
	$transaction_request = $this->parseTransactionRequest($this->_request);
	// store transaction num, we'll use it later to see if there was a transaction or not
	$transactionNum = $transaction_request['transactionNum'];
	
	$idx = 0;
	$queueFailed = false;
	
	// make request list
	$req_list = array();
	$res_list = array();
	$operations = $transaction_request['operations'];
	// create transaction
	R::begin();
	try{
		foreach($operations as $op)
		{    
			 $request = new DSRequest($op);
			 //execute the request and get the response
			 $response = $request->execute($request);
			 // safeguard, if was null, create an empty one with failed status
			 if(empty($response))
			 {
				 $response = new DSResponse();
				 $response->setStatus(-1);
			 }                             
			 // if request execution failed, mark the flag variable
			 if($response->getStatus() == -1 )
			 {
				 $queueFailed = true;
			 }
			 // store the response for later
			 $res_list[] = $response;
		}                            
		
		// if there were no errors, commit the transaction
		if (!$queueFailed)      
		{
			 R::commit();
		}                 
	}
	catch(Exception $e) {
		R::rollback();
	}                                                  
	// if we have only one object, send directly the DSResponse
	if( $transactionNum == -1 )                
	{
		$response = new DSResponse();                
		$response->setData($res_list[0]->data);
		$response->setStartRow($res_list[0]->startRow);
		$response->setEndRow($res_list[0]->endRow);
		$response->setTotalRows($res_list[0]->totalRows);
		$response->setStatus($res_list[0]->status);
		
		return $this->buildResult($response);
	}
	
	// iterate over the responses and create a instance of an anonymous class
	// which mimics the required json
	$responses = array();
	foreach($res_list as $res)
	{
		$response = new DSResponse();
        $response->setData($res->data);
        $response->setStartRow($res->startRow);
        $response->setEndRow($res->endRow);
        $response->setTotalRows($res->totalRows);
        $response->setStatus($res->status);
        
		$responses[] = $response;
	}
	return $this->buildResult($responses);
}

There are a couple of items to explain here. If the JSON sent to the server represents a transaction, we need to parse to the JSON format using parseTransactionRequest method.

RPCManager.php
protected function parseTransactionRequest($request)
{   
	$result = preg_replace('/\r|\n|\t|>>|\/\//', '', $result);
	
	$json_result = json_decode($result, true);            
	return $json_result['transaction'];
}

After the transaction request is retrieved, the list of DSRequests is copied to an array of DSReqeusts. Also, save transactionNum for later. 

Once this is done,  create an array to store the responses for each request, then iterate over the list of requests, and for each request call the execute() method to execute the request and get the DSResponse object.

Some validations are performed, such as creating a DSResponse object signaling an error (status = -1) if a request fails.

Once the iteration is complete if there was no individual operation that failed, commit the transaction, otherwise it will be automatically rolled back when exiting the scope of using{}. This completes the processing part of DSRequests. A response now needs to be built to be sent back to the controller.

If transactionNum is -1, it means initially there was only a single request. In this case, only the appropriate DSResponse object should be returned, wrapped accordingly to match the response format, otherwise iterate over the list of responses and for each response create a wrapper object to be sent back to the front-end.

Notice that in the case of transaction request the response the server expects back is different from the response sent back in previous samples, as it has the additional field, queueStatus. This allows each individual response to determine whether the queue as a whole succeeded. For example, if the first update succeeded but the second failed, the first response would have a status of 0, but a queueStatus of -1, while the second response would have both properties set to -1.

Example of response:

[
    {
        response: {
            queueStatus: 0,
            status: 0,
            data: [{
                countryName: "Edited Value",
                gdp: 1700.0,
                continent":"Edited Value",
                capital: "Edited Value",
                pk: 1
            }]
        }
    },
        .. another responses, same format ..
 ]

Note that in the code where the DSResponse object properties are copied, they are set up with the queueStatus property as -1 if there is an error and the commit had to be rolled back, otherwise it will be 0.

Finally,  call the helper method to build the result from these responses and send it back to the controller. The helper method needs to be refactored to move out the wrapping to an anonymous class, 

RPCManager.php
private function buildResult($response)
{
	$data = array();
	$data['response'] = (Array)$response;
	echo json_encode($data);      
}

There are other minor changes needed to this class, such as properties for storing the list of DSRequest objects, their JSON representation, the database transaction, etc. For full details of RPCManager please review the attached archive. 

Changing ListGrid to send transactions

In order to enable the ListGrid to send transactions, turn off autoSaveEdits:true and perform the save manually. To do this, set autoSaveEdits:false as the property on the ListGrid:

ui.js
isc.ListGrid.create({
    "ID": "supplyItem",
    "width": 700, "height": 224, "alternateRecordStyles": true,
    "dataSource": supplyItem,
    "showFilterEditor": true,
    "autoFetchData":true,
    "dataPageSize":20,
    "canEdit":true,
    "canRemoveRecords":true,
    "autoSaveEdits": false
});

Additionally,add a button which will perform the save request:

ui.js
isc.IButton.create({
    "top": 250,
    "left": 100,
    "title": "Save all",
    "click": "supplyItem.saveAllEdits()"
});

At this point, multiple edits are made on the grid and then the Save all button is clicked, the grid will send all transactions for the save. However, if only one row is changed, the request will happen as in the previous sample. Server-side code must handle both cases, and the example code above does so by converting singular requests into the format used for transactional requests.

If  the sample is now run, notice that the transactions are only sent when the first operation contains multiple items. For all other operations, it will contain a separate request (For example if  two rows are updated and two other rows are deleted, on save there will be 3 requests, one transaction and two remove operations). The reason for this is that DataSource has a different URL for each operation. In order to combine the requests in a single queue/transaction request, the URLs would need to be changed to point to a single target. In this example, it will be called  process(). After refactoring, the DataSource definition will look like this:

supplyItem.js
isc.RestDataSource.create({
    "ID": "suppyItem",
    "fields":[
        {"name":"itemID", "type":"sequence", "hidden":"true", "primaryKey":"true"},
        {"name":"itemName", "type":"text", "title":"Item", "length":"128", "required":"true"},
        {"name":"SKU", "type":"text", "title":"SKU", "length":"10", "required":"true"},
        {"name":"description", "type":"text", "title":"Description", "length":"2000"},
        {"name":"category", "type":"text", "title":"Category", "length":"128", "required":"true" },
        {"name":"units", "type":"enum", "title":"Units", "length":"5",
            "valueMap":["Roll", "Ea", "Pkt", "Set", "Tube", "Pad", "Ream", "Tin", "Bag", "Ctn", "Box"]
        },
        {"name":"unitCost", "type":"float", "title":"Unit Cost", "required":"true",
            "validators":[
                {"type":"floatRange", "min":"0", "errorMessage":"Please enter a valid (positive) cost"},
                {"type":"floatPrecision", "precision":"2", "errorMessage":"The maximum allowed precision is 2"}
            ]
        },
  
        {"name":"inStock", "type":"boolean", "title":"In Stock"},
        {"name":"nextShipment", "type":"date", "title":"Next Shipment"}
    ],  
    "dataFormat":"json",
    "operationBindings": [
                { "operationType": "fetch", "dataProtocol": "postMessage", "dataURL": "process.php" },
				{ "operationType": "add", "dataProtocol": "postMessage", "dataURL": "process.php" },
				{ "operationType": "update", "dataProtocol": "postMessage", "dataURL": "process.php" },
				{ "operationType": "remove", "dataProtocol": "postMessage", "dataURL": "process.php" }
            ] 
});

Process handler 

process.php
<?php                           
    require_once 'RPCManager.php';
    require_once 'rb.php';
    
    R::setup('mysql:host=localhost;dbname=smartclient','root','apmsetup');      
    $rpc_manager = new RPCManager($HTTP_RAW_POST_DATA);
    R::close();
    return $rpc_manager->processRequest();                  
?>

The complete code for this sample project can be downloaded from here.