Search code examples
azurescalabilityazure-web-rolesazure-worker-roles

Most scalable means of returning results to client from Azure worker role

I'm implementing a web service using Azure Cloud Services. The web service needs to...

  • accept input data
  • run a potentially lengthy calculation on the input data
  • return results of the calculation

From what I've read, it is recommended that a worker role hosts the calculation, whilst a web role handles the HTTP requests/responses and a queue is used to pass the input data between the 2 types of role. This architecture allows web roles to be scaled up to meet an increased demand in web traffic and/or worker roles to be scaled up to enable parallel processing of results.

What I'm not sure about is the best way to deliver the calculated results back to the client that sent the original HTTP request to the web role. In terms of scalability, is it better to provide an additional web service method that clients can call to return results (if they have been processed) or is it better to return the results as part of the HTTP response to the original request?

If the latter, what is the most straightforward way to do this using PHP?

Any advice would be much appreciated.

Solution

  • Here's a concept you can borrow from Azure's API: http://msdn.microsoft.com/en-us/library/windowsazure/ee460783.aspx

    1. Client makes request to web role for computation.
    2. Web role creates a GUID for this operation.
    3. Web role add requests to Azure Storage Queue & Service Bus, adds GUID to the queued record.
    4. Web role responds back to the client with HTTP 201 Accepted and GUID as OperationId in the response body.
    5. Client polls the API endpoint on web role to see if the operation with that id has finished.
    6. Web role queries some table storage (db or Azure Table) to see if that operation record is marked as finished.
    7. Meanwhile worker role pickes up operation from the queue, processes it and when done, puts the result back to the table storage (or db) so web role can access using OperationId
    8. Meanwhile client was polling the API with GetOperationStatus(operationId) call every 1 minute or so. When task is finished and results are Ready, client can call if GetOperationStatus(operationId) == finished { GetOperationResults(operationId) } through API endpoint on web role.

    Here, client has never contacted with worker role directly. Because worker roles are supposed to run background operations. And this is done through messaging (queues, service bus etc.)