Search code examples
azureazure-functionsazure-queues

How do I use an Azure Storage Queue in a request / response mode?

As we port more of our node.js code into Azure Functions we see references in the Azure docs that using Storage Queues is the preferred way to delegate processing responsibility instead of using http requests to call other functions.

What is the request/response design pattern we should use for this delegation? Specifically, how can the response sent back through a queue be delivered only to the Function where the request originated?

Here's an example of what we want to do:

  1. Request comes in to an HTTP Trigger Function A
  2. Function A places message into Queue X (in JSON format) with the first key being the unique requestId: "ABC345"
  3. Function A starts listening to Queue Y for the response
  4. Function B dequeues this message and does its work
  5. Function B places message with work results added into Queue Y with requestId: "ABC345"
  6. Function A sees this message with requestId: "ABC345" and returns the HTTP response

How can we get Function A to pick up only the request that it is waiting for?

The getMessage method doesn't seem to be able to selectively listen to a queue, only to grab the top message:

getMessage(queue [, options], callback)

Another angle on this would be if we want multiple Worker Functions to listen to Queue X. Function C would process all requests that have requestType: "query" and Function D would process all requestType: "blob". Without this filtering we would use one queue per Worker function. Is that the right way to do it, too?

Note: We're using node.js but I'm assuming that the Queue API's are equivalent across all SDK's.

Solution

  • Azure Queues really don't do request-response. Http processing should not be waiting on queues. Http messages should return quickly (synchronously, < 1 minute), whereas queues are used for longer asynchronous background processing. If Http is waiting on queues, it should be using a 202 long-running pattern.

    Consider:

    1. keep using Http. You can port to Functions and keep your underlying patterns.
    2. Use queues in a fully asynchronous matter. So A queues a message to kick off B and returns; A2 listens on the response from B.
    3. Check out the Durable Functions preview. This allows synchronous calls exactly like what you want. It's in preview, but see https://github.com/Azure/azure-functions-durable-extension