Search code examples
azureazure-storageapache-spark-2.0

Apache Spark connector to read from Azure Queue service?


This can be more of a configuration question, but could not find a specific answer to the problem I am trying to solve.

I am looking for a connector to read from Azure Storage Queue Service through Spark, though there are connectors available for Azure Blob storage. Want to confirm if there is a connector to read from Azure Storage Queue or somebody have solved the problem before.


Solution

  • @jdk2588, there isn't a specific connector from Azure Storage Queue Service via Spark specifically, however you should consider the use of process workflows: "Azure Queue storage provides cloud messaging between application components. Queue storage also supports managing asynchronous tasks and building process work flows." More details can be found here: https://learn.microsoft.com/en-us/connectors/azurequeues/