How to use Async or Multithread on Azure Managed Endpoint

Question

I deployed a model using Azure ML managed endpoint, but I found a bottleneck.

I'm using Azure ML Managed Endpoint to host ML models for object prediction. Our endpoint receives a URL of a picture and is responsible for downloading and predicting the image.

The problem is the bottleneck: each image is downloaded one at a time (synchronously), which is very slow.

Is there a way to download images async or to create multiple threads ? I expected a way to make if faster.

Answer 1

We recommend to use Azure Blob storage to host the images and then use blob storage SDK to fetch the images.