I am trying to deploy a pre-trained ML model (saved as .h5 file) to Azure ML. I have created an AKS cluster and trying to deploy the model as shown below:
from azureml.core import Workspace
from azureml.core.model import Model
from azureml.core.environment import Environment
from azureml.core.conda_dependencies import CondaDependencies
from azureml.core.model import InferenceConfig
from azureml.core.webservice import AksWebservice, LocalWebservice
from azureml.core.compute import ComputeTarget
workspace = Workspace.from_config(path="config.json")
env = Environment.get(workspace, name='AzureML-TensorFlow-1.13-GPU')
# Installing packages present in my requirements file
with open('requirements.txt') as f:
dependencies = f.readlines()
dependencies = [x.strip() for x in dependencies if '# ' not in x]
dependencies.append("azureml-defaults>=1.0.45")
env.python.conda_dependencies = CondaDependencies.create(conda_packages=dependencies)
# Including the source folder so that all helper scripts are included in my deployment
inference_config = InferenceConfig(entry_script='app.py', environment=env, source_directory='./ProcessImage')
aks_target = ComputeTarget(workspace=workspace, name='sketch-ppt-vm')
# Deployment with suitable config
deployment_config = AksWebservice.deploy_configuration(cpu_cores=4, memory_gb=32)
model = Model(workspace, 'sketch-inference')
service = Model.deploy(workspace, "process-sketch-dev", [model], inference_config, deployment_config, deployment_target=aks_target, overwrite=True)
service.wait_for_deployment(show_output = True)
print(service.state)
My main entry script requires some additional helper scripts, which I include by mentioning the source folder in my inference config.
I was expecting that the helper scripts I add should be able to access the packages installed while setting up the environment during deployment, but I get ModuleNotFoundError.
Here is the error output, along with the a couple of environment variables I printed while executing entry script:
AZUREML_MODEL_DIR ---- azureml-models/sketch-inference/1
PYTHONPATH ---- /azureml-envs/azureml_6dc005c11e151f8d9427c0c6091a1bb9/lib/python3.6/site-packages:/var/azureml-server:
PATH ---- /azureml-envs/azureml_6dc005c11e151f8d9427c0c6091a1bb9/bin:/opt/miniconda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/intel/compilers_and_libraries/linux/mpi/bin64
Exception in worker process
Traceback (most recent call last):
File "/azureml-envs/azureml_6dc005c11e151f8d9427c0c6091a1bb9/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/azureml-envs/azureml_6dc005c11e151f8d9427c0c6091a1bb9/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/azureml-envs/azureml_6dc005c11e151f8d9427c0c6091a1bb9/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/azureml-envs/azureml_6dc005c11e151f8d9427c0c6091a1bb9/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/azureml-envs/azureml_6dc005c11e151f8d9427c0c6091a1bb9/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/azureml-envs/azureml_6dc005c11e151f8d9427c0c6091a1bb9/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/azureml-envs/azureml_6dc005c11e151f8d9427c0c6091a1bb9/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
__import__(module)
File "/var/azureml-server/wsgi.py", line 1, in <module>
import create_app
File "/var/azureml-server/create_app.py", line 3, in <module>
from app import main
File "/var/azureml-server/app.py", line 32, in <module>
from aml_blueprint import AMLBlueprint
File "/var/azureml-server/aml_blueprint.py", line 25, in <module>
import main
File "/var/azureml-app/main.py", line 12, in <module>
driver_module_spec.loader.exec_module(driver_module)
File "/structure/azureml-app/ProcessImage/app.py", line 16, in <module>
from ProcessImage.samples.coco.inference import run as infer
File "/var/azureml-app/ProcessImage/samples/coco/inference.py", line 1, in <module>
import skimage.io
ModuleNotFoundError: No module named 'skimage'
The existing answers related to this aren't of much help. I believe there must be a simpler way to fix this, since AzureML specifically provides the feature to setup environment with pip/conda packages installed either by supplying requirements.txt file or individually.
What am I missing here? Kindly help.
So, after some trial and error, creating a fresh environment and then adding the packages solved the problem for me. I am still not clear on why this didn't work when I tried to use Environment.from_pip_requirements(). A detailed answer in this regard would be interesting to read.
My primary task was inference - object detection given an image, and we have our own model developed by our team. There are two types of imports I wanted to have:
1. Standard python packages (installed through pip)
This was solved by creating conda dependencies and add it to env object (Step 2)
2. Methods/vars from helper scripts (if you have pre/post processing to be done during model inference):
This was done by mentioning source_directory in InferenceConfig (step 3)
Here is my updated script which combines Environment creation, Inference and Deployment configs and using existing compute in the workspace (created through portal).
from azureml.core import Workspace
from azureml.core.model import Model
from azureml.core.environment import Environment, DEFAULT_GPU_IMAGE
from azureml.core.conda_dependencies import CondaDependencies
from azureml.core.model import InferenceConfig
from azureml.core.webservice import AksWebservice, LocalWebservice
from azureml.core.compute import ComputeTarget
# 1. Instantiate the workspace
workspace = Workspace.from_config(path="config.json")
# 2. Setup the environment
env = Environment('sketchenv')
with open('requirements.txt') as f: # Fetch all dependencies as a list
dependencies = f.readlines()
dependencies = [x.strip() for x in dependencies if '# ' not in x]
env.docker.base_image = DEFAULT_GPU_IMAGE
env.python.conda_dependencies = CondaDependencies.create(conda_packages=['numpy==1.17.4', 'Cython'], pip_packages=dependencies)
# 3. Inference Config
inference_config = InferenceConfig(entry_script='app.py', environment=env, source_directory='./ProcessImage')
# 4. Compute target (using existing cluster from the workspacke)
aks_target = ComputeTarget(workspace=workspace, name='sketch-ppt-vm')
# 5. Deployment config
deployment_config = AksWebservice.deploy_configuration(cpu_cores=6, memory_gb=100)
# 6. Model deployment
model = Model(workspace, 'sketch-inference') # Registered model (which contains model files/folders)
service = Model.deploy(workspace, "process-sketch-dev", [model], inference_config, deployment_config, deployment_target=aks_target, overwrite=True)
service.wait_for_deployment(show_output = True)
print(service.state)