What is the best way to containarise model using Azure ML Pipeline

Question

We are migrating from Azure ML Python SDK V1 to V2.

I have a requirement to train a model, containerize using a custom Dockerfile, and push it to a container registry.

Previously with V1 SDK, I used docker-in-docker build option to achieve the same. To accomplish this I created a RunConfiguration(), expose docker.sock inside the docker by setting volume mount command ["-v", "/var/run/docker.sock:/var/run/docker.sock"] in docker.arguments and used docker build command inside the container.

But looks like with V2 SDK, RunConfiguration() is removed and command() component is introduced. This command component has a docker_args to pass extra arguments to docker run command.

But, when I set the same volume mount command in this parameter, I get the error:

create /var/run/docker.sock: \" /var/run/docker.sock\" includes invalid characters for a local volume name, only \"[a-zA-Z0-9][a-zA-Z0-9_.-]\" are allowed. If you intended to pass a host directory, use absolute path"}. Container using custom Docker arguments failed to start. Try reviewing and adjusting the Docker arguments.

There also is Bug similar to this open for some time: https://github.com/Azure/azure-sdk-for-python/issues/30466

This got me wondering what is the best way or the recommended way to build container images out of Azure ML Pipelines.

Any input and ideas on how anyone is handling such requirements will be greatly beneficial.

Sample Code:

Following a sample model packaging command component.

model_package_cmd = command(
  name="Package model",
  display_name="Package model",
  description="Package model as container image",
  code="./src",
  command="./build/docker-build.sh ${{inputs.model_path}}",
  compute="training-cluster",
  environment=get_package_runtime_env(),
  environment_variables=get_runtime_env_vars(),
  docker_args="-v /var/run/docker.sock:/var/run/docker.sock",
  is_deterministic=False,
  inputs={
    "model_path": Input(type=AssetTypes.URI_FOLDER),
    "test_reports": Input(type=AssetTypes.URI_FOLDER),
  },
)

When the pipeline runs for this model packaging node, it throws up the error:

Failed to start Docker container 132f7af967554b3188faed6a1b791434-execution-wrapper due to: API queried with a bad parameter: {"message":"create  /var/run/docker.sock: \" /var/run/docker.sock\" includes invalid characters for a local volume name, only \"[a-zA-Z0-9][a-zA-Z0-9_.-]\" are allowed. If you intended to pass a host directory, use absolute path"}.
Container using custom Docker arguments failed to start. Try reviewing and adjusting the Docker arguments.

Answer 1

You give args like below.

-v=/var/run/docker.sock:/var/run/docker.sock

or

--mount=target=/tmp/docker.sock,source=/var/run/docker.sock,type=bind

Below is the .sh file using.

Output:

Code:

model_package_cmd = command(
  name="package-model16",
  display_name="Package model",
  description="Package model as container image",
  code="./src",
  command="./docker.sh",
  compute="jgs-cluster",
  environment=env_docker_context,
  environment_variables=dict(COMPOSE_CONVERT_WINDOWS_PATHS=1),
  docker_args="-v=/var/run/docker.sock:/var/run/docker.sock", #or --mount=target=/tmp/docker.sock,source=/var/run/docker.sock,type=bind
  is_deterministic=False
)