Run !docker build from Managed Notebook cell in GCP Vertex AI Workbench

Question

I am trying to push a docker image on Google Cloud Platform container registry to define a custom training job directly inside a notebook.

After having prepared the correct Dockerfile and the URI where to push the image that contains my train.py script, I try to push the image directly in a notebook cell.

The exact command I try to execute is: !docker build ./ -t $IMAGE_URI, where IMAGE_URI is the environmental variable previously defined. However I try to run this command I get the error: /bin/bash: docker: command not found. I also tried to execute it with the magic cell %%bash, importing the subprocess library and also execute the command stored in a .sh file.

Unfortunately none of the above solutions work, they all return the same command not found error with code 127.

If instead I run the command from a bash present in the Jupyterlab it works fine as expected.

Is there any workaround to make the push execute inside the jupyter notebook? I was trying to keep the whole custom training process inside the same notebook.

Answer 1

If you follow this guide to create a user-managed notebook from Vertex AI workbench and select Python 3, then it comes with Docker available.

So you will be able to use Docker commands such as ! docker build . inside the user-managed notebook.

Example: