Search code examples
google-cloud-platformjupyter-labgoogle-cloud-vertex-aigcp-ai-platform-notebook

Create custom kernel via post-startup script in Vertex AI User Managed notebook


I am trying to use a post-startup script to create a Vertex AI User Managed Notebook whose Jupyter Lab has a dedicated virtual environment and corresponding computing kernel when first launched. I have had success creating the instance and then, as a second manual step from within the Jupyter Lab > Terminal, running a bash script like so:

#!/bin/bash
cd /home/jupyter
mkdir -p env
cd env
python3 -m venv envName --system-site-packages
source envName/bin/activate
envName/bin/python3 -m pip install --upgrade pip
python -m ipykernel install --user --name=envName
pip3 install geemap --user 
pip3 install earthengine-api --user 
pip3 install ipyleaflet --user 
pip3 install folium --user 
pip3 install voila --user 
pip3 install jupyterlab_widgets
deactivate
jupyter labextension install --no-build @jupyter-widgets/jupyterlab-manager jupyter-leaflet
jupyter lab build --dev-build=False --minimize=False
jupyter labextension enable @jupyter-widgets/jupyterlab-manager

However, I have not had luck using this code as a post-startup script (being supplied through the console creation tools, as opposed to command line, thus far). When I open Jupyter Lab and look at the relevant structures, I find that there is no environment or kernel. Could someone please provide a working example that accomplishes my aim, or otherwise describe the order of build steps that one would follow?


Solution

  • Post startup scripts run as root. When you run:

    python -m ipykernel install --user --name=envName
    

    Notebook is using current user which is root vs when you use Terminal, which is running as jupyter user.

    Option 1) Have 2 scripts:

    • Script A. Contents specified in original post. Example: gs://newsml-us-central1/so73649262.sh
    • Script B. Downloads script and execute it as jupyter. Example: gs://newsml-us-central1/so1.sh and use it as post-startup script.
    #!/bin/bash
    
    set -x
    
    gsutil cp gs://newsml-us-central1/so73649262.sh /home/jupyter
    chown jupyter /home/jupyter/so73649262.sh
    chmod a+x /home/jupyter/so73649262.sh
    su -c '/home/jupyter/so73649262.sh' jupyter
    

    Option 2) Create a file in bash using EOF. Write the contents into a single file and execute it as mentioned above.