I have a CICD pipeline to execute some infrastructure deployment stuff with Pulumi, setup in the following .gitlab-ci.yml:
prod_job:
stage: prod
only:
- main
before_script:
- eval "$(ssh-agent -s)"
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
- apt-get update -qy
- apt-get install -y python3-venv
- apt-get install -y python3-pip
script:
- git submodule init
- git submodule update
- python3 -m venv venv
- source venv/bin/activate
- pip3 install -r requirements.txt
- python3 infra.py
The results of a pip3 list if run after pip3 install in the CICD pipeline are:
Package Version
---------- -------
Arpeggio 2.0.0
attrs 23.1.0
dill 0.3.6
grpcio 1.51.3
parver 0.4
pip 23.0.1
protobuf 4.23.3
pulumi 3.73.0
pulumi-ec 0.5.1
PyYAML 6.0
semver 2.13.0
setuptools 66.1.1
six 1.16.0
However, when the infra.py file executes, it fails noting that there is no such file or directory called Pulumi:
File "/usr/lib/python3.11/subprocess.py", line 1024, in init self._execute_child(args, executable, preexec_fn, close_fds, File "/usr/lib/python3.11/subprocess.py", line 1901, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: 'pulumi'
The relevant sections of the 'infra.py' file are:
import sys
import json
import pulumi
import pulumi_ec as ec
from pulumi import automation as auto
Why is this failing to execute if I have properly sourced the virutal environment, and installed the requirements? Thanks
You need to install pulumi (which is separate from just installing the Python SDK for Pulumi) in your GitLab job. That is to say, you should be able to call the pulumi binary (such as pulumi version to show the version of pulumi installed).
The Python code is trying to call this binary and it is failing because it is either not installed or not on PATH.