I'm working with a project that uses espressif and to build it on my machine with docker way I do the following:
docker run --rm -v $PWD:/project -w /project espressif/idf:v4.2.2 idf.py build
I would like to elaborate a declarative pipeline, and I would like to execute the command equivalent to the one above. The way I implemented it based on other examples that worked, and the log result below.
I don't understand why the way to pass these 'idf.py build' arguments in the 'steps' block is not working. Does anyone have any ideas?
Reading the log and doing some google searches, I believe it's the jenkins plugin that can't handle the command because the image uses the entrypoint feature.
My pipeline:
pipeline {
agent any
environment {
PROJ_NAME = 'test'
}
stages {
stage('Checkout') {
steps {
git url: 'ssh://[email protected]/john/iot-project.git'
}
}
stage('Build') {
agent {
docker {
image 'espressif/idf:v4.2.2'
args '--rm -v $PWD:/project -w /project'
reuseNode true
}
}
steps{
sh 'idf.py build'
}
}
}
}
Error snippet:
[Pipeline] withDockerContainer
Jenkins does not seem to be running inside a container
$ docker run -t -d -u 1000:1000 --rm -v $PWD:/project -w /project -w /var/lib/jenkins/workspace/iot-project-TEST -v /var/lib/jenkins/workspace/iot-project-TEST:/var/lib/jenkins/workspace/iot-project-TEST:rw,z -v /var/lib/jenkins/workspace/iot-project-TEST@tmp:/var/lib/jenkins/workspace/iot-project-TEST@tmp:rw,z -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** espressif/idf:v4.2.2 cat
$ docker top 81920a1146eabe9bf5c08339a682d81ac23777de0421895e1184d2a8ef27fc8c -eo pid,comm
ERROR: The container started but didn't run the expected command. Please double check your ENTRYPOINT does execute the command passed as docker run argument, as required by official docker images (see https://github.com/docker-library/official-images#consistency for entrypoint consistency requirements).
Alternatively you can force image entrypoint to be disabled by adding option `--entrypoint=''`.
[Pipeline] {
[Pipeline] sh
+ idf.py build
/var/lib/jenkins/workspace/iot-project-TEST@tmp/durable-b8bf6ce0/script.sh: 1: /var/lib/jenkins/workspace/iot-project-TEST@tmp/durable-b8bf6ce0/script.sh: idf.py: not found
[Pipeline] }
$ docker stop --time=1 81920a1146eabe9bf5c08339a682d81ac23777de0421895e1184d2a8ef27fc8c
$ docker rm -f 81920a1146eabe9bf5c08339a682d81ac23777de0421895e1184d2a8ef27fc8c
[Pipeline] // withDockerContainer
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 127
Finished: FAILURE
UPDATE1: The project build with the official espressif image works when I run the command directly, for example:
pipeline {
agent any
environment {
PROJ_NAME = 'test'
}
stages {
stage('Checkout') {
steps {
git url: 'ssh://[email protected]/john/iot-project.git'
}
}
stage('Build') {
steps{
sh 'docker run --rm -v $WORKSPACE/ESPComm:/project -w /project espressif/idf:v4.2.2 idf.py build'
}
}
}
}
UPDATE2:
Without the --entrypoint='' argument an error is always thrown, so I keep that argument. I will present the log of ls and pwd commands after running docker. Note: cat and top are jenkins' own tricks so that the commands inside the step block are executed
pipeline {
agent any
environment {
PROJ_NAME = 'test'
}
stages {
stage('Checkout') {
steps {
git url: 'ssh://[email protected]/john/iot-project.git'
}
}
stage('Build') {
agent {
docker {
image 'espressif/idf:v4.2.2'
args '''--rm -v $PWD:/project -w /project --entrypoint='' '''
reuseNode true
}
}
steps{
/*sh '''
source /opt/esp/idf/export.sh
idf.py build
'''*/
sh 'ls'
sh 'pwd'
}
}
}
}
[Pipeline] withDockerContainer
Jenkins does not seem to be running inside a container
$ docker run -t -d -u 1000:1000 --rm -v $PWD:/project -w /project --entrypoint= -w /var/lib/jenkins/workspace/iot-project-TEST -v /var/lib/jenkins/workspace/iot-project-TEST:/var/lib/jenkins/workspace/iot-project-TEST:rw,z -v /var/lib/jenkins/workspace/iot-project-TEST@tmp:/var/lib/jenkins/workspace/iot-project-TEST@tmp:rw,z -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** espressif/idf:v4.2.2 cat
$ docker top 69d5a4450c2463a6d8153582248796a15fa94ed04ef3d45c76c9a2358b8740cd -eo pid,comm
[Pipeline] {
[Pipeline] sh
+ ls
ESPComm
ESPComm@tmp
Grafana
README.md
xctu_template.xml
[Pipeline] sh
+ pwd
/var/lib/jenkins/workspace/iot-project-TEST
[Pipeline] }
$ docker stop --time=1 69d5a4450c2463a6d8153582248796a15fa94ed04ef3d45c76c9a2358b8740cd
$ docker rm -f 69d5a4450c2463a6d8153582248796a15fa94ed04ef3d45c76c9a2358b8740cd
[Pipeline] // withDockerContainer
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
UPDATE3: Now running without entrypoint command. Check in log the error:
ERROR: The container started but didn't run the expected command. Please double check your ENTRYPOINT does execute the command passed as docker run argument, as required by official docker images (see https://github.com/docker-library/official-images#consistency for entrypoint consistency requirements).
Alternatively you can force image entrypoint to be disabled by adding option `--entrypoint=''`.
pipeline {
agent any
environment {
PROJ_NAME = 'test'
}
stages {
stage('Checkout') {
steps {
git url: 'ssh://[email protected]/john/iot-project.git'
}
}
stage('Build') {
agent {
docker {
image 'espressif/idf:v4.2.2'
args '''--rm -v $PWD:/project -w /project '''
reuseNode true
}
}
steps{
sh '''
pwd
ls
#source /opt/esp/idf/export.sh
. $IDF_PATH/export.sh
idf.py build
'''
}
}
}
}
[Pipeline] withDockerContainer
Jenkins does not seem to be running inside a container
$ docker run -t -d -u 1000:1000 --rm -v $PWD:/project -w /project -w /var/lib/jenkins/workspace/iot-project-TEST -v /var/lib/jenkins/workspace/iot-project-TEST:/var/lib/jenkins/workspace/iot-project-TEST:rw,z -v /var/lib/jenkins/workspace/iot-project-TEST@tmp:/var/lib/jenkins/workspace/iot-project-TEST@tmp:rw,z -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** espressif/idf:v4.2.2 cat
$ docker top 217167975bdf63d861215e12f5e3b2ef35d21681fe77bb7608a6c1cc0d03c237 -eo pid,comm
ERROR: The container started but didn't run the expected command. Please double check your ENTRYPOINT does execute the command passed as docker run argument, as required by official docker images (see https://github.com/docker-library/official-images#consistency for entrypoint consistency requirements).
Alternatively you can force image entrypoint to be disabled by adding option `--entrypoint=''`.
[Pipeline] {
[Pipeline] sh
+ pwd
/var/lib/jenkins/workspace/iot-project-TEST
+ ls
ESPComm
ESPComm@tmp
Grafana
README.md
xctu_template.xml
+ . /opt/esp/idf/export.sh
+ idf_export_main
+ [ -n ]
+ [ -z /opt/esp/idf ]
+ [ ! -d /opt/esp/idf ]
+ [ ! -f /opt/esp/idf/tools/idf.py ]
+ [ ! -f /opt/esp/idf/tools/idf_tools.py ]
+ export IDF_PATH=/opt/esp/idf
+ old_path=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ echo Detecting the Python interpreter
Detecting the Python interpreter
+ . /opt/esp/idf/tools/detect_python.sh
+ ESP_PYTHON=python
+ echo Checking "python" ...
Checking "python" ...
+ python -c import sys; print(sys.version_info.major)
+ [ 3 = 3 ]
+ ESP_PYTHON=python
+ break
+ python --version
Python 3.6.9
+ echo "python" has been detected
"python" has been detected
+ echo Adding ESP-IDF tools to PATH...
Adding ESP-IDF tools to PATH...
+ export IDF_TOOLS_EXPORT_CMD=/opt/esp/idf/export.sh
+ export IDF_TOOLS_INSTALL_CMD=/opt/esp/idf/install.sh
+ python /opt/esp/idf/tools/idf_tools.py export
+ idf_exports=export OPENOCD_SCRIPTS="/opt/esp/tools/openocd-esp32/v0.10.0-esp32-20200709/openocd-esp32/share/openocd/scripts";export IDF_PYTHON_ENV_PATH="/opt/esp/python_env/idf4.2_py3.6_env";export PATH="/opt/esp/tools/xtensa-esp32-elf/esp-2020r3-8.4.0/xtensa-esp32-elf/bin:/opt/esp/tools/xtensa-esp32s2-elf/esp-2020r3-8.4.0/xtensa-esp32s2-elf/bin:/opt/esp/tools/esp32ulp-elf/2.28.51-esp-20191205/esp32ulp-elf-binutils/bin:/opt/esp/tools/esp32s2ulp-elf/2.28.51-esp-20191205/esp32s2ulp-elf-binutils/bin:/opt/esp/tools/cmake/3.16.4/bin:/opt/esp/tools/openocd-esp32/v0.10.0-esp32-20200709/openocd-esp32/bin:/opt/esp/python_env/idf4.2_py3.6_env/bin:/opt/esp/idf/tools:$PATH"
+ eval export OPENOCD_SCRIPTS="/opt/esp/tools/openocd-esp32/v0.10.0-esp32-20200709/openocd-esp32/share/openocd/scripts";export IDF_PYTHON_ENV_PATH="/opt/esp/python_env/idf4.2_py3.6_env";export PATH="/opt/esp/tools/xtensa-esp32-elf/esp-2020r3-8.4.0/xtensa-esp32-elf/bin:/opt/esp/tools/xtensa-esp32s2-elf/esp-2020r3-8.4.0/xtensa-esp32s2-elf/bin:/opt/esp/tools/esp32ulp-elf/2.28.51-esp-20191205/esp32ulp-elf-binutils/bin:/opt/esp/tools/esp32s2ulp-elf/2.28.51-esp-20191205/esp32s2ulp-elf-binutils/bin:/opt/esp/tools/cmake/3.16.4/bin:/opt/esp/tools/openocd-esp32/v0.10.0-esp32-20200709/openocd-esp32/bin:/opt/esp/python_env/idf4.2_py3.6_env/bin:/opt/esp/idf/tools:$PATH"
+ export OPENOCD_SCRIPTS=/opt/esp/tools/openocd-esp32/v0.10.0-esp32-20200709/openocd-esp32/share/openocd/scripts
+ export IDF_PYTHON_ENV_PATH=/opt/esp/python_env/idf4.2_py3.6_env
+ export PATH=/opt/esp/tools/xtensa-esp32-elf/esp-2020r3-8.4.0/xtensa-esp32-elf/bin:/opt/esp/tools/xtensa-esp32s2-elf/esp-2020r3-8.4.0/xtensa-esp32s2-elf/bin:/opt/esp/tools/esp32ulp-elf/2.28.51-esp-20191205/esp32ulp-elf-binutils/bin:/opt/esp/tools/esp32s2ulp-elf/2.28.51-esp-20191205/esp32s2ulp-elf-binutils/bin:/opt/esp/tools/cmake/3.16.4/bin:/opt/esp/tools/openocd-esp32/v0.10.0-esp32-20200709/openocd-esp32/bin:/opt/esp/python_env/idf4.2_py3.6_env/bin:/opt/esp/idf/tools:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ which python
+ echo Using Python interpreter in /opt/esp/python_env/idf4.2_py3.6_env/bin/python
Using Python interpreter in /opt/esp/python_env/idf4.2_py3.6_env/bin/python
+ echo Checking if Python packages are up to date...
Checking if Python packages are up to date...
+ python /opt/esp/idf/tools/check_python_dependencies.py
Python requirements from /opt/esp/idf/requirements.txt are satisfied.
+ IDF_ADD_PATHS_EXTRAS=/opt/esp/idf/components/esptool_py/esptool
+ IDF_ADD_PATHS_EXTRAS=/opt/esp/idf/components/esptool_py/esptool:/opt/esp/idf/components/espcoredump
+ IDF_ADD_PATHS_EXTRAS=/opt/esp/idf/components/esptool_py/esptool:/opt/esp/idf/components/espcoredump:/opt/esp/idf/components/partition_table
+ IDF_ADD_PATHS_EXTRAS=/opt/esp/idf/components/esptool_py/esptool:/opt/esp/idf/components/espcoredump:/opt/esp/idf/components/partition_table:/opt/esp/idf/components/app_update
+ export PATH=/opt/esp/idf/components/esptool_py/esptool:/opt/esp/idf/components/espcoredump:/opt/esp/idf/components/partition_table:/opt/esp/idf/components/app_update:/opt/esp/tools/xtensa-esp32-elf/esp-2020r3-8.4.0/xtensa-esp32-elf/bin:/opt/esp/tools/xtensa-esp32s2-elf/esp-2020r3-8.4.0/xtensa-esp32s2-elf/bin:/opt/esp/tools/esp32ulp-elf/2.28.51-esp-20191205/esp32ulp-elf-binutils/bin:/opt/esp/tools/esp32s2ulp-elf/2.28.51-esp-20191205/esp32s2ulp-elf-binutils/bin:/opt/esp/tools/cmake/3.16.4/bin:/opt/esp/tools/openocd-esp32/v0.10.0-esp32-20200709/openocd-esp32/bin:/opt/esp/python_env/idf4.2_py3.6_env/bin:/opt/esp/idf/tools:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ [ -n ]
+ echo Updated PATH variable:
Updated PATH variable:
+ echo /opt/esp/idf/components/esptool_py/esptool:/opt/esp/idf/components/espcoredump:/opt/esp/idf/components/partition_table:/opt/esp/idf/components/app_update:/opt/esp/tools/xtensa-esp32-elf/esp-2020r3-8.4.0/xtensa-esp32-elf/bin:/opt/esp/tools/xtensa-esp32s2-elf/esp-2020r3-8.4.0/xtensa-esp32s2-elf/bin:/opt/esp/tools/esp32ulp-elf/2.28.51-esp-20191205/esp32ulp-elf-binutils/bin:/opt/esp/tools/esp32s2ulp-elf/2.28.51-esp-20191205/esp32s2ulp-elf-binutils/bin:/opt/esp/tools/cmake/3.16.4/bin:/opt/esp/tools/openocd-esp32/v0.10.0-esp32-20200709/openocd-esp32/bin:/opt/esp/python_env/idf4.2_py3.6_env/bin:/opt/esp/idf/tools:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
/opt/esp/idf/components/esptool_py/esptool:/opt/esp/idf/components/espcoredump:/opt/esp/idf/components/partition_table:/opt/esp/idf/components/app_update:/opt/esp/tools/xtensa-esp32-elf/esp-2020r3-8.4.0/xtensa-esp32-elf/bin:/opt/esp/tools/xtensa-esp32s2-elf/esp-2020r3-8.4.0/xtensa-esp32s2-elf/bin:/opt/esp/tools/esp32ulp-elf/2.28.51-esp-20191205/esp32ulp-elf-binutils/bin:/opt/esp/tools/esp32s2ulp-elf/2.28.51-esp-20191205/esp32s2ulp-elf-binutils/bin:/opt/esp/tools/cmake/3.16.4/bin:/opt/esp/tools/openocd-esp32/v0.10.0-esp32-20200709/openocd-esp32/bin:/opt/esp/python_env/idf4.2_py3.6_env/bin:/opt/esp/idf/tools:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ unset old_path
+ unset paths
+ unset path_prefix
+ unset path_entry
+ unset IDF_ADD_PATHS_EXTRAS
+ unset idf_exports
+ unset ESP_PYTHON
+ echo Done! You can now compile ESP-IDF projects.
Done! You can now compile ESP-IDF projects.
+ echo Go to the project directory and run:
Go to the project directory and run:
+ echo
+ echo idf.py build
idf.py build
+ echo
+ unset realpath_int
+ unset idf_export_main
+ idf.py build
Executing action: all (aliases: build)
CMakeLists.txt not found in project directory /var/lib/jenkins/workspace/iot-project-TEST
Your environment is not configured to handle unicode filenames outside of ASCII range. Environment variable LC_ALL is temporary set to C.UTF-8 for unicode support.
[Pipeline] }
$ docker stop --time=1 217167975bdf63d861215e12f5e3b2ef35d21681fe77bb7608a6c1cc0d03c237
$ docker rm -f 217167975bdf63d861215e12f5e3b2ef35d21681fe77bb7608a6c1cc0d03c237
[Pipeline] // withDockerContainer
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 2
Finished: FAILURE
Example with an image I created, it uses ubuntu as a base, but has no entrypoint. I've already managed to successfully eclipse headless-build as follows. :
stage('Build') {
agent {
docker {
image 'tool/stm32-cubeide-image:1.0'
reuseNode true
}
}
steps {
sh '/opt/stm32cubeide/headless-build.sh -importAll $WORKSPACE -data $WORKSPACE -cleanBuild $DIR/$OPT_BUILD'
}
}
You can do something like the below. Before executing the build command try sourcing the /opt/esp/idf/export.sh which will set the environment so you can execute the build command.
sh'''
source /opt/esp/idf/export.sh
idf.py build
'''
Here is your full pipeline with the necessary changes.
pipeline {
agent any
environment {
PROJ_NAME = 'test'
}
stages {
stage('Checkout') {
steps {
git url: 'ssh://[email protected]/john/iot-project.git'
}
}
stage('Build') {
agent {
docker {
image 'espressif/idf:v4.2.2'
args '--rm -v $PWD:/project -w /project'
reuseNode true
}
}
steps{
sh '''
#source /opt/esp/idf/export.sh
. $IDF_PATH/export.sh
idf.py build
'''
}
}
}
}
Update
Following is the content in the entrypoint.
#!/usr/bin/env bash
set -e
. $IDF_PATH/export.sh
exec "$@"
So executing the build following ways seems to work for me.
sh'''
. $IDF_PATH/export.sh
idf.py build
'''
or
sh'''
sh /opt/esp/entrypoint.sh idf.py build
'''