I would like to define a CMake (v3.30.2) function that uses FetchContent_Declare() and FetchContent_MakeAvailable() to download some artifact from my GitLab instance.
The URL is
https://gitlab.example.com/api/v4/projects/<project-id>/jobs/<job-id>/artifacts
where
<project-id> is the project's ID<job-id> is the job ID from some pipeline that is part of the projectSince it's a private repository I need to pass an access token. In addition, in order to make it more flexible, I would like to set the project-id and job-id using my function's arguments.
Below is the current state of my function
include(FetchContent)
function(download_file_gitlab name token proj_id job_id)
set(URL_STR "https://gitlab.example.com/api/v4/projects/${proj_id}/jobs/${job_id}/artifacts")
message("URL set to ${URL_STR}")
FetchContent_Declare(download_${name}
QUIET
URL "${URL_STR}"
HTTP_HEADER: "PRIVATE-TOKEN: ${token}"
# BINARY_DIR: "${CMAKE_BINARY_DIR}/${name}"
)
if(NOT download_${name}_POPULATED)
FetchContent_MakeAvailable(download_${name})
endif()
endfunction(download_file_gitlab)
which I call like this
download_file_gitlab(
release
sometokenthatworks
68522
7013628
)
The rest of the CMakeLists.txt can be anything so the content is not important. I am currently literally using an almost empty one with the cmake_minimum_required() and project() followed by the FetchContent part and function call.
cmake -Bbuild -G "Visual Studio 16 2019"
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19043.
-- The C compiler identification is MSVC 19.29.30152.0
-- The CXX compiler identification is MSVC 19.29.30152.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
URL set to https://gitlab.example.com/api/v4/projects/68522/jobs/7013628/artifacts
CMake Warning (dev) at C:/Program Files/CMake/share/cmake-3.30/Modules/FetchContent.cmake:1373 (message):
The DOWNLOAD_EXTRACT_TIMESTAMP option was not given and policy CMP0135 is
not set. The policy's OLD behavior will be used. When using a URL
download, the timestamps of extracted files should preferably be that of
the time of extraction, otherwise code that depends on the extracted
contents might not be rebuilt if the URL changes. The OLD behavior
preserves the timestamps from the archive instead, but this is usually not
what you want. Update your project to the NEW behavior or specify the
DOWNLOAD_EXTRACT_TIMESTAMP option with a value of true to avoid this
robustness issue.
Call Stack (most recent call first):
CMakeLists.txt:26 (FetchContent_Declare)
CMakeLists.txt:56 (download_file_gitlab)
This warning is for project developers. Use -Wno-dev to suppress it.
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19043.
CMake Error at C:/Program Files/CMake/share/cmake-3.30/Modules/ExternalProject/shared_internal_commands.cmake:1127 (message):
At least one entry of URL is a path (invalid in a list)
Call Stack (most recent call first):
C:/Program Files/CMake/share/cmake-3.30/Modules/ExternalProject.cmake:3035 (_ep_add_download_command)
CMakeLists.txt:22 (ExternalProject_Add)
-- Configuring incomplete, errors occurred!
CMake Error at C:/Program Files/CMake/share/cmake-3.30/Modules/FetchContent.cmake:1906 (message):
CMake step for download_imgui-release failed: 1
Call Stack (most recent call first):
C:/Program Files/CMake/share/cmake-3.30/Modules/FetchContent.cmake:1609 (__FetchContent_populateSubbuild)
C:/Program Files/CMake/share/cmake-3.30/Modules/FetchContent.cmake:2145:EVAL:2 (__FetchContent_doPopulation)
C:/Program Files/CMake/share/cmake-3.30/Modules/FetchContent.cmake:2145 (cmake_language)
C:/Program Files/CMake/share/cmake-3.30/Modules/FetchContent.cmake:2384 (__FetchContent_Populate)
CMakeLists.txt:33 (FetchContent_MakeAvailable)
CMakeLists.txt:56 (download_file_gitlab)
-- Configuring incomplete, errors occurred!
Note that the artifact does exist and I am able to download it, e.g. using cURL inside the Git Bash:
curl -X GET -H "PRIVATE-TOKEN: sometokenthatworks" https://gitlab.example.com/api/v4/projects/68522/jobs/7013628/artifacts > artifacts.zip
The resulting ZIP archive can be viewed and extracted.
I am not even sure if I can pass variables to FetchContent_Declare() and was unable to find information on that in the CMake documentation (v3.30.2).
The inclusion of : after some of the parameters is incorrect, which renders those invalid and not being taken into consideration by the FetchContent module
The fixed version can be seen below:
function(download_file_gitlab name token proj_id job_id)
set(_URL "https://gitlab.example.com/api/v4/projects/${proj_id}/jobs/${job_id}/artifacts")
set(_HEADER "PRIVATE-TOKEN: ${token}")
message("GET Request: ${_URL} (Headers: ${_HEADER})")
FetchContent_Declare(download_${name}
#QUIET
URL ${_URL}
HTTP_HEADER ${_HEADER}
DOWNLOAD_NO_EXTRACT 0
DOWNLOAD_EXTRACT_TIMESTAMP 1
DOWNLOAD_NO_PROGRESS 0
)
if(NOT download_${name}_POPULATED)
FetchContent_MakeAvailable(download_${name})
endif()
endfunction(download_file_gitlab)
NOTE: The header needs to be patched if the CMake configuration takes place inside a CI job and the token is automatically generated. In that case JOB-TOKEN instead of PRIVATE-TOKEN has to be used. This can be done by replacing the specific substring or patching the file with the function, or having two versions of it that are used based on the environment (e.g. passing a CMake cmd parameter that ensures the proper function is used).