Search code examples
urlcmaketeamcity

Non-ASCII in URL in CMake ExternalProject_Add

I've had this ExternalProject_Add call in my CMake:

set(SIM_URL https://teamcity.vh.com/guestAuth/rep/download/Sim_Feat/.lastSuccessful/vhnHilsimPlantModel-{build.number}.zip)
set(SIM_FILENAME vhnHilsimPlantModel-{build.number}.zip)

# Download and install
include(ExternalProject)
ExternalProject_Add(get_sim
  DOWNLOAD_NAME ${SIM_FILENAME}
  URL ${SIM_URL}
  CMAKE_ARGS -DCMAKE_INSTALL_PREFIX:PATH=<INSTALL_DIR>
             -DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}
  BUILD_BYPRODUCTS <INSTALL_DIR>/lib/sim_lib.a
  STEP_TARGETS update)

It worked fine with the 2017 version of TeamCity. However, after our DevOps upgraded TeamCity to the 2019 version, this code started failing: 

  file='/home/user/sim/sim-prefix/src/vhnHilsimPlantModel-{build.number}.zip'
Old file will be removed and new file downloaded from URL.
-- Downloading...
   dst='/home/user/build/apps/sim/sim-prefix/src/vhnHilsimPlantModel-{build.number}.zip'
   timeout='none'
-- Using src='https://teamcity.vh.com/guestAuth/rep/download/Sim_Feat/.lastSuccessful/vhnHilsimPlantModel-{build.number}.zip'
-- Retrying...
-- Using src='https://teamcity.vh.com/guestAuth/rep/download/Sim_Feat/.lastSuccessful/vhnHilsimPlantModel-{build.number}.zip'
-- Retry after 5 seconds (attempt #2) ...
-- Using src='https://teamcity.vh.com/guestAuth/rep/download/Sim_Feat/.lastSuccessful/vhnHilsimPlantModel-{build.number}.zip'
-- Retry after 5 seconds (attempt #3) ...
-- Using src='https://teamcity.vh.com/guestAuth/rep/download/Sim_Feat/.lastSuccessful/vhnHilsimPlantModel-{build.number}.zip'
-- Retry after 15 seconds (attempt #4) ...
-- Using src='https://teamcity.vh.com/guestAuth/rep/download/Sim_Feat/.lastSuccessful/vhnHilsimPlantModel-{build.number}.zip'
-- Retry after 60 seconds (attempt #5) ...
-- Using src='https://teamcity.vh.com/guestAuth/rep/download/Sim_Feat/.lastSuccessful/vhnHilsimPlantModel-{build.number}.zip'
CMake Error at vahana_sim-stamp/download-vahana_sim.cmake:157 (message):
  Each download failed!

    error: downloading 'https://teamcity.vh.com/guestAuth/rep/download/Sim_Feat/.lastSuccessful/vhnHilsimPlantModel-{build.number}.zip' failed
         status_code: 22
         status_string: "HTTP response code said error"
         log:
         --- LOG BEGIN ---
           Trying 172.31.21.90...

The only way to fix it is to change { and } to %7B and %7D, respectively. This, technically should have been done in the first place, but it was working. Does anyone know why this change has to be done and/or why the non-ASCII characters have to be encoded when specifying the URL?

I am using CMake 3.10.2.

Solution

  • The TeamCity 9.X documentation for Obtaining Artifacts confirms that curly braces {build.number} could be used in the URL:

    ARTIFACT_PATH is a path to the artifact on the TeamCity server. This path can contain a {build.number} pattern which will be replaced with the build number of the build whose artifact is retrieved.

    However, this caused a few issues (one example here), as there are several reserved characters in the RFC's pertaining to URI/URL standards (in this case RFC 1738). The RFC 1738 standard deems a group of characters "unsafe" (including curly braces), in addition to the reserved characters for URIs:

    Other characters are unsafe because gateways and other transport agents are known to sometimes modify such characters. These characters are "{", "}", "|", "", "^", "~", "[", "]", and "`". [...] All unsafe characters must always be encoded within a URL.

    Thus, the TeamCity software was updated thereafter to support the curly brace encodings as well. The TeamCity 10.X documentation for Obtaining Artifacts was updated to align with the standards for URL encoding:

    ARTIFACT_PATH is a path to the artifact on the TeamCity server. This path can contain a {build.number} pattern (%7Bbuild.number%7D) which will be replaced by TeamCity with the build number of the build whose artifact is retrieved.

    It seems that by the TeamCity 2019 versions, the support for using raw curly braces in the URL has been dropped altogether in alignment with the standard, and only the encoded version is allowed. So you can update your CMake code to something like this:

    set(SIM_FILENAME vhnHilsimPlantModel-%7Bbuild.number%7D.zip)
set(SIM_URL https://teamcity.vh.com/guestAuth/rep/download/Sim_Feat/.lastSuccessful/${SIM_FILENAME})