Trying to copy data in the same bucket via Google Cloud Workflows triggers not found exception

Question

I have this really simple Google Cloud Workflow, that is copying a file inside the same bucket:

main:
  steps:
    - list:
        call: googleapis.storage.v1.objects.list
        args:
          bucket: "**************"
          prefix: "xxx/extra_cost_commute.txt"
        result: listResult
    - log-files:
        call: sys.log
        args:
          data: ${listResult}
          severity: "INFO"
    - copy:
        call: googleapis.storage.v1.objects.copy
        args:
          sourceBucket: "**************"
          sourceObject: "xxx/extra_cost_commute.txt"
          destinationBucket: "**************"
          destinationObject: "yyy/new_extra_cost_commute.txt"
        result: copyResult
    - log-copy:
        call: sys.log
        args:
          data: ${copyResult}
          severity: "INFO"

The list step is returning the object xxx/extra_cost_commute.txt properly, but the copy step is triggering a 404 Not found exception.

HTTP server responded with error code 404
in step "copy", routine "main", line: 15
{
  "body": "Not Found",
  "code": 404,
  "headers": {
    "Alt-Svc": "h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000",
    "Cache-Control": "no-cache, no-store, max-age=0, must-revalidate",
    "Content-Length": "9",
    "Content-Type": "text/html; charset=UTF-8",
    "Date": "Mon, 13 May 2024 19:42:39 GMT",
    "Expires": "Mon, 01 Jan 1990 00:00:00 GMT",
    "Pragma": "no-cache",
    "Server": "UploadServer",
    "Vary": "Origin, X-Origin",
    "X-Guploader-Uploadid": ""
  },
  "message": "HTTP server responded with error code 404",
  "tags": [
    "HttpError"
  ]
}

Why? What am I doing wrong? I'm lost.

NOTE: I tested to copy using gcloud in a gcr.io/google.com/cloudsdktool/google-cloud-cli:alpine via a batch connector and it is working fine.

Answer 1

Since your sourceObject and DestinationObject contains /, you should encode the path. In Workflows you can utilize the function text.url_encode. I tried your workflow and it should work if you change your copy step to:

copy:
    call: googleapis.storage.v1.objects.copy
    args:
      sourceBucket: "bucket-name"
      sourceObject: ${text.url_encode("xxx/extra_cost_commute.txt")}
      destinationBucket: "bucket-name"
      destinationObject: ${text.url_encode("yyy/new_extra_cost_commute.txt")}
    result: copyResult

Also, take a look at this GCC post.