Search code examples
node.jsgoogle-cloud-storagegoogle-cloud-cdn

Google Cloud CDN cannot access file with white space from GCS backend service

I had been using Google Cloud CDN for caching content from a GCS bucket for months. Until yesterday (2019/09/19), I noticed that I cannot access an object with white spaces in its name. Usually, I apply encodeURIComponent to the object's name before signing the whole URL, which had been working fine until yesterday.

Here's what I've tried so far with gcloud utility:

  1. Sign the URL without URL-encoding the file's name:

    $ gcloud compute sign-url --key-name my-key --key-file my-key --expires-in 15m "https://cdn.example.com/file-with-white space.txt"

    I then accessed the URL with and without %20. The result is 403, shown in the picture.

  2. Sign the URL with the URL-encoded file's name (this is what I'd been doing for months, and it worked fine):

    $ gcloud compute sign-url --key-name my-key --key-file my-key --expires-in 15m "https://cdn.example.com/file-with-white%20space.txt"

    The result is also 403 but with different message:

Anonymous caller does not have storage.objects.get access to bucket/file name

I have also tried using the Go code from this link. The results are the same.

Please note that files without white spaces in its name can still be accessed successfully through the CDN.

Update

  1. To clarify, I think the CDN's behavior has changed.
  2. I have granted the CDN access to the GCS bucket. That's why the CDN worked without problems earlier. I actually have just run gsutil iam ch serviceAccount:service-PROJECT_NUM@cloud-cdn-fill.iam.gserviceaccount.com:objectViewer gs://[BUCKET] twice to ensure this.
  3. I have tried signing GCS URLs using gsutil directly without using the CDN, and the signed URL worked.

Update 2

I have tried out the --validate option. This is what I got:

$ gcloud compute sign-url --key-name cdn-signing-key \
  --key-file cdn-signing-key --expires-in 15m \
  --validate "https://cdn.domain.com/file%20with%20space"

signedUrl: https://cdn.domain.com/file%20with%20space?Expires=1569075302&KeyName=cdn-signing-key&Signature=e3SANudKHIT5txHWVlO1oijItXw=
validationResponseCode: 200

And yet, I still received 403 when accessing the "signedUrl" through a browser. The result is an XML page with <Code>AccessDenied</Code>.

Solution

  • This morning I tried accessing a newly signed URL of a file with spaces again. I noticed something interesting.

    • Sometimes, the CDN works and returns 200 along with the content of the file.
    • Some other times, the CDN gets 301 as a response from the backend service (in this case, a GCS bucket), and then redirects me to the bucket URL which in turn returns 403 along with the XML page.
    • So I tried changing the permission of the bucket to Public. If the CDN works, I will receive 200 as before. If the CDN redirects me to the bucket, now I will receive 200 along with the file.

    So I'm closing this question since I think it has nothing to do with the file name anymore. It seems like the CDN doesn't cache the content for me and tries to direct me the original file in the bucket instead. I'm not sure if this is a normal behavior of CDNs, but I'll do some more research on that.