Search code examples
google-cloud-platformgoogle-cloud-storagecloudvirtual-machinegoogle-compute-engine

I can't store in cloud storage , I get acces denied when I try to export data from vm to my cloud storage

I cretaed a vm in compute engine, and I opened jupyter-notebook, I can read file in my bucket but can't save other. I created the bucket and the vm myself, so I guess I am an ownner?

import pandas as pd

df = pd.read("gs://my-bucket/my_file_v1.csv")

#some code in here

df.to_csv("gs://my-bucket/my_file_2.csv") #I get this error OSError: Forbidden: https://storage.googleapis.com/upload/storage/v1/b/pacific_peche/o Access denied.

I tried to change the IAM policy, the bucket is in not public

and I dded me as principal and I give it the permission needed: Cloud storage viewer

Cloud storage admin ...

Solution

  • It's a common mistake. When you create a Compute Engine, with default parameters, the console propose to use the default compute engine service account as identity of your Compute engine enter image description here

    The legacy behavior is when you use the default compute engine service account the subsequent scope applies. If you click on "set access for each API" you will see the default scope set on your service account: Cloud Storage is in read only mode enter image description here

    Therefore, your error

    To solve that, 2 solutions.

    Firstly stop the Compute Engine and edit it.

    1. Either add/change the missing scopes to access the APIs
    2. (Prefered) don't use the default compute engine service account, and use a user managed service account with only the permission that you want on it. When you use a user managed service account, scopes does not apply. In addition, the default service account has the editor role by default which is too broad.