The user doesn't have the required permission ml.versions.predict on the resource (Cloud ML Engine)

Question

I have a service account that I have given Viewer role, and have downloaded the credentials json file and set the correct environment variable for it. I am trying to run the example here:

def predict_json(project, model, instances, version=None):
    """Send json data to a deployed model for prediction.

    Args:
        project (str): project where the Cloud ML Engine Model is deployed.
        model (str): model name.
        instances ([Mapping[str: Any]]): Keys should be the names of Tensors
            your deployed model expects as inputs. Values should be datatypes
            convertible to Tensors, or (potentially nested) lists of datatypes
            convertible to tensors.
        version: str, version of the model to target.
    Returns:
        Mapping[str: any]: dictionary of prediction results defined by the
            model.
    """
    # Create the ML Engine service object.
    # To authenticate set the environment variable
    # GOOGLE_APPLICATION_CREDENTIALS=<path_to_service_account_file>
    service = googleapiclient.discovery.build('ml', 'v1beta1')
    name = 'projects/{}/models/{}'.format(project, model)

    if version is not None:
        name += '/versions/{}'.format(version)

    response = service.projects().predict(
        name=name,
        body={'instances': instances}
    ).execute()

    if 'error' in response:
        raise RuntimeError(response['error'])

    return response['predictions']

However, this gives me a 403 and the error The user doesn't have the required permission ml.versions.predict on the resource projects/project/models/model/versions/version. I am not sure what I am doing incorrectly - I am setting the correct environment variables for the credentials and according to their documentation, the service account only needs Viewer role to access this endpoint. What am I doing incorrectly?

Answer 1

tl;dr the discovery.build may not be using the expected service account as it tries many authentication options

I'd suggest to be explicit instead of relying on the default behavior as in: Using CloudML prediction API in production without gcloud. Also, its possible your project IAM settings don't include the service account if you call:

gcloud --project "$PROJECT" get-iam-policy 

Do you see the expected service account with roles/viewer or above? If not you need to grant it permission. Its presence in the service accounts page only means you have that service account, not that it is allowed to do anything!