"Deadline" error when embedding video with Google Vertex AI multimodal embedding modal

Question

I am currently using Vertex AI's Multimodal Embedding Model (https://cloud.google.com/vertex-ai/generative-ai/docs/embeddings/get-multimodal-embeddings)

I was able to get the Image and Text examples running just fine using the Python SDK and POST, however, when I try to run the following Video example (from the above link), I get the following error

    import vertexai
    
    from vertexai.vision_models import MultiModalEmbeddingModel, Video
    
    project_id = 'project_name'
    location = 'us-central1'
    
    vertexai.init(project=project_id, location=location)
    
    # Document metadata
    video_path = 'gs://test-public-bucket-123/dog_jumping_short.mp4' # public small 1 MB 7 second video file
    description = 'dogs jumping'
    
    model = MultiModalEmbeddingModel.from_pretrained("multimodalembedding@001")
    
    video = Video.load_from_file(video_path)
    
    embeddings = model.get_embeddings(
        video=video
    )
    
    # Video Embeddings are segmented based on the video_segment_config.
    print("Video Embeddings:")
    for video_embedding in embeddings.video_embeddings:
        print(
            f"Video Segment: {video_embedding.start_offset_sec} - {video_embedding.end_offset_sec}"
        )
        print(f"Embedding: {video_embedding.embedding}")
    
    print(f"Text Embedding: {embeddings.text_embedding}")

The error I get when running this sample code is related to Deadline

Traceback (most recent call last):
  File "/Users/me/Code/multi-modal-test/multi-modal/lib/python3.12/site-packages/google/api_core/grpc_helpers.py", line 76, in error_remapped_callable
    return callable_(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/Code/multi-modal-test/multi-modal/lib/python3.12/site-packages/grpc/_channel.py", line 1181, in __call__
    return _end_unary_response_blocking(state, call, False, None)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/Code/multi-modal-test/multi-modal/lib/python3.12/site-packages/grpc/_channel.py", line 1006, in _end_unary_response_blocking
    raise _InactiveRpcError(state)  # pytype: disable=not-instantiable
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.UNAUTHENTICATED
        details = "Video embedding failed with the following error: Deadline"
        debug_error_string = "UNKNOWN:Error received from peer ipv4:142.250.81.234:443 {grpc_message:"Video embedding failed with the following error: Deadline", grpc_status:16, created_time:"2024-05-23T15:53:15.429704-04:00"}"

Note that I have been able to embed contextual_text and image just fine with the same authentication, so I am fairly certain this has nothing to do with authentication even though the error says so.

I have also tried to POST using the following cURL, but I get the same error response Deadline

    curl -X POST \ 
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://us-central1-aiplatform.googleapis.com/v1/projects/multi/locations/us-central1/publishers/google/models/multimodalembedding@001:predict"

I have the following APIs enabled (note that according to docs, I only need to have Vertex AI API enabled)

These are my IAM permissions

- members:
  - user:[email protected]
  role: roles/aiplatform.admin
- members:
  - user:[email protected]
  role: roles/aiplatform.user
- members:
  - user:[email protected]
  role: roles/ml.admin
- members:
  - user:[email protected]
  role: roles/owner
- members:
  - user:[email protected]
  role: roles/storage.admin
- members:
  - user:[email protected]
  role: roles/visionai.admin
etag: BwYaJsGtzOk=
version: 1

I am not using a Service Account, but a User Account.

Anyone know what I can do here?

Answer 1

This might come from a discrepancy between the "current project" of gcloud, and the "Application Defaults Credentials". This can be confusing because the exact order of these commands matter.

To make sure, could you try in this order:

1.

gcloud config set project MY_PROJECT_ID

(doc)

2.

gcloud auth application-default login

(doc)

3. Run your python program (or your cuRL command)