Search code examples
protocol-buffersgrpcgrpc-gogrpcurl

How to upload an image in chunks with client-side streaming gRPC using grpcurl


I have been trying to upload an image in chunks with client side streaming using grpcurl. The service is working without error except that at the server, image data received is 0 bytes.
The command I am using is:

grpcurl -proto image_service.proto -v -d @ -plaintext localhost:3010 imageservice.ImageService.UploadImage < out

This link mentions that the chunk data should be base64 encode and so the contents of my out file are:

{"chunk_data": "<base64 encoded image data>"}

This is exactly what I am trying to achieve, but using grpcurl.
Please tell what is wrong in my command and what is the best way to achieve streaming via grpcurl.
I have 2 more questions:

  1. Does gRPC handles the splitting of data into chunks?
  2. How can I first send a meta-data chunk (ImageInfo type) and then the actual image data via grpcurl?

Here is my proto file:

syntax = "proto3";
package imageservice;
import "google/protobuf/wrappers.proto";
option go_package = "...";

service ImageService {

  rpc UploadImage(stream UploadImageRequest) returns (UploadImageResponse) {}

}

message UploadImageRequest {
  oneof data {
    ImageInfo info = 1;
    bytes chunk_data = 3;
  };
}

message ImageInfo {
  string unique_id = 1;
  string image_type = 2;
}

message UploadImageResponse {
  string url = 1;
}

Solution

  • Interesting question. I've not tried streaming messages with (the excellent) grpcurl.

    The documentation does not explain how to do this but this issue shows how to stream using stdin.

    I recommend you try it that way first to ensure that works for you.

    If it does, then bundling various messages into a file (out) should also work.

    Your follow-on questions suggest you're doing this incorrectly.

    • chunk_data is the result of having split the file into chunks; i.e. each of these base64-encoded strings should be a subset of your overall image file (i.e. a chunk).

    • your first message should be { "info": "...." }, subsequent messages will be { "chunk_data": "<base64-encoded chunk>" } until EOF.