How can I create a "duck-typed" grpc protobuf Message at runtime in Go?

Question

I'm using grpc protobuf Message definitions and implementing them in Go.

My ultimate goal is for my rpc to retrieve some json on a user and return a Profile Message with optional, nested Messages to be used for unmarshalling a subset of the json.

using this rpc:

  rpc GetUser (GetUserRequest) returns (Profile) {
    option (google.api.http) = {
      get: "/user/{id=*}"
    };
  }

and assuming the following json:

{
  "profile": {
     "foo": {
        "name": "tim"
        "age": 22
      },
     "bar": {
       "level": 5
     }
  }
}

I want to return a Profile Message containing only "foo", "bar", or both, as nested Messages based on an incoming runtime scope parameter of the grpc request (currently, scope would be a list of strings containing the Message names to be used for subsetting the json into respective Messages e.g. ["Foo","Bar"]).

given these Message definitions:

message Profile {
  //both Foo & Bar are optional by default
  Foo foo = 1 [json_name="foo"];
  Bar bar = 2 [json_name="bar"];
}

message Foo {
  string name = 1 [json_name="name"];
  int32 age = 2 [json_name="age"];
}

message Bar {
  string level = 1 [json_name="level"];
}

then in the case that scope is ["Foo"], I'd like the rpc to return:

Profile{
  Foo: // Foo Message unmarshalled from json
}

or if "scope" is ["Foo","Bar"] then:

Profile{
  Foo: 
  Bar:
}

The problem seems to boil down to "duck-typing" a Message type.

Attempt 1

I got close to finding a solution using protoreflect & protoregistry by doing:

import(
    "google.golang.org/protobuf/reflect/protoregistry"
    "google.golang.org/protobuf/reflect/protoreflect"
)

var scope protoreflect.FullName = "Foo"
var types = new(protoregistry.Types)
var message, errs = types.FindMessageByName(scope)
var almost_foo = message.New()

// using `myJson` object without top level "profile" key to make testing more simple at the moment
var myJson = `{ "foo": { .. }, "bar", { ... } }`
err = json.Unmarshal([]byte(myJson), almost_foo)

but when I attempt to create Profile Message using almost_foo:

var profile = &pb.Profile{almost_foo}

I get error: cannot use almost_foo (type protoreflect.Message) as type *package_name.Foo in field value

Attempt 2

using

import(
  "github.com/jhump/protoreflect/desc"
  "github.com/jhump/protoreflect/dynamic"
)

I try to dynamically create the message again:

var fd, errs = desc.LoadFileDescriptor("github.com/package/path/..")
var message_desc = fd.FindMessage("Foo")
var almost_foo = dynamic.NewMessage(message_desc)

and a similar error occurs: cannot use almost_foo (type *dynamic.Message) as type *package_name.Foo in field value

Both attempts almost create a Message but the type system still doesn't allow either to actually be used.

Any help is appreciated.

Answer 1

I managed to solve this by whittling down the json to what I want first and then putting it through Unmarshal.

I created a top level Message that takes Profile as a field:

message GetUserResponse {
  Profile profile = 1;
}

and redefined the response:

rpc GetUser (GetUserRequest) returns (GetUserResponse)

To build the allowed/scoped json, I use:

"github.com/tidwall/gjson"
"github.com/tidwall/sjson"

I copy the json dictated by the scope paths (dot-delimited e.g. ["profile.foo"]) over to an initially empty, output, response_json string, and build up only the allowed json to be returned.

var response_json string

var scopes = []string{"profile.foo"}

for _, scope := range scopes {
    // copy allowed json subset in raw form
    value := gjson.Get(myJson, scope + "|@ugly")

    // if scope didn't copy a value, skip
    if value.String() == "" {
        continue
    } else {
        // write value to output json using SetRaw since value can be an object
        response_json, _ = sjson.SetRaw(response_json, scope, value.Raw)
    }
}

response_json looks like:

{
  "profile": {
     "foo": {
        "name": "tim"
        "age": 22
      }
  }
}

and then Unmarshal:

response = &pb.VerifyUserResponse{}
err = json.Unmarshal([]byte(response_json), response)