Search code examples
scalaprotocol-buffersscalapb

Unknown fields in proto generated by scalapb

I'm new to scalapb and protobuf.

I'm trying to create unit test for my scalapb's generators. I've generated proto files and trying to use them in tests.

I've got this proto file:

syntax = "proto3";

package hellogrpc.calc;

import "google/api/annotations.proto";

option (scalapb.options) = {
  flat_package: true
};

service CalcService {

    rpc CalcSum (SumRequest) returns (CalcResponse) {
        option (google.api.http) = {
          post: "/calcService/sum"
          body: "*"
        };
    }

}

There is method CalcSum which is annotated

And corresponding generated proto file:

// Generated by the Scala Plugin for the Protocol Buffer Compiler.
// Do not edit!
//
// Protofile syntax: PROTO3

package hellogrpc.calc



object CalcServiceProto extends _root_.com.trueaccord.scalapb.GeneratedFileObject {
  lazy val dependencies: Seq[_root_.com.trueaccord.scalapb.GeneratedFileObject] = Seq(
    com.trueaccord.scalapb.scalapb.ScalapbProto,
    com.google.api.annotations.AnnotationsProto
  )
  lazy val messagesCompanions: Seq[_root_.com.trueaccord.scalapb.GeneratedMessageCompanion[_]] = Seq(
    hellogrpc.calc.SumRequest,
    hellogrpc.calc.CalcResponse
  )
  private lazy val ProtoBytes: Array[Byte] =
      com.trueaccord.scalapb.Encoding.fromBase64(scala.collection.Seq(
  """ChtoZWxsb2dycGMvQ2FsY1NlcnZpY2UucHJvdG8SDmhlbGxvZ3JwYy5jYWxjGhVzY2FsYXBiL3NjYWxhcGIucHJvdG8aHGdvb
  2dsZS9hcGkvYW5ub3RhdGlvbnMucHJvdG8iKAoKU3VtUmVxdWVzdBIMCgFhGAEgASgFUgFhEgwKAWIYAiABKAVSAWIiJgoMQ2FsY
  1Jlc3BvbnNlEhYKBnJlc3VsdBgBIAEoBVIGcmVzdWx0Mm8KC0NhbGNTZXJ2aWNlEmAKB0NhbGNTdW0SGi5oZWxsb2dycGMuY2FsY
  y5TdW1SZXF1ZXN0GhwuaGVsbG9ncnBjLmNhbGMuQ2FsY1Jlc3BvbnNlIhuC0+STAhUiEC9jYWxjU2VydmljZS9zdW06ASpCBeI/A
  hABYgZwcm90bzM="""
      ).mkString)
  lazy val scalaDescriptor: _root_.scalapb.descriptors.FileDescriptor = {
    val scalaProto = com.google.protobuf.descriptor.FileDescriptorProto.parseFrom(ProtoBytes)
    _root_.scalapb.descriptors.FileDescriptor.buildFrom(scalaProto, dependencies.map(_.scalaDescriptor))
  }
  lazy val javaDescriptor: com.google.protobuf.Descriptors.FileDescriptor = {
    val javaProto = com.google.protobuf.DescriptorProtos.FileDescriptorProto.parseFrom(ProtoBytes)
    com.google.protobuf.Descriptors.FileDescriptor.buildFrom(javaProto, Array(
      com.trueaccord.scalapb.scalapb.ScalapbProto.javaDescriptor,
      com.google.api.annotations.AnnotationsProto.javaDescriptor
    ))
  }
  @deprecated("Use javaDescriptor instead. In a future version this will refer to scalaDescriptor.", "ScalaPB 0.5.47")
  def de
```scriptor: com.google.protobuf.Descriptors.FileDescriptor = javaDescriptor
}

I inspect CalcServiceProto.javaDescriptor in intellj idea:

enter image description here

Method descriptor has this proto definition:

name: "CalcSum"
input_type: ".hellogrpc.calc.SumRequest"
output_type: ".hellogrpc.calc.CalcResponse"
options {
  72295728: "\"\020/calcService/sum:\001*"
}

But generator works just fine! I debug generator, toggled breakpoint on generator and method CalcSum has this proto definition:

name: "CalcSum"
input_type: ".hellogrpc.calc.SumRequest"
output_type: ".hellogrpc.calc.CalcResponse"
options {
  [google.api.http] {
    post: "/calcService/sum"
    body: "*"
  }
}

May be this works differently because i didn't register extensions like generator do.

Any way i want this test to be passed:

  val s = CalcServiceProto.javaDescriptor.getServices.get(0)

  val m = s.getMethods.get(0)

  m.getOptions.getExtension(AnnotationsProto.http).getPost shouldBe "/calcService/sum"
Solution

  • If you need Java extensions to be available you need to generate your code with Java conversions enabled. This will make javaDescriptor rely on the official Java implementation and your test will pass.

    When Java conversions are disabled, ScalaPB parses the the descriptor, but it can't guarantee the Java extensions are even compiled so it doesn't attempt to register them.

    What I'd like to have is that the Scala descriptors would work in this case, however they are not supporting services and methods yet. I filed https://github.com/scalapb/ScalaPB/issues/382 to track progress on this.

    In the mean time, like I wrote above, you can use java conversions to force ScalaPB to provide you the Java descriptor. In your build.sbt, have:

    PB.targets in Compile := Seq(
  scalapb.gen(grpc=true, javaConversions=true) -> (sourceManaged in Compile).value,
  PB.gens.java -> (sourceManaged in Compile).value
)