Search code examples
gogrpc

Import "protoc-gen-openapiv2/options/annotations.proto" was not found or had errors


I have a proto file where I want to add pagination. The technology stack is GoLang with gRPC.

import "google/api/annotations.proto";
import "protoc-gen-openapiv2/options/annotations.proto";
import "google/protobuf/timestamp.proto";

When try to add the dependency via go install or go get. I get

go install protoc-gen-openapiv2/options/annotations.proto@latest                                                       
go: protoc-gen-openapiv2/options/annotations.proto@latest: malformed module path "protoc-gen-openapiv2/options/annotations.proto": missing dot in first path element

go get protoc-gen-openapiv2/options/annotations.proto                                                                  
go: go.mod file not found in current directory or any parent directory.
        'go get' is no longer supported outside a module.

When I read the documentation it appears I am doing this correctly. Has anyone else had this issue?

message BlogAllRequest {
    option (grpc.gateway.protoc_gen_openapiv2.options.openapiv2_schema) = {
        json_schema: {
            required: ["filter"]
        }
    };

    string id = 1;
    int32 page = 2;
    int32 per_page = 3 [json_name="per_page"];
    SortOrder order = 4 [json_name = "order"];

}

Solution

  • NOTE Apologies I'm less familiar with gRPC Gateway but I'm very familiar with Protocol Buffers (and gRPC). So, I can't help with the full solution but I can show you how to compile your proto.

    With Protocol Buffers you need source files locally (e.g. .proto files). You cannot go get nor go install Protocol Buffer files.

    The grpc-gateway protos that you need are:

    For the next step, it's probably easiest to git clone the grpc-gateway repo. The protos mentioned above are in the repo's ./protoc-gen-openapiv2 folder.

    I've cloned it into my working directory (see tree below).

    The first step is to then protoc compile your proto referencing these protos. Here's my completion of your proto file:

    foo.proto:

    syntax = "proto3";
    
    option go_package = "{MODULE}/protos";
    
    import "protoc-gen-openapiv2/options/annotations.proto";
    
    message BlogAllRequest {
        option (grpc.gateway.protoc_gen_openapiv2.options.openapiv2_schema) = {
            json_schema: {
                required: ["filter"]
            }
        };
    
        enum SortOrder{
            BAR = 0;
            BAZ = 1;
        }
    
        string id = 1;
        int32 page = 2;
        int32 per_page = 3 [json_name="per_page"];
        SortOrder order = 4 [json_name = "order"];
    }
    

    NOTE Replace {MODULE} with your repo name. Generally your repo name will match your Go module name. As I've done, you may want to add e.g. /protos to your {MODULE} value for the value of go_package as this provides the protoc-generated sources with their own package name. I also add an arbitrary SortOrder type.

    You can now protoc compile your proto:

    # Your go.mod module value and probably matching your Git repo 
    MODULE="..."
    
    # Doing this here to prove the point
    go mod init ${MODULE}
    
    # Protoc will output foo.pb.go in protos folder
    # This will make it the ${MODULE}/protos package
    mkdir protos
    
    protoc \
    --proto_path=${PWD} \
    --proto_path=${PWD}/grpc-gateway \
    --go_out=./protos \
    --go_opt=module=${MODULE} \
    ${PWD}/foo.proto
    

    protoc is very particularly about paths. I always root it on my working directory (${PWD}) and, in this case, I've cloned gprc-gateway into the working directory. Both folders contain protos and so both must be proto_path'd. Because I used ${PWD} for the proto_path, I must prefix proto filenames with ${PWD} too (i.e. ${PWD}/foo.proto). It's protoc's way.

    Importantly, I want your proto compiled into a package ending protos and so I need to reflect this in the working folder too

    To ensure protoc knows that my working directory (with go.mod) is my Go module root, you can use --go_opt=module=${MODULE} to place the files correctly.

    I've called your partial proto foo.proto.

    The grpc-gateway repo not only includes the protos (./protoc-gen-openapiv2/options/*.proto) but it also includes the protoc-generated Golang sources (./protoc-gen-openapiv2/options/*.pb.go). This is a (helpful) choice (I think, good practice) made by the repo maintainers and it means that you don't need to generate these sources for yourself.

    In fact, your protoc-generated foo.pb.go should include the correct references to these sources and you will just need to go mod tidy to go get them. Here's the beginning of the foo.pb.go that I generated:

    foo.pb.go:

    // Code generated by protoc-gen-go. DO NOT EDIT.
    // versions:
    //  protoc-gen-go v1.28.1
    //  protoc        v3.21.12
    // source: foo.proto
    
    package protos
    
    import (
        _ "github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-openapiv2/options"
        protoreflect "google.golang.org/protobuf/reflect/protoreflect"
        protoimpl "google.golang.org/protobuf/runtime/protoimpl"
        reflect "reflect"
        sync "sync"
    )
    

    Running tree:

    tree
    .
    ├── foo.proto
    ├── go.mod
    ├── grpc-gateway
    │   ├── protoc-gen-openapiv2
    │   │   └── options
    │   │       ├── annotations.pb.go
    │   │       ├── annotations.proto
    │   │       ├── annotations.swagger.json
    │   │       ├── BUILD.bazel
    │   │       ├── openapiv2.pb.go
    │   │       ├── openapiv2.proto
    │   │       └── openapiv2.swagger.json
    ├── protoc-21.12-linux-x86_64
    │   ├── bin
    │   │   └── protoc
    │   ├── include
    │   │   └── google
    │   │       └── protobuf
    │   │           ├── any.proto
    │   │           ├── api.proto
    │   │           ├── compiler
    │   │           │   └── plugin.proto
    │   │           ├── descriptor.proto
    │   │           ├── duration.proto
    │   │           ├── empty.proto
    │   │           ├── field_mask.proto
    │   │           ├── source_context.proto
    │   │           ├── struct.proto
    │   │           ├── timestamp.proto
    │   │           ├── type.proto
    │   │           └── wrappers.proto
    │   └── readme.txt
    └── protos
        └── protos
            └── foo.pb.go