Search code examples
bashprotoc

Compile .proto files on every directory


I have a directory structure like this:

Makefile
README.md
go.mod
google/
  -> Protobuf dependencies (ignore)
gopb/
  -> Compiled Go protobuf files (ignore)
protoc-gen-openapiv2/
  -> Protobuf OpenAPI depdencies (ignore)
providers/
  - polygon/
    - tickers.proto
    - query.proto

I'd like to setup a single command that can find all directories with .proto files (except the ones indicated above) and run protoc on each of these, with output going to a directory of the same name under the gopb directory. So far, I have been able to create a find command that returns the directory names:

$ find -not \( -path "*.git*" -or -path "*google*" -or -path "*protoc*" -or -path . -or -path "*gopb*" \) -type d -print

./providers
./providers/polygon

Obviously, this is not entirely accurate and I'd like to fix that, but the main issue is that, when I use the -exec flag to call protoc:

$ find -not \( -path "*.git*" -or -path "*google*" -or -path "*protoc*" -or -path . -or -path "*gopb*" \) -type d \
> -exec protoc --go_out=./gopb/ --go_opt=paths=source_relative {}/*.proto \;

This prints two errors:

Could not make proto path relative: ./providers/*.proto: No such file or directory
Could not make proto path relative: ./providers/polygon/*.proto: No such file or directory

I'm not sure what I'm doing wrong here because, if I call protoc --go_out=./gopb/ --go_opt=paths=source_relative ./providers/polygon/*.proto, it creates the files as expected. What am I doing wrong here?


Solution

  • With help from @Fravadona, I realized that the method I was using to compile my .proto files wasn't going to work. So, I moved all my protobuf files that I wanted to compile into a new directory called protos, which modified my directory structure to:

    Makefile
    README.md
    go.mod
    google/
      -> Protobuf dependencies (ignore)
    gopb/
      -> Compiled Go protobuf files (ignore)
    protoc-gen-openapiv2/
      -> Protobuf OpenAPI depdencies (ignore)
    protos/
      - providers/
        - polygon/
          - tickers.proto
          - query.proto
    

    and then I modified the command to be this:

    find ./protos -type f -name "*.proto" -exec protoc --go_out=../../../ {} \;
    

    This new command finds all .proto files in the /protos directory and calls protoc on each. I removed the source-relative option which also required some code changes in my imports as well. But, this works for my needs. Note I had to use ../../../ for my output directory because my protobuf repo exists inside of github.com/myrepo/protobuf and I want the files written to github.com/myrepo/protobuf/gopb.