Search code examples
pythonideprotocol-buffers

How to get IDEs (VS Code) to work with syntax highlighting with Protobufs (Python)


I am using VS Code, but I suspect the following problem exists across all IDEs.


When compiling a protobuf file (.proto) to Python (_pb2.py), the Messages will be available as classes, but Google's mechanism to expose the classes is difficult for IDEs to discover.

For instance, with the following toy example

person.proto

syntax = "proto3";
message Person {
  string name = 1;
  string address = 2;
  int32 age = 3;
}

if I run

> protoc -I=. --python_out=. person.proto

it will generate a person_pb2.py file that will have a Person class that I can create an object from, looking something like this:

from person_pb2 import Person

me = Person(name="me", address="heaven", age=2)

But the line from person_pb2 import Person will be highlighted as an error.

Specifically, in VS Code with Pylance installed, it will say

"Person" is unknown import symbol [Pylance (reportAttributeAccessIssue)]

(import) Person: Unknown

The reason for this is clear: if I were to look at the person_pb2.py file there is no top level object Person exported! This will be true for any protobuf messages, at any depth. They all get mangled into complicated objects and then populated to the runtime via

_builder.BuildMessageAndEnumDescriptors(DESCRIPTOR, globals())
_builder.BuildTopDescriptorsAndMessages(DESCRIPTOR, 'person_pb2', globals())

This is a very strange edge case for populating the environment that I wouldn't really expect the IDEs to discover.

So, what can I do - or what do most people do - to have the IDE properly identify protobufs?

For now, I am just littering all my code with # type: ignore annotation, which is not ideal, since I'm "solving" the problem by just turning off the tool. Is there a better option?


[To be clear, I currently "solve" this by doing things like from person_pb2 import Person # type: ignore in order to turn off Pylance's complaints. But I actually like Pylance's advice... the problem is that Google is hiding its exposed objects through this weird globals() trick.]


Solution

  • IIUC you can enable PYI using by adding --pyi_out=. to your protoc command. VS Code should then be able to annotate the stubs:

    protoc \
    --proto_path=. \
    --python_out=. \
    --pyi_out=. \
    person.proto
    

    NOTE This currently doesn't extend to gRPC (Type hints in gRPC Python stubs) but should be fine for protobuf Messages etc.