How to compile nanopb .proto file into .h and .c files using nanopb and protobuf (`protoc` compile question)

Question

Old title: How to compile nanopb/examples/simple/simple.proto file into simple.h and simple.c using nanopb and protobuf

Regarding this library: https://github.com/nanopb/nanopb

My goal is to follow this tutorial: https://jpa.kapsi.fi/nanopb/docs/concepts.html to convert nanopb/examples/simple/simple.proto into a .h and .c source file. I need simple instructions to do this on Ubuntu. I've been attempting it for days and am unable to get it to work.

The commands the tutorial says to do are:

protoc -omessage.pb message.proto
python ../generator/nanopb_generator.py message.pb

I cloned the nanopb repo, cd'ed into nanopb/examples/simple, and then substituting in simple.proto instead of message.proto in the commands above, I ran the following:

protoc -osimple.pb simple.proto

It worked fine, producing a simple.pb file.

The 2nd part, however, fails. When running from inside the nanopb/examples/simple folder, I get:

$ python ../../generator/nanopb_generator.py simple.pb

         ********************************************************************
         *** Failed to import the protocol definitions for generator.     ***
         *** You have to run 'make' in the nanopb/generator/proto folder. ***
         ********************************************************************

Traceback (most recent call last):
  File "../../generator/nanopb_generator.py", line 39, in <module>
    import proto.nanopb_pb2 as nanopb_pb2
  File "/home/gabriels/GS/dev/Protocol_Buffers/Nanopb/source/nanopb/generator/proto/nanopb_pb2.py", line 11, in <module>
    from google.protobuf import symbol_database as _symbol_database
ImportError: cannot import name symbol_database

Running make does nothing (says it's already done):

nanopb/generator/proto $ make
make: Nothing to be done for `all'.

Note that I am running the latest version of protoc, built from the Google protobuf repo from source: https://github.com/protocolbuffers/protobuf.

I have also sought help from nanopb here, but am unable to figure it out, and feel like there's something basic here I'm missing because I just don't know enough: https://github.com/nanopb/nanopb/issues/417. Feels like I'm beating my head into the wall on something that should be simple and has already been done by at least 1448+ people before me (the number of stars on nanopb).

Answer 1

Solved. @PetteriAimonen had given me the missing clue:

the protoc version needs to match with the python library version

Then it occurred to me: originally, when compiling protobuf from scratch, I followed only the C++ installation instructions, as shown here: https://github.com/protocolbuffers/protobuf/tree/master/src. But, what if I follow the Python installation instructions too? https://github.com/protocolbuffers/protobuf/tree/master/python

So, that's what I did.

TLDR; Do the Python installation of the protobuf library too (not just the C++ installation):

Protobuf Python installation steps I followed:

python -V # See if I have Python 2.7 or newer (I must to continue)
cd protobuf/python # cd into Python source directory
python setup.py build
python setup.py test
(cd .. && make)
(cd .. && sudo make install)
python setup.py build --cpp_implementation
python setup.py test --cpp_implementation  # look to see all tests pass
sudo python setup.py install

Two-step compile of .proto files:

That all worked, so now lets go back and try compiling our simple.proto file again.

cd into nanopb/examples/simple. We already ran the first command to produce the simple.pb file, so now just run the 2nd command that previously would fail, and it works!

2nd Command only:

nanopb/examples/simple $ python ../../generator/nanopb_generator.py simple.pb

Output:

nanopb/examples/simple $ python ../../generator/nanopb_generator.py simple.pb  
Writing to simple.pb.h and simple.pb.c

2-commands shown together again for completeness:

protoc -osimple.pb simple.proto
nanopb/examples/simple $ python ../../generator/nanopb_generator.py simple.pb

Beautiful! IT WORKED! simple.pb.h and simple.pb.c are now created!

Now build the "simple" project:

make

And run it:

./simple

And the output is:

nanopb/examples/simple $ ./simple  
Your lucky number was 13!

Now I can study the project to see how simple.proto was turned into simple.pb.h and simple.pb.c, and I can study simple.c (which contains the main() function) to see a full usage of these auto-generated .h and .c files, including looking at the following header files which it includes:

#include <pb_encode.h> # found up 2 levels, in "nanopb" folder
#include <pb_decode.h> # found up 2 levels, in "nanopb" folder
#include "simple.pb.h" # just generated right here in "nanopb/examples/simple" folder

One-line command to build .proto files:

Instead of doing the two-line command to build .proto files:

# From inside folder "/home/gabriels/GS/dev/Protocol_Buffers/Nanopb/source/nanopb/examples/simple":
protoc -osimple.pb simple.proto
python ../../generator/nanopb_generator.py simple.pb

we can do a one-line command to build .proto files which just uses the protoc executable plus the protoc-gen-nanopb plugin:

protoc --plugin=protoc-gen-nanopb=/home/gabriels/GS/dev/Protocol_Buffers/Nanopb/source/nanopb/generator/protoc-gen-nanopb --nanopb_out=. simple.proto

And then, of course, we still need to make and run the main C project:

# From inside folder "/home/gabriels/GS/dev/Protocol_Buffers/Nanopb/source/nanopb/examples/simple":
make && ./simple