Search code examples
pythonprotocol-buffers

Creating a protobuf factory for a dynamically generated message?


Based on Dynamically create a new protobuf message, I have this Python code which dynamically generates a protobuf containing two ints and a string. (Context: a dynamic data server will transfer query results via protobuf.)

I can create the descriptor properly, and I believe the code for instantiating and populating the message class is correct.

But, when trying to generate the factory

# Step 5: Create the message class                                             
factory = message_factory.MessageFactory(pool)                                 
DynamicMessageClass = factory.GetPrototype(message_descriptor) <-- error here               

I receive this error.

/Users/mark/g/private/lessons/proto_dynamic/g2.py:40: UserWarning: MessageFactory class is deprecated. Please use GetMessageClass() instead of MessageFactory.GetPrototype. MessageFactory class will be removed after 2024.
  DynamicMessageClass = factory.GetPrototype(message_descriptor)
Traceback (most recent call last):
  File "/Users/mark/g/private/lessons/proto_dynamic/g2.py", line 40, in <module>
    DynamicMessageClass = factory.GetPrototype(message_descriptor)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mark/miniconda3/lib/python3.11/site-packages/google/protobuf/message_factory.py", line 185, in GetPrototype
    return GetMessageClass(descriptor)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mark/miniconda3/lib/python3.11/site-packages/google/protobuf/message_factory.py", line 70, in GetMessageClass
    concrete_class = getattr(descriptor, '_concrete_class', None)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: No message class registered for 'dynamic_package.DynamicMessage'

What do I need to do to generate my class from my descriptor?

Full code, for reference:

from google.protobuf import descriptor_pb2, descriptor_pool, message_factory   
                                                                               
# Step 1: Create the DescriptorProto for the message                           
message_descriptor_proto = descriptor_pb2.DescriptorProto()                    
message_descriptor_proto.name = "DynamicMessage"                               
message_descriptor_proto.field.add(                                            
    name="first_int",                                                          
    number=1,                                                                  
    type=descriptor_pb2.FieldDescriptorProto.TYPE_INT32,                       
    label=descriptor_pb2.FieldDescriptorProto.LABEL_OPTIONAL,                  
)                                                                              
message_descriptor_proto.field.add(                                            
    name="second_int",                                                         
    number=2,                                                                  
    type=descriptor_pb2.FieldDescriptorProto.TYPE_INT32,                       
    label=descriptor_pb2.FieldDescriptorProto.LABEL_OPTIONAL,                  
)                                                                              
message_descriptor_proto.field.add(                                            
    name="a_string",                                                           
    number=3,                                                                  
    type=descriptor_pb2.FieldDescriptorProto.TYPE_STRING,                      
    label=descriptor_pb2.FieldDescriptorProto.LABEL_OPTIONAL,                  
)                                                                              
                                                                               
# Step 2: Create a FileDescriptorProto including the DescriptorProto           
file_descriptor_proto = descriptor_pb2.FileDescriptorProto()                   
file_descriptor_proto.name = "dynamic_message.proto"                           
file_descriptor_proto.package = "dynamic_package"                              
file_descriptor_proto.message_type.add().CopyFrom(message_descriptor_proto)    
                                                                               
# Step 3: Add the FileDescriptorProto to a DescriptorPool                      
pool = descriptor_pool.DescriptorPool()                                        
file_descriptor = pool.Add(file_descriptor_proto)                              
                                                                               
# Step 4: Retrieve the message descriptor by name from the DescriptorPool      
message_descriptor = pool.FindMessageTypeByName("dynamic_package.DynamicMessage")
                                                                               
# Step 5: Create the message class                                             
factory = message_factory.MessageFactory(pool)                                 
DynamicMessageClass = factory.GetPrototype(message_descriptor)                 
                                                                               
# Step 6: Instantiate the message class and populate with data                 
dynamic_message = DynamicMessageClass()                                        
dynamic_message.first_int = 123                                                
dynamic_message.second_int = 456                                               
dynamic_message.a_string = "Hello, World!"                                     
                                                                               
# Serialize the message to a byte string                                       
serialized_data = dynamic_message.SerializeToString()                          
                                                                               
# Show the serialized data                                                     
print(serialized_data)                                                         

Solution

  • The answer comes from the documentation of the message_factory module:

    Provides a factory class for generating dynamic messages.
    
    The easiest way to use this class is if you have access to the FileDescriptor
    protos containing the messages you want to create you can just do the following:
    
    message_classes = message_factory.GetMessages(iterable_of_file_descriptors)
    my_proto_instance = message_classes['some.proto.package.MessageName']()
    

    So, one way to make it work is:

    dynamic_message = message_factory.GetMessages([file_descriptor_proto])['dynamic_package.DynamicMessage']()
    dynamic_message.first_int = 123
    dynamic_message.second_int = 456
    dynamic_message.a_string = "Hello, World!"
    print(dynamic_message)
    first_int: 123
    second_int: 456
    a_string: "Hello, World!"
    

    Here the file_descriptor_proto is the one generated at the end of step two.