Search code examples
apache-kafkanuclio

How will a nuclio based kafka triggered service behave when it receives a serialized message


I am writing a nuclio based service that is kafka triggered. As per my understanding, whenever a message is produced to topic(s) subscribed by the service, the handler function will receive the message, and they can be accessed using the event object.

If the messages are avro/protobuf serialized, will the messages received by handler function be deserialized already or will I have to write custom code to deserialize these messages?

I am fairly new to nuclio based services so pardon my lack of knowledge, and please feel to correct my understanding if wrong.


Solution

  • Kafka data is always serialized. Therefore must be deserialized, and any Kafka consumer must set deserializer settings. Any framework you're using should document how to configure the consumer.

    If you're restricted to byte/string deserialization, then yes, you'd configure for bytes or otherwise get a base64 string, for example, then need to use your own decode / deserialize function (using builtin Kafka deserializers, or external ones, such as from Confluent) in a handler

    https://github.com/nuclio/nuclio/issues/916