Search code examples
apache-kafkaspring-cloudspring-cloud-streamamazon-kinesis

spring cloud stream support json header?(in the body)

Our all consumer application use spring-cloud-stream.
But our one big producer application(legacy..) can't use spring-cloud-stream because spring boot version is 1.4.

We tried to send message with kinesis message broker.
We want to send custom headers through kinesis, but kinesis doesn't support sending headers as you know.
So we decided to send a header in body. 

{
  "header": {
    ~~
  }
  "payload": {
    ~~
  }
}

But spring-cloud-stream can't receive header in body.
We thought we could receive a header in body using the embeddedHeaders option(in spring-cloud-stream). 

{channelName}:
    destination: local-event
    consumer:
        header-mode: embeddedHeaders

But... it's not working.

So.. is there any way to receive header in body?
Or we have to select another way?

Solution

  • Embedded header values are individual JSON snippets, but they are encoded into the message using a binary format - see EmbeddedHeaderUtils.

    Encodes requested headers into payload with format 

    0xff, n(1), [ [lenHdr(1), hdr, lenValue(4), value] ... ].

    The 0xff indicates this new format; n is number of headers (max 255); for each header, the name length (1 byte) is followed by the name, followed by the value length (int) followed by the value (json).