Search code examples
goapache-beamdataflow

Group by key collection type in Apache Beam pipeline

I have a pipeline in Apache Beam Go SDK.

pcoll := beam.GroupByKey(s, src)

The thing is that after GroupByKey transformation I want to process it further with ParDo transformation. And I have the problem with type since Go want me to define ParDo function input as follows:

value CoGBK<[]uint8,[]uint8>

but there is no type CoGBK in Go. Is there a way do define such type in Apache Beam Go SDK?

Solution

  • Ok, when we have a message that we need value CoGBK<[]uint8,[]uint8> type, what we actually should do is apply the following transform: beam.ParDo0(s, func(key []uint8, values func(*[]uint8) bool) {}, pcoll) And that's interpreted by Go as CoGBK<[]uint8,[]uint8>.