Kafka CooperativeStickyAssignor assigns all partitions to 1 consumer

setup:

kafka 2.8 (server) and java org.apache.kafka kafka-clients version: 3.0.0
4 pods on kubernetes running kafka consumers
consumer properties: max poll interval of 5 minutes, session of 45 seconds
all consumers "subscribe" to the same topic, using a consumer group.id
Setting partition.assignment.strategy to CooperativeStickyAssignor
1024 partitions

Issues

I'm monitoring which partitions get assigned to which consumer/pod and I see some big issues when I play with the number of pods (deleting/recreating 1 pod or all pods at the same time)

For 5 minutes in a row, it happens sometimes that not all partitions get assigned, and then they are all assigned and well balanced. I don't understand why the rebalance doesn't happen earlier since the session length is 45 seconds
sometimes ALL partitions get assigned to 1 consumer. Looking at logs it says that partitions get assigned to consumer 1 and consumer 2 (and nothing "revoked"), but really when testing all partitions are consumed by only 1 consumer

Any idea what can be wrong about the configuration? I'm trying to keep it as close possible to the default. Any help greatly appreciated

Solution

I believe this is related to this issue: https://issues.apache.org/jira/browse/KAFKA-14639