Search code examples
apache-kafkaksqldb

KSQL pull query not working properly if Primary KEY is integer

I have create a table from another aggregate table, but the pull query seemed not functioning properly, it only working properly for emit changes, or in clause (at least 2 arguments) I am using Kubernetes Confluent Helm version 7.5.1


ksql> select custid from AAAAA where custid=15555;
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|CUSTID                                                                                                                                                                                                    |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Query terminated
ksql> select custid from AAAAA where custid in (15555, 0);
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|CUSTID                                                                                                                                                                                                    |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|15555                                                                                                                                                                                                     |
Query terminated
ksql> select custid from AAAAA where custid=15555 emit changes limit 1;
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|CUSTID                                                                                                                                                                                                    |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|15555                                                                                                                                                                                                     |
Limit Reached
Query terminated

And always generating this warning when running aggregation query, just a while only, i assume it's waiting for topic creation?

[2023-10-30 10:44:04,596] WARN [Consumer clientId=_confluent-ksql-kafka-sgquery_CTAS_AAAAA_1881-9fb9d9e1-4d97-4747-9146-bd9b74f5ad2f-StreamThread-1-consumer, groupId=_confluent-ksql-kafka-sgquery_CTAS_AAAAA_1881] Error while fetching metadata with correlation id 68 : {_confluent-ksql-kafka-sgquery_CTAS_AAAAA_1881-KTABLE-FK-JOIN-SUBSCRIPTION-REGISTRATION-0000000012-topic=UNKNOWN_TOPIC_OR_PARTITION, _confluent-ksql-kafka-sgquery_CTAS_AAAAA_1881-Aggregate-GroupBy-repartition=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient)
Solution

  • Ok, End up is PROTOBUF format issue. I have changed all the format to AVRO and working fine with integer now