Search code examples
apache-kafkaprimary-keyksqldb

Kafka/KsqlDb : Why is PRIMARY KEY appending chars?


I intend to create a TABLE called WEB_TICKETS where the PRIMARY KEY is equal to the key->ID value. For some reason, when I run the CREATE TABLE instruction the PRIMARY KEY value is appended with the chars 'JO' - why is this happening?

KsqlDb Statements

These work as expected

CREATE STREAM STREAM_WEB_TICKETS (
  ID_TICKET STRUCT<ID STRING> KEY
)
WITH (KAFKA_TOPIC='web.mongodb.tickets', FORMAT='AVRO');

CREATE STREAM WEB_TICKETS_REKEYED
WITH (KAFKA_TOPIC='web_tickets_by_id') AS
SELECT *
FROM STREAM_WEB_TICKETS
PARTITION BY ID_TICKET->ID;

PRINT 'web_tickets_by_id' FROM BEGINNING LIMIT 1;

key: 5d0c2416b326fe00515408b8

The following successfully creates the table but the PRIMARY KEY value isn't what I expect:

CREATE TABLE web_tickets (
    id_pk STRING PRIMARY KEY
)
WITH (KAFKA_TOPIC = 'web_tickets_by_id', VALUE_FORMAT = 'AVRO');

select id_pk from web_tickets EMIT CHANGES LIMIT 1;

|ID_PK|                                                                                                                                                                                                                                                                         
|J05d0c2416b326fe00515408b8

As you can see the ID_PK value has the characters JO appended to it. Why is this?


Solution

  • It appears as though I wasn't properly setting the KEY FORMAT. The following command produces the expected result.

    CREATE TABLE web_tickets_test_2 (
        id_pk VARCHAR PRIMARY KEY
    )
    WITH (KAFKA_TOPIC = 'web_tickets_by_id', FORMAT = 'AVRO');