the segments are rolling over so quickly in our Kafka cluster, and it might be because we are producing records into the kafka cluster with a micro second timestamp, and Kafka expects it to be in milliseconds. Is it possible to change it to milliseconds in debezium postgres connector? Which config should I use to make data coming in milliseconds? Our kafka cluster is being unstable due to this issue.
config
plugin.name: wal2json_streaming
max.queue.size: 65536
max.batch.size: 16384
slot.max.retries: 20
slot.name: {{ name }}
database.hostname: {{ hostname }}
database.port: 5432
database.user: {{ user }}
database.password: {{ password }}
database.dbname: {{ name }}
schema.whitelist: public
table.whitelist: public.outbox
database.server.name: {{ server.name }}
database.server.id: {{ server.id }}
database.history.kafka.bootstrap.servers: {{ bootstrapServers }}
This issue is fixed in debezium 1.1.0 version