What if debezium source connector of kafka connect fails to produce to kafka

Question

I use debezium source connector for connecting mongoDB with SMT and AvroConverter.

When facing serialization error due to compatibility, does the connector skip the error records, or it retries sending the error record?

I set error.tolerance = 'all' (should beerrors.tolerance = 'all'). I think the connector does not commit the offset back to kafka so the connector keeps retrying sending the record, am I right?

Answer 1

If I understand correctly, you can configure behaviour of retries, e.g.:

# retry for at most 10 minutes times waiting up to 30 seconds between consecutive failures
errors.retry.timeout=600000
errors.retry.delay.max.ms=30000

If the retry limit for a failure is reached, then the tolerance limit is used to determine if this record should be skipped (errors.tolerance=all), or if the task is to be killed (errors.tolerance=none).

More details in the KIP-298: Error Handling in Connect.