Search code examples
apache-kafkaapache-kafka-connectmongodb-kafka-connector

delete.on.null.values in MongoDB Kafka Connector not working


I'm working on MongoDB Kafka Connector and i'm able to successfully Create and Update records but not able to delete records in connector config I'm using this config

{
"name": "mongo-sink",
"config": {
    "connector.class": "com.mongodb.kafka.connect.MongoSinkConnector",
    "tasks.max": "1",
    "topics": "test",
    "connection.uri": "mongodb://mongo1:27018,mongo2:27019,mongo3:27020",
    "database": "accounting",
    "collection": "test",
    "key.converter": "org.apache.kafka.connect.storage.StringConverter",
    "value.converter": "io.confluent.connect.avro.AvroConverter",
    "value.converter.schema.registry.url": "http://schema-registry:8081",
    "value.converter.schemas.enable": "false",
    "document.id.strategy": "com.mongodb.kafka.connect.sink.processor.id.strategy.PartialKeyStrategy",
    "document.id.strategy.partial.key.projection.list":"productId",
    "document.id.strategy.partial.key.projection.type":"ALLOWLIST",
    "writemodel.strategy":"com.mongodb.kafka.connect.sink.writemodel.strategy.ReplaceOneBusinessKeyStrategy",
    "delete.on.null.values": "true",
    "transforms":"WrapKey",
    "transforms.WrapKey.type":"org.apache.kafka.connect.transforms.HoistField$Key",
    "transforms.WrapKey.field":"_id"
}

}

not able to delete records if I'm sending the null records the document is updated with null not deleting any idea?


Solution

  • As stated in the MongoDB Sink docs,

    delete.on.null.values

    boolean

    Whether the connector should delete documents with matching key values when value is null.

    You have to set your message in the format :

    {"key" : "xyz", "value" : null, ... }