Neo4j write to Kafka Topic from Trigger

Question

I am looking for a way to write to a Kafka Topic from a Neo4j-Trigger to implement event driven notification. I tried using a Kafka Producer in UDFs/UDPs like this:

@UserFunction(name = "kafka.write_to_topic")
public void writeToKafka(@Name("topic") String topic, @Name("message") String message) {
    Properties producerConfig = new Properties();
    producerConfig.put(BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
    Producer<String, String> producer = new KafkaProducer<String, String>(producerConfig, new StringSerializer(), new StringSerializer());
    producer.send(new ProducerRecord<>(topic, null, message));
}

But sadly the method is not recognized. Since I cannot find an error message, I tested the function line by line, and the problematic line of code is

    Producer<String, String> producer = new KafkaProducer<String, String>(producerConfig, new StringSerializer(), new StringSerializer());

Adding this line leads to the method not being recognized in the cypher query. I would like to have something similar to CALL streams.publish('my-topic', 'Hello World from Neo4j!'), which could be called from a trigger. But since the Neo4j-Streams-Procedures are deprecated, I don't want to use them. The Neo4j-Kafka-Connector only offers periodical querying, but I'm trying to find an event-driven way. Any ideas if this is possible?

Answer 1

The most reliable solution is now to use the Change Data Capture feature of Neo4j available since 5.13.

Basically, you would create a small Java application that uses the CDC procedures with the Neo4j driver and publish change events to Kafka as they happen.

There might be a Kafka connect connector using CDC coming in the future.