Search code examples
spring-bootapache-kafkamicroservicesspring-kafkachange-data-capture

Transactional outbox pattern vs ChainedKafkaTransactionManager in Microservices


Using Spring-Kafkas ChainedKafkaTransactionManager I cannot see any point in implementing the transactional outbox pattern in a Spring Boot microservices context.

Putting message producer (i.e. KafkaTemplate's send method) and DB operation in the same transactional block solves exactly the problem that should be solved by the outbox pattern: If any exception is raised in the transactional code neither the db op is commited nor the message is read on the consumer side (configured with read_committed)

This way I dont need an additional table nor any type of CDC code. In summary the Spring Kafka way of transaction synchronization seems much easier to use and implement to me than any implementation of transactional outbox pattern.

Am I missing anything?

public ChainedKafkaTransactionManager chainedTransactionManager(
                        JpaTransactionManager transactionManager,
                        KafkaTransactionManager kafkaTransactionManager) {
        ChainedKafkaTransactionManager chainedKafkaTransactionManager = 
            new ChainedKafkaTransactionManager<>(transactionManager, 
                                                 kafkaTransactionManager);
        
        return chainedKafkaTransactionManager;
    }

    @Bean
    @Primary
    public JpaTransactionManager transactionManager(EntityManagerFactory 
        entityManagerFactory) {
        JpaTransactionManager jpaTransactionManager = 
                    new JpaTransactionManager(entityManagerFactory);
    
        return jpaTransactionManager;
    }

    @Bean
    public KafkaTransactionManager<Object, Object> 
             kafkaTransactionManager(ProducerFactory producerFactory) {
        KafkaTransactionManager kafkaTransactionManager = 
           new KafkaTransactionManager<>(producerFactory);
        
        return kafkaTransactionManager;
    }


    @Transactional(value = "chainedTransactionManager")
    public Customer createCustomer(Customer customer) {
        customer = customerRepository.save(customer);
        kafkaTemplate.send("customer-created-topic","Customer created");
          
        return customer;
    }

Solution

  • I think it doesn't give you the same level of safety. What if something fails between Kafka commit and DB commit.

    https://medium.com/dev-genius/transactional-integration-kafka-with-database-7eb5fc270bdc