How to synchronize transactions with Spring Tx?

Question

I have three different sources to which I want to send information:

1. An Oracle database, related to a JpaRepository.
2. A MongoDB, related to a MongoRepository.
3. And a Kafka topic, which has configured via yml all the necessary data for producing into that topic.

I want to synchronize these resources with Spring Tx. I have a JpaTransactionManager related to the Oracle db, another MongoTransactionManager related to the MongoDB, and the KafkaTransactionManager created by Spring automatically when the transactions are activated.

The idea I had in mind was the following one:

1. First, I would have a basic controller that would autowire a service:

@RestController
public class SimpleController {

  @Autowired
  private SimpleService simpleService;

  @PostMapping(...)
  public Response doTx () {
    simpleService.doTx();
}

2. Then, that service would autowire at the same time another three services, one for each resource I have:

@Service
public class SimpleService {

  @Autowired
  private OracleService oracleService;

  @Autowired
  private MongoService mongoService;

  @Autowired
  private KafkaService kafkaService;
  
  ...
}

3. Such SimpleService, would have a method anotated with the @Transactional annotation in which these services are called:

@Service
public class SimpleService {

  @Transactional
  public void doTx () {

    oracleService.doTx();
    mongoService.doTx();
    kafkaService.doTx();

  }

}

4. And, each of these calls to these methods would be annotated with the @Transactional annotation too:

public class OracleService {

  @Autowired
  JpaRepository jpaRepository

  @Transactional
  public void doTx(){
    jpaRepository.save();
  }
}

What I would hope of this, is to initiate a transaction in SimpleService, and each of the other services that I call, join such transaction, and commit one after the other just when the method is about to return, at the end.

But this doesn't work. The only way it "works" (and I say "works" because it doesn't allow me to fully customize the order of committing the resources) is having all the resources in one service, all called from the same method:

@Service
public class SimpleService {

  @Autowired
  private JpaRepository oracleJpaRepository;

  @Autowired
  private MongoRepository mongoRepository;

  @Autowired
  private KafkaTemplate kafkaTemplate;

  @Transactional(transactionManager = "oracleJpaTransactionManager")
  public void doTx() {
    oracleJpaRepository.save();
    mongoRepository.save();
    kafkaTemplate.send();

  }

So the questions is, basically, if my first approach a valid approach (which I think it is since it appears at some Spring Kafka code samples included in the documentation) and how to make it work (without the ChainTransactionManager which is deprecated).

All help is appreciated :D!

I tried to synchronize transactions with the above approach, but it would just commit whenever the method was called, not joining the previous transaction.

EDIT: I'm not trying to achieve full transactionality. Just synchronization, a Best Effort 1 Phase Commit. I know there may be some data inconsistency when synchronizing these resources, compensation is not a problem, the key is the chaining of the commits at the end of the method.

EDIT 2:

Following Spring Kafka Documentation: https://docs.spring.io/spring-kafka/reference/html/#ex-jdbc-sync

@Transactional("dstm")
public void someMethod(String in) {
    this.jdbcTemplate.execute("insert into mytable (data) values ('" + in + "')");
    sendToKafka(in);
}

@Transactional("kafkaTransactionManager")
public void sendToKafka(String in) {
    this.kafkaTemplate.send("topic2", in.toUpperCase());
}

The following example shows how transactions are nested, and they do synchronize.

Answer 1

So, in short terms, transactional synchronization with more than one database and kafka doesn't work.

Transactional synchronization at its full capacity looks like it will only work with a deprecated Spring Transaction class, ChainedTransactionManager, which does synchronize at the end of the method all operations made with Transaction Managers specified in such ChainedTransactionManager.

Nested calls to Transactional annotated methods won't synchronize.

Databases and Kafka will only be synchronized if only one database is put together with Kafka, as the comments to the question states it will break once a second database enters the game.

You can either use another pattern (Transactional Outbox sounds solid for these cases, but it comes with a higher temporal frame of data inconsistency), or copypaste the ChainedTransactionManager class into your libraries/sample and play with it.

Note that the compensation once a commit fails will have to be done manually, since this is not XA or similar. So if you synchronize multiple databases and kafka, be sure to catch the Heuristic exceptions and compensate properly.

If you understand the drawbacks of using the ChainedTransactionManager it's an excellent tool to obtain such synchronization when your application needs to commit all resources together without business logic in between. Don't expect a Two-phase commit, this is a Best Effort 1 phase commit. So when a commit fails, all the commits made earlier won't roll back, you have to compensate manually.