Spring Batch Bulk Insert with OneToMany Hierarchy

Question

I am using spring batch to read and transform a tree-like entity and write to an oracle database with the JpaItemWriter. For tree-like entity I mean:

class A {
    @OneToMany
    List<B> bList;
}

class B {
    @OneToMany
    List<C> cList;
}

class C {
    @OneToMany
    List<D> dList;
}

class D {

}

the size of the relations can grow fast but for business reasons is required to persist the entire entity at once in one transaction. with this 4-level hierarchy I want to know if is It possible to use some Hibernate (JPA preferred) approach to call bulk inserts in a top-down strategy using the spring JpaItemWriter to persist all "B" entities with one call to the database then all "C" entities and so on... instead of one insert for each entity as my debug log is showing. Thanks in advance for the help.

Answer 1

The short answer is that it should be possible if you are using Hibernate for your ORM implementation.

The Hibernate Docs delve into batch processing somewhat here. If correctly configured, Hibernate will attempt to order your inserts and updates by entity type during each flush() (at the end of each chunk).

You'll also want to make sure you have the following properties set and that you don't use IDENTITY generators for your IDs:

<prop key="hibernate.jdbc.batch_size">50<!--or some other number--></prop>
<prop key="hibernate.order_inserts">true</prop>
<prop key="hibernate.order_updates">true</prop>
<prop key="hibernate.jdbc.batch_versioned_data">true</prop>

This Red Hat example shoes how to do the same thing with a persistence unit.

Now, all that being said, Hibernate can be touchy, so you may want to bump up the logging for the BatchingBatcher class to confirm it is actually batching up your inserts. Finally, watch out for the saveOrUpdate() method in any entities that do not use sequences. If you are defining your own ID and your object is not yet in the session (which it won't be), Hibernate will first issue a select against the DB to determine whether an insert or update is required.