I am trying load some data from a CSV file into a clean db instance. I did this by creating an ETL process using Apache Camel it reads the input CSV split the file into multiple lines and process each line in parallel using one transaction per line.
class ImportRouteBuilder extends RouteBuilder {
private final Importer importer
private final String endpoint
public ImportRouteBuilder(Importer importer, String endpoint) {
this.endpoint = endpoint
this.importer = importer
}
@Override
void configure() throws Exception {
from(endpoint)
.unmarshal(buildCsvDataFormat())
.split(body())
.parallelProcessing()
.bean(importer)
}
private static CsvDataFormat buildCsvDataFormat() {
CsvDataFormat csv = new CsvDataFormat();
csv.skipHeaderRecord = true
csv
}
}
@Slf4j
@Service
class CountryCsvImporter implements Importer {
@Autowired
private CountryRepository countryRepository
@Autowired
private Session session
@Override
void process(List record) {
Transaction tx = session.beginTransaction();
try {
importCountry(record)
tx.commit()
}catch (Throwable t) {
tx.rollback()
}
tx.close()
}
I did this because did not wanted to use LOAD CSV cypher or Neo4j import tool while my model is still evolving since it is very handy to use OGM while prototyping. But now I found a wall at the middle of the process OGM generates this TransactionManagerException, it looks like it was some threading issue managing transactions.
at com.cartrawler.service.CountryImportSpecification.Should be able to import country csv(CountryImportSpecification.groovy:9) Caused by: org.neo4j.ogm.exception.TransactionManagerException: Transaction is not current for this thread at org.neo4j.ogm.session.transaction.DefaultTransactionManager.rollback(DefaultTransactionManager.java:78) at org.neo4j.ogm.transaction.AbstractTransaction.rollback(AbstractTransaction.java:65) at org.neo4j.ogm.drivers.embedded.transaction.EmbeddedTransaction.rollback(EmbeddedTransaction.java:60) at com.cartrawler.service.CountryCsvImporter.process(CountryCsvImporter.groovy:28)
Thank you, kindly Luis Oscar
Turn off parallel processing as transaction is single thread only.