I am running a batch merge operation in Neo4j, but it keeps on failing. I increased the heap size and heap limit but is seems like my operations is perhaps not supported or there's a more appropriate method of doing this.
I have 10k of such MERGE statements
var transaction = `
MERGE (n0: Account {sfRecordId:$Id0})
ON CREATE SET ...
ON MATCH SET ...
MERGE (n1: Account {sfRecordId:$Id1})
ON CREATE SET ...
ON MATCH SET ...
MERGE ... //10k
// then send to Neo4j via the Javascript driver:
tx.run(transaction, bindParams)
My heap size is set to
server.memory.heap.initial_size=2G
server.memory.heap.max_size=3G
I have tried sending them all at once, I have tried batching them into batches of 100, 500, 1, and 10,000.
None of these seems to get my 10k to be inserted.
The first random 500-4k get in, but then the server crashes with an out of memory error.
Generating a very large statement to bulk insert data is not recommended. A good way of inserting such bulk data is to prepare a list of objects, and use UNWIND
to iterate through it:
UNWIND $data as item
MERGE (n:Account {sfRecordId:item.Id})
ON CREATE SET ...
ON MATCH SET ...