When doing load testing, I am seeing Deadlock errors happening occasionally, I added retries and its been working well. However, how do I know how to improve? The errors are quite vague so I cannot tell which query is causing this
{code: Neo.TransientError.Transaction.DeadlockDetected} {message: ForsetiClient[transactionId=75637, clientId=2431] can't acquire ExclusiveLock{owner=ForsetiClient[transactionId=75634, clientId=2450]} on NODE_RELATIONSHIP_GROUP_DELETE(116) because holders of that lock are waiting for ForsetiClient[transactionId=75637, clientId=2431].
The weird thing is I dont think I have code that does a relationship delete. I am actually doing MERGE to add nodes and relationships along the lines of:
MERGE (o:Org {id: $orgId})
SET o.name=$name
MERGE (a:Item {id: $itemId})
SET a.orgId=$orgId,
a.name=$itemName
MERGE (s:SubItem {id: $subItemId})
SET s.name=$subItemName
MERGE (o)-[:HAS_ITEM]->(a)
MERGE (a)-[:HAS_SUB_ITEM]->(s)
UPDATE:
Managed to identity the part causing the deadlock. I am modelling tags as a node and I suspect when many items use the same tag, and I try to merge the relationship, it causes a deadlock
MATCH (t:Tag {name: "a"})
MATCH (i:Item {name: "x"})
MERGE (i)-[:TAGGED]->(t)
I think this is the part causing deadlock often, how might I avoid this?
Found the reason why deadlocks happen very frequently already. Also thanks to @Tomaž Bratanič's whose link to the docs helped. Specifically this part
Lock taken on a node during the transaction creation phase to prevent deletion of that node and/or relationship group. This is different from the NODE lock in order to allow concurrent label and property changes together with relationship modifications.
And
Table 2. Obtained locks for graph modifications Creating a relationship* If the node is sparse: NODE lock. If a node is dense: NODE DELETE prevention lock.
Important part is the part about dense node. Because I am modelling :Category as a node, 1 category can have many edges creating a "dense node".
Googling more about it, I found https://medium.com/neo4j/graph-modeling-all-about-super-nodes-d6ad7e11015b and started attempting to make the node less dense with some trade offs. Basically I made it a category specific to the "user" rather than globally. So instead of the category being uniqued by name, I also added a userId of sort so its a "composite key". I also added a TEXT index on name such that I can query easily for all category regardless of user quickly