How to use/close Gremlin Transactions Properly in Javascript/AWS Lambda

Question

I've encountered some behavior with gremlin transactions in javascript and my websocket connection being stuck in open. I've read

https://docs.aws.amazon.com/neptune/latest/userguide/access-graph-gremlin-transactions.html https://docs.aws.amazon.com/neptune/latest/userguide/best-practices-gremlin-java-close-connections.html https://docs.aws.amazon.com/neptune/latest/userguide/lambda-functions.html

but am not 100% sure about whats happening or what I'm supposed to do with regards to transactions and their behaviors. Based on the docs, it seems like in a lambda environment, I don't call await dc.close()

Locally, I just run node testNeptune. This code locally makes all writes fail afterwards

const makeHangingTransaction = async function () {
    let dc = new DriverRemoteConnection(
        '<myurl>',
        { pingInterval: 5000, pingTimeout: 5000 }
    );
    let graph = new Graph();
    let g = graph.traversal().withRemote(dc);
    await dropAll(g);
    let tx = g.tx();
    let gtx = tx.begin();
    try {
        console.log('transaction test1');
        console.log(await gtx.addV('person').next());
        console.log(await gtx.V().toList());
        console.log();
        await tx.commit();
    } finally {
        tx.close();
    }
    try {
        console.log('transaction test1');
        console.log(await gtx.addV('person').next());
        console.log(await gtx.V().toList());
        console.log();
        await tx.commit();
    } finally {
        tx.close();
    }
};
makeHangingTransaction();

Script runs, and then dies with Error: Connection has been closed.; my guess is that tx.close() closes, then the "parent" dc connection itself dies and throws this error without terminating the actual websocket connection. I know tx.close() is supposed to be awaited.

This creates a "hanging" transaction, I can no longer do any write commands from anywhere; both my aws jupyter and local will hang and then fail with a Failed to complete Insert operation for a VertexProperty due to conflicting concurrent operations. Please retry. 0 transactions are currently rolling back . I have to wait some minutes before the websocket(?) times out and I can do writes again.

Questions:

1. If my code messes up and the websocket connection is in an "open" state, is there a way to force-reset this or create a new connection?
2. Why would my jupyter writes fail when as far as I know, it should be using a different connection to my neptune server
3. Should I always manually call await dc.close() locally?
4. Would I ever use await tx.close()? This actually never did anything for me; my script kept waiting. I had to call await dc.close() after to exit.

Thanks

Answer 1

Hopefully the notes below can help you with your current issues:

For #1 - You can cancel an on-going query/transaction via the Gremlin Status API and query cancellation API: https://docs.aws.amazon.com/neptune/latest/userguide/gremlin-api-status-cancel.html

For #2 - It ultimately depends on what has been locked in the database and if you have reached a deadlock situation. This includes gap locks that are taken, given Neptune's indexing structure. Here's a broader explanation: https://docs.aws.amazon.com/neptune/latest/userguide/transactions-neptune.html#transactions-neptune-false-conflicts

For #3 and #4 - You don't need to "close" a transaction. Committing it is enough, as if the commit fails, the transaction will automatically rollback. As noted here: https://docs.aws.amazon.com/neptune/latest/userguide/access-graph-gremlin-transactions.html#access-graph-gremlin-transactions-session-bound you can force a rollback if you get to a portion of your code where the transaction is still open and you expect it to be committed by that point.

If you're using AWS Lambda, it is general good practice to establish a connection outside of the Lambda handler and assume that connection is always open for any following Lambda function invocations. This reduces the overhead of opening/closing connections on every invocation. A good starting point for a well structured Javascript Lambda function for use with Neptune can be found here: https://docs.aws.amazon.com/neptune/latest/userguide/lambda-functions-examples.html#lambda-functions-examples-javascript. This includes exception handling, retries, and reconnect logic that you would typically need for Lambda-based application using Neptune.