pg-promise transaction with dependent queries in forEach loop gives warning Error: Querying against a released or lost connection

Question

I am trying to insert linked data in a pg-promise transaction. The transaction successfully inserts all the data correctly, but gives a warning UnhandledPromiseRejectionWarning: Error: Querying against a released or lost connection..

The code that causes this was originally:

const item = {
  batch: { batch_number: 1 },
  ingredients: [
    { amount: '12', unit: 'kg' },
    { amount: '4', unit: 'L' }
  ],
}


return await db.tx(async t => {
  const batchQueryString = pgp.helpers.insert(item.batch, null, 'ionomer_batch')
  const batchQuery = await t.one(batchQueryString + ' RETURNING ionomer_batch_id')

  item.ingredients.forEach(async ingredient => {
    const ingredientQueryString = pgp.helpers.insert(ingredient, null, 'ingredient')
    const ingredientQuery = await t.one(ingredientQueryString + ' RETURNING ingredient_id')
    await t.none(
            `INSERT INTO ionomer_batch_step(ionomer_batch_id, ingredient_id) 
            VALUES(${batchQuery.ionomer_batch_id}, ${ingredientQuery.ingredient_id})`
            )
   })

   return batchQuery
  }).then(data => {
    return {success: true, response: data}
  }).catch(error => {
    return {success: false, response: error}
})

I got it to work without producing a warning by doing

return await db.tx(async t => {
  const batchQueryString = pgp.helpers.insert(item.batch, null, 'ionomer_batch')
  const batchQuery = await t.one(batchQueryString + ' RETURNING ionomer_batch_id')

  const ingredientQueries = []
  // this can't be async
  item.ingredients.forEach(ingredient => {
    const ingredientQueryString = pgp.helpers.insert(ingredient, null, 'ingredient')
    const ingredientQuery = t.one(ingredientQueryString + ' RETURNING ingredient_id')
    ingredientQueries.push(ingredientQuery)
   })
   const resolvedIngredientQueries = await t.batch(ingredientQueries)
   resolvedIngredientQueries.forEach(async ingredientQuery => {
     await t.none(
       `INSERT INTO ionomer_batch_step(ionomer_batch_id, ingredient_id) 
       VALUES(${batchQuery.ionomer_batch_id}, ${ingredientQuery.ingredient_id})`
      )
    })

   return batchQuery
  }).then(data => {
    return {success: true, response: data}
  }).catch(error => {
    return {success: false, response: error}
})

but I now have to loop through twice instead of once, and I lose the async in the first loop. It seems like there should be a way to do something closer to the first attempt without hitting a warning about released or lost connections. I played around with chaining the queries, but couldn't get that to work.

Answer 1

As an addition to @deanna own answer...

You do not really need to implement a loop. You can just re-map requests into array of promises, and then resolve it:

await db.tx(async t => {
    const batchQueryString = pgp.helpers.insert(item.batch, null, 'ionomer_batch');
    const batchQuery = await t.one(`${batchQueryString} RETURNING ionomer_batch_id`);

    const inserts = item.ingredients.map(async i => {
        const query = pgp.helpers.insert(i, null, 'ingredient');
        const ingredientQuery = await t.one(`${query} RETURNING ingredient_id`);
        return t.none(
            `INSERT INTO ionomer_batch_step(ionomer_batch_id, ingredient_id)
            VALUES($/batchQuery.ionomer_batch_id/, $/ingredientQuery.ingredient_id/)`,
            {batchQuery, ingredientQuery});
   });

    await t.batch(inserts); // settle all generated promises

    return batchQuery;
});

Also, you can see from the change, you should never use ES6 value injection like that. See here:

IMPORTANT: Never use the reserved ${} syntax inside ES6 template strings, as those have no knowledge of how to format values for PostgreSQL. Inside ES6 template strings you should only use one of the 4 alternatives - $(), $<>, $[] or $//.