I am having further problems with async code in Jest. My previous question (for the same project) was related to running async code in a Jest bootstrap. My new issue relates to running async database calls within the tests. My aim is to connect to database services and to make calls to make sure they read and write to the database correctly. I have the tests running in one Docker container, connecting to a MySQL instance in another container.
I am using the mysql2/promise Node library, which as the same suggests, wraps callback-based database operations in a Promise. Most of the operations are async, except for connection closing (and a few others). Indeed, I wonder if this is relevant.
I should start with some code. Here is my test:
import TestDatabase from '../TestDatabase';
var config = require('../../config/config.json');
import FetchDonations from "../../src/services/FetchDonations";
const envName = 'test';
let database = new TestDatabase(config);
// Connect before all tests
beforeAll(() => {
console.log('Connect Jest database');
return database.connect(envName);
});
// Disconnect after all tests
afterAll(async done => {
console.log('Disconnect Jest database');
database.close();
done();
});
describe('Database tests', () => {
// Before every test
beforeEach(() => database.beforeEachTest(envName));
test('Describe this demo test', () => {
console.log('Test #1');
expect(true).toEqual(true);
});
test('Describe this demo test 2', () => {
console.log('Test #2');
expect(true).toEqual(true);
});
});
This just runs a couple of dummy tests. They don't do anything, I'm just trying to get the before/after hooks working. These are what they should do:
database.beforeEachTest() before every test, this truncates the tables in the database (asynchronous ops)Here is what TestDatabase looks like - these are utility methods I've written to help with database testing:
const mysql = require('mysql2/promise');
export default class TestDatabase {
constructor(config) {
this.config = config;
}
beforeEachTest(environmentName) {
console.log('Before a test');
return this.setForeignKeyChecks(false).then(() => {
return this.truncateTables();
}).then(() => {
return this.setForeignKeyChecks(true);
}).catch((error) => {
console.log('Failed to clear down database: ' + error);
});
}
connect(environmentName) {
const config = this.getEnvConfig(environmentName);
return mysql.createConnection({
host: config.host, user: config.username,
password: config.password
}).then((connection) => {
this.connection = connection;
return this.useDatabase(environmentName);
}).catch((error) => {
console.log('Failed to connect to the db');
});
}
getConnection() {
if (!this.connection) {
throw 'Database not connected';
}
return this.connection;
}
dropDatabase(environmentName) {
const config = this.getEnvConfig(environmentName);
return this.getConnection().query(
`DROP DATABASE IF EXISTS ${config.database}`
);
}
createDatabase(environmentName) {
const config = this.getEnvConfig(environmentName);
return this.getConnection().query(
`CREATE DATABASE IF NOT EXISTS ${config.database}`
);
}
useDatabase(environmentName) {
const config = this.getEnvConfig(environmentName);
return this.getConnection().query(
`USE ${config.database}`
);
}
setForeignKeyChecks(value) {
// Make injected value safe
var boolStr = value ? '1' : '0';
return this.getConnection().query(
`SET FOREIGN_KEY_CHECKS = ${boolStr}`
);
}
getTables() {
return ['contribution', 'donation', 'expenditure',
'tag', 'expenditure_tag'];
}
truncateTables() {
return Promise.all(
this.getTables().map(table => this.truncateTable(table))
);
}
truncateTable(table) {
return this.getConnection().query(
`TRUNCATE TABLE ${table}`
);
}
/**
* Close is synchronous so there is no returned promise
*/
close() {
this.getConnection().close();
}
getEnvConfig(environmentName) {
if (!environmentName) {
throw 'Please supply an environment name'
}
if (!this.config[environmentName]) {
throw 'Cannot find database environment data'
}
return this.config[environmentName];
}
}
Now, if I run the tests, they pass and finish, but there are two oddities. Firstly, the some of the async console.log output is being output after the test summary, so I think I am not handling async in the way Jest wants it. In other words, I think the summary should be rendered after all of this:
/project/node_modules/.bin/jest tests
console.log
Connect Jest database
at Object.<anonymous> (tests/database/TestDemo.test.js:29:11)
console.log
Before a test
at TestDatabase.beforeEachTest (tests/TestDatabase.js:10:13)
PASS tests/database/TestDemo.test.js
Database tests
✓ Describe this demo test (72ms)
✓ Describe this demo test 2 (58ms)
Test Suites: 1 passed, 1 total
Tests: 2 passed, 2 total
Snapshots: 0 total
Time: 2.118s, estimated 3s
Ran all test suites matching /tests/i.
console.log
Test #1
at Object.<anonymous> (tests/database/TestDemo.test.js:46:13)
console.log
Before a test
at TestDatabase.beforeEachTest (tests/TestDatabase.js:10:13)
console.log
Test #2
at Object.<anonymous> (tests/database/TestDemo.test.js:51:13)
console.log
Disconnect Jest database
at _callee$ (tests/database/TestDemo.test.js:35:11)
As you can see, output from both Tests appears after the summary, but output from the beforeEach for the first test appears before the test summary.
Moreover, if I add real tests that use the database, I get errors saying that I have unhandled promises, and that I should try Jest's unhandled promise detector (--detectOpenHandles). Moreover, in that situation, Jest stops in a loop and needs ^C to give the console prompt back.
So, I am trying --detectOpenHandles with the current code, and although I don't get a Jest freeze, I get the following.
Jest has detected the following 1 open handle potentially keeping Jest from exiting:
● TCPWRAP
22 | const config = this.getEnvConfig(environmentName);
23 |
> 24 | return mysql.createConnection({
| ^
25 | host: config.host, user: config.username,
26 | password: config.password
27 | }).then((connection) => {
at new Connection (node_modules/mysql2/lib/connection.js:35:27)
at Object.<anonymous>.exports.createConnection (node_modules/mysql2/index.js:10:10)
at Object.createConnection (node_modules/mysql2/promise.js:230:31)
at TestDatabase.connect (tests/TestDatabase.js:24:18)
at Object.<anonymous> (tests/database/TestDemo.test.js:30:19)
My view is that this is directly connected to the freeze I get with more tests, and that I should fix that before attempting to add more tests.
I have been through several investigation loops to determine what might cause this, and the code has been tweaked several times:
afterAll and beforeEach are async ops, so they need to be returned to Jest, so Jest knows to wait for them to resolve.afterAll does a db close, but this is not async, so I am using Jest's done() here, though it also did not work if it is done without done().TestDatabase contains two main methods, beforeEachTest and connect, and I have been very careful to ensure they return Promises.dropDatabase, createDatabase, setForeignKeyChecks, truncateTables, truncateTable all return Promises.I am fairly new to Jest, and not too experienced in JS async either. Every time I think I have an improved understanding of async, I get a fresh curveball. However, I wonder if this is more Jest oddities, rather than a difficulty understanding raw async.
It's best to move the server connection code into a function into a separate file then export it, and call it in your jest tests. That may stop the errors open handle potentially keeping jest from exiting.
It's dangerous to use --forceExit as it can prematurely terminate an operation that hasn't completed (e.g. a DB cleanup operation), if that runs after the tests have completed.