AVA tests called within a function working on mac, but not on linux

Question

I have some AVA tests that I am trying to run multiple times, but on different mocks. For example, I want to run the same 5 tests, but ensure that these tests work for variously structured data. I've devised a way to import mocks into the test file and compare them to how they should parse out which looks similar to the following:

import test from 'ava';
import * as fs from 'fs';

let mockSampleEmail;
let mockDecomposedEmail;

function readJsonFile(fname) {
  return JSON.parse(fs.readFileSync(fname).toString());
}

fs.readdir('pre', (err, files) => {
  files.forEach(file => {
    if (!file.match(/\.json$/)) {
      return null;
    }
    mockSampleEmail     = readJsonFile("pre/" + file);
    mockDecomposedEmail = readJsonFile("post/" + file);
    runEmailDecomposerTests(mockSampleEmail, mockDecomposedEmail, mockRiskyUpdates, file);
  })
})

The runEmailDecomposerTests function looks like this:

const runEmailDecomposerTests = (mockSampleEmail, mockDecomposedEmail, mockRiskyUpdates, fname) => {
  test(`(${fname})  Converts the email message received from the websocket into a simple email object`, t => {
    //Assertions here
  })
}

My Directory structure looks like this:

test/
--emails/
----emailDecomposer.spec.js
----pre/
------sampleEmail.json
----post/
------sampleEmail.json

This all works locally on my mac, but when I push it up to our jenkins server to test for continuous integration, it fails with the error:

 ✖ No tests found in test/emails/emailDecomposer.spec.js

Even though I can confirm that the runEmailDecomposerTests function is definitely being called. The confusing part is that it passes locally on my mac. The jenkins server is a Linux vm so that's why I'm leaning towards a mac/linux issue, but I can't be sure. Even further, about one in 5 times it passes on our CI server, so maybe its a race condition of some kind?

Answer 1

From the AVA docs:

You must define all tests synchronously. They can't be defined inside setTimeout, setImmediate, etc.

fs.readdir is async, so I'm surprised it actually works on macOS, but as you've experienced, it leads to race issues. I would suggest either switching to fs.readdirSync or doing the async operation in a test.before() hook and then take advantage of t.context.