express body-parser utf-8 error in test

Question

Super stumped by this. I have some server code that for some reason throws a UTF-8 error in my tests but works fine when running the server normally:

code:

export default ({ projectId = PROJECT_ID, esHost = ES_HOST } = {}) => {
  let app = express();
  app.use(cors());
  app.use(bodyParser.json({ limit: '50mb' }));

  let http = Server(app);
  let io = socketIO(http);

  let server = {
    app,
    io,
    http,
    status: 'off',
    listen(
      port = PORT,
      cb = () => {
        rainbow(`⚡️ Listening on port ${port} ⚡️`);
      },
    ) {
      this.http.listen(port, () => {
        main({ io, app, projectId, esHost, port });
        this.status = 'on';
        cb();
      });
    },
    close(cb = () => {}) {
      if (this.http) {
        this.http.close(() => {
          this.status = 'off';
          cb();
        });
      } else {
        throw '❗️ cannot close server that has not been started ❗️';
      }
    },
  };

  return server;
};

usage (exactly the same, but in jest test body-parser isn't working properly):

import createServer from '../server'

let server = createServer()
server.listen(5050);

I'm using postman, post response outside of test:

{
    "projects": [
        {
            "id": "test",
            "active": true,
            "timestamp": "2018-02-25T21:33:08.006Z"
        },
        {
            "id": "TEST-PROJECT",
            "active": true,
            "timestamp": "2018-03-05T21:34:34.604Z"
        },
        {
            "id": "asd",
            "active": true,
            "timestamp": "2018-03-06T23:29:55.348Z"
        }
    ],
    "total": 3
}

unexpected post response inside jest test server:

Error

UnsupportedMediaTypeError: unsupported charset "UTF-8"
    at /Users/awilmer/Projects/arranger/node_modules/body-parser/lib/read.js:83:18
    at invokeCallback (/Users/awilmer/Projects/arranger/node_modules/raw-body/index.js:224:16)
    at _combinedTickCallback (internal/process/next_tick.js:131:7)
    at process._tickCallback (internal/process/next_tick.js:180:9)

Answer 1

So I was able to reproduce the issue and find the source of the issue and the workaround to make it work. The issue is caused by jest framework.

Before you jump on reading the rest of the thread, I would suggest you read another Jest thread I answer long back. This would help get some context internals about the require method in jest

Specify code to run before any Jest setup happens

Cause

The issue happens only in test and not in production. This is because of jest require method.

When you run your tests, it starts a express server, which calls the node_modules/raw-body/index.js as shown in below image

As you can see the encodings is null. This is because the iconv-lite module does a lazy loading of encodings. The encodings are only loaded when getCodec method gets executed.

Now when your test has fired the API, the server needs to read the body so the getCodec gets called

This then goes through the jest-runtime/build/index.js custom require method (which is overloaded if you read the previous link).

The execModule has a check for this._environment.global, which is blank in this case and hence a null value is returned and the module never gets executed

Now when you look at the exports of the encodings module, it just is a blank object

So the issue is purely a jest. A feature jest lacks or a bug mostly?

Related Issues

Related issues have already been discussed on below threads

https://github.com/facebook/jest/issues/2605

https://github.com/RubenVerborgh/N3.js/issues/120

https://github.com/sidorares/node-mysql2/issues/489#issuecomment-313374683

https://github.com/ashtuchkin/iconv-lite/issues/118

https://github.com/Jason-Rev/vscode-spell-checker/issues/159

Fix

The fix to the problem is that we load the module during our test itself and force a early loading instead of lazy loading. This can be done by adding a line to your index.test.js at the top

import encodings from '../../node_modules/iconv-lite/encodings';
import createServer from '@arranger/server';

After the change all the test pass, though you have a error in the url of the test so you get Cannot POST /