Search code examples
node.jsdockeramazon-ecs

Node ECS Task Not Crashing


Node ECS Task Not Crashing

I have an ECS worker running Node:Boron (v6.9.0).
Every now and then I get a uncaught exception.
For Example:

{
  "date": "Mon Jul 03 2017 09:09:29 GMT+0000 (UTC)",
  "process": {
  "pid": 28,
  "uid": 0,
  "gid": 0,
  "cwd": "/usr/src/app",
  "execPath": "/usr/local/bin/node",
  "version": "v6.11.0",
  "argv": [
    "/usr/local/bin/node",
    "/usr/src/app/src/poll.js"
  ],
  "memoryUsage": {
    "rss": 67502080,
    "heapTotal": 33660928,
    "heapUsed": 25145608,
    "external": 8981275
  }
},
{
  "os": {
    "loadavg": [
      0.65869140625,
      0.44921875,
      0.4541015625
    ],
    "uptime": 218908
  },
  "trace": [
    {
      "column": null,
      "file": null,
      "function": "Error",
      "line": null,
      "method": null,
      "native": true
    }
  ],
  "stack": [
    "Error: ENOENT: no such file or directory, open 'FILE.EXE'",
    " at Error (native)"
  ],
  "level": "error",
  "message": "uncaughtException: ENOENT: no such file or directory, open 'FILE.EXE'",
  "message": "uncaughtException: ENOENT: no such file or directory, open 'FILE.EXE'",
  "timestamp": "2017-07-03T09:09:29.645Z"
}

What I expect to happen is for the app to crash, restart and try again later. And if it can't process it x times, it goes in the dead letter queue.

The problem is, it just hangs and when I look at the task in ECS, it says "RUNNING".

So, it never restarts as it doesn't crash.

Is there any sort of configuration I need to do to get a Node Docker App to crash on an uncaught exception?


Solution

  • This is the piece of code that was failing:

    try {
      const file = fs.ReadStream(filepath);
      file.on('data', (d) => {
        task();
      }).on('end' () => {
        task();
      }
    } catch (e) {
      logger.FATAL(err, {backtrace: err.stack}, () => process.exit(1));
    }
    

    Thanks to @AndyShinn, I added a listener for the error event:

    .on('error', () => process.exit(1))
    

    And all works now.

    So, by adding an explicit error listener for a readable stream it is fixed.