Search code examples
node.jsexpressproxyrestifynode-http-proxy

Error: connect EMFILE and node-http-proxy


I have a few node processes that I'm trying to reverse proxy into one localhost port. Node-http-proxy seemed like the simplest solution. I'm proxying to a couple of node.js process running express (port 3100 & 3000 in the example below), and a process running node.js with restify (2700).

var http = require('http'),
    httpProxy = require('http-proxy');

var proxy = httpProxy.createProxyServer({});

var server = require('http').createServer(function(req, res) {
    if (req.url.match(/^(\/api\/search|\/api\/suggest)/g)) {
        proxy.web(req, res, { target: 'http://127.0.0.1:2700' });
    } else if (req.url.match(/^\/pages\//g)){
        proxy.web(req, res, { target: 'http://127.0.0.1:3100' });
    } else {
        proxy.web(req, res, { target: 'http://127.0.0.1:3000' });
    }
});

server.listen(9999);

So, during testing I started to realize that the sever at 9999 stopped serving files after about 100 serves and saw that the node-http-proxy process was throwing:

{ [Error: connect EMFILE] code: 'EMFILE', errno: 'EMFILE', syscall: 'connect' }

I know that EMFILE is usually caused by a limit OS on files open. I could up the limit but I don't think that would help. I tried running access the servers at 3000, 3100, and 2700 by doing a looping connection every 100ms and everything went fine- going for thousands of serves without any issues. I have also run this behind a nginx reverse proxy and it is working successfully over thousands of serves. I feel like I'm doing something wrong with the node-http-proxy - like I'm not closing something out. Any ideas?


Solution

  • This is most likely because there no agent passed so Node assign new agent to handle every request and because the keep-alive the node will not kill the connection after the request so it will leaks

    temporary solutions:-

    1 - you can assign new agent to every request

    var proxy = httpProxy.createProxyServer({agent: new http.Agent()});
    

    2- you send connection close to request headers

    server = http.createServer(function(req, res) {
      req.headers.connection = "Close";
      return proxy.web(req, res, {
        target: 'http://127.0.0.1'
      });
    });
    

    reference:-

    https://github.com/nodejitsu/node-http-proxy/issues/570

    https://github.com/nodejitsu/node-http-proxy/pull/572

    https://github.com/nodejitsu/node-http-proxy/pull/573