NodeJS/ExpressJS send response of large amount of data in 1 stream

Question

I'm prototyping an app using the native mongo rest api where Node returns about 400K of json. I use the following to maket he request to mongo's native api and return the result:

http.request(options, function(req)
  {
    req.on('data', function(data)
      {
console.log(data,data.rows);
        response.send( 200, data );
      }
    );
  }
)
.on('error', function(error)
  {
console.log('error\t',error);
    response.send(500, error);
  }
)
.end();

When I hit http://localhost:8001/api/testdata via curl, the response is proper (both what is outputted to Node's console from the console.log and what is received by curl). But when I hit it via ajax in my app, the stream is…interupted, even data outputted to Node's console (Terminal) is odd: It has multiple EOFs, and the Network > response for the call in chrome's dev tools ends at the first EOF.

One other strange thing: data looks like:

{
    "offset": 0,
    "rows": [ … ]
}

but in neither Node nor client-side (angular) can I reference data.rows (it returns undefined). typeof data returns [object Object].

EDIT The request headers for both curl and angular (as reported by Node) are:

req.headers: {
  'x-action': '',
  'x-ns': 'test.headends',
  'content-type': 'text/plain;charset=utf-8',
  connection: 'close',
  'content-length': '419585'
}

EDIT I checked response headers in both angular and curl directly (instead of from Node), annnd there's a disagreement (same output from both curl and angular directly instead of from node):

access-control-allow-headers: "Origin, X-Requested-With, Content-Type, Accept"
access-control-allow-methods: "OPTIONS,GET,POST,PUT,DELETE"
access-control-allow-origin: "*"
connection: "keep-alive"
content-length: "65401" // <---------------- too small!
content-type: "application/octet-stream"
//             ^-- if i force "application/json"
// with response.json() instead of response.send() in Node,
// the client displays octets (and it takes 8s instead of 0s)
date: "Mon, 15 Jul 2013 18:36:50 GMT"
etag: ""-207110537""
x-powered-by: "Express"

Answer 1

Node's http.request() returns data in chunks for streaming (would be nice if they explicitly state this). Thus it's necessary to write each chunk to the body of Express's response, listen for the end of the http request (which is not really documented), and then call response.end() to actually finish the response.

var req = http.request(options, function(res)
  {
    res.on( 'data', function(chunk) { response.write(chunk); } );
    res.on( 'end', function() { response.end(); } );
  }
);
req.on('error', function(error) { … });
req.end();

Where response is Express's response the the initial client request (curl or angular's ajax call).