Search code examples
javascriptnode.jsxmlmeteorxmlhttprequest

GET response content is empty if called from server (Meteor)

So what I am trying to do is to parse this XML-based RSS feed on Meteor server-side using standard HTTP package. The confusion is that I receive an empty content in response if called from the server, but if I call the same endpoint from the client then I receive a normal XML content. I tried to call the server method both from a client using Meteor.call and also I tried to call it using meteor shell. Same result

Here is the Meteor method:

readRssFeed: function() {
    try {
        HTTP.get("http://agenda.regiolive.ch/feed/", {}, function(error, response) {
            console.log(response)
            //parse xml
        });
    } catch (error) {
        console.log(error);
    }
},

The response is:

I20170514-15:20:06.474(3)? { statusCode: 200,
I20170514-15:20:06.474(3)?   content: '',
I20170514-15:20:06.474(3)?   headers:
I20170514-15:20:06.474(3)?    { date: 'Sun, 14 May 2017 12:12:04 GMT',
I20170514-15:20:06.474(3)?      server: 'Apache-Coyote/1.1',
I20170514-15:20:06.474(3)?      'access-control-allow-origin': '*',
I20170514-15:20:06.474(3)?      'content-type': 'application/xml;charset=UTF-8',
I20170514-15:20:06.474(3)?      'content-length': '0',
I20170514-15:20:06.474(3)?      'set-cookie':
I20170514-15:20:06.474(3)?       [ 'cfid=9d25f146-1fca-42d2-927e-88fb9e458bfa;Path=/;Expires=Mon, 13-May-2047 20:03:34 GMT;HTTPOnly',
I20170514-15:20:06.475(3)?         'cftoken=0;Path=/;Expires=Mon, 13-May-2047 20:03:34 GMT;HTTPOnly',
I20170514-15:20:06.475(3)?         'CF_CLIENT_AGENDA_REGIOLIVE_CH1104_LV=1494764405749;Path=/;Expires=Wed, 24-May-2017 12:20:06 GMT',
I20170514-15:20:06.475(3)?         'CF_CLIENT_AGENDA_REGIOLIVE_CH1104_TC=1494764405749;Path=/;Expires=Wed, 24-May-2017 12:20:06 GMT',
I20170514-15:20:06.475(3)?         'CF_CLIENT_AGENDA_REGIOLIVE_CH1104_HC=2;Path=/;Expires=Wed, 24-May-2017 12:20:06 GMT' ],
I20170514-15:20:06.475(3)?      'keep-alive': 'timeout=5, max=100',
I20170514-15:20:06.475(3)?      connection: 'Keep-Alive' },
I20170514-15:20:06.475(3)?   data: null }

If I execute the same code on client, I receive the expected response with the XML content

{
    "statusCode": 200,
    "content": "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<feed xmlns=\"http://www.w3.org/2005/Atom\"\nxmlns:ev=\"http://purl.org/rss/1.0/modules/event/\"\nxmlns:geo=\"http://www.w3.org/2003/01/g.....>,
    "headers": {
        "content-type": "application/xml;charset=UTF-8"
    },
    "data": null
}

I also tried to use a Node.js-style HTTP request. Same result

Solution

  • If you navigate directly to http://agenda.regiolive.ch/feed/ in a browser, you get a gzipped response:

    HTTP/1.1 200 OK
Date: Sun, 14 May 2017 13:33:08 GMT
Server: Apache-Coyote/1.1
Access-Control-Allow-Origin: *
Content-Type: application/xml;charset=UTF-8
Content-Encoding: gzip
                  ^^^^

    Given that to indicate they accept gzipped responses, browsers send the following request header:

    Accept-Encoding: gzip, deflate

    …then it appears that if http://agenda.regiolive.ch/feed/ doesn’t get that request header, it sends no response body at all; but it if does get the header, then it sends the XML as expected—but gzipped.

    So you could try making your code also send a request with Accept-Encoding: gzip, deflate, and you’d get a response body with the XML as expected too—but gzipped.

    But you don’t really need a gzipped response, so what you can try is, make your code instead add an Accept-Encoding: identity request header to the request.

    https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Encoding

    identity
          Indicates the identity function (i.e. no compression, nor modification).

    Sending Accept-Encoding: identity with curl gets the (non-null) XML response as expected:

    $ curl -s -H "Accept-Encoding: identity" http://agenda.regiolive.ch/feed/ | head

<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom"
xmlns:ev="http://purl.org/rss/1.0/modules/event/"
xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#">
<author>
<name>Regiolive Agenda</name>
</author>
<title>Regiolive Agenda</title>
<id>http://regiolive.ch/</id>
<updated>2017-05-14T17:00:00Z</updated>

    So if you make your code add Accept-Encoding: identity, you should get the XML you need.