How to query my own web api from my own webapp?

Question

I've created a web application and I added my own API to it:

/api/v1.0/articles/new/{since}

This API returns JSON of new articles since {since}, where {since} is unix timestamp.

I'd like to add automatic updates to the site and automatically query articles while the user is on the site but I don't quite know how to do that.

I'm thinking of using the following javascript (jQuery) to do it:

setInterval(function () {
    $.get('/api/v1.0/articles/new/{since}', function (data) {
        /* parse data, add new articles to page */
        /* since = now */
    });
}, 30*1000)

Would this be the right solution? I'm worried that if 500 people visit my site, then I'll get 500 requests every 30 seconds, which might kill my server.

What other possible solutions I could try? Is there a way for the server to push data to webapp when there is something new, rather than me pulling it and querying? (Is this called comet ajax?)

Answer 1

Doing 30 second polling with with 500 users, you would be serving about 17 requests per second on average. Whether or not that would kill your server depends on what you are doing on each request, what other requests you are serving, what hardware your server is running, etc. If all you are doing on each request is a simple database query against a table--presumably indexed by timestamp--17 requests per second is pretty light. However, in my opinion you really should implement some form of server push--what you are calling "comet ajax". For one thing, if any of your API calls takes longer than 30 seconds, you will do another API call before the previous one finishes, potentially causing a "snowball" effect.

Exactly how you do server push depends on what is available on your server stack, and what you expect your client browsers to support. For example, if you are running Node.js, you can use the socket.io package to do server push using web sockets. But older browsers do not support web sockets, and not all browser implementations of web sockets are the same, so you may need to consider approaches based on AJAX long-polling, or multi-part document streaming. In general, you will need to install some code or package on your server to support the approach you use, though there are some solutions that will transparently determine which approach is best for each client connection. (For example, SignalR does this, but that is for the Microsoft / .NET stack.)

You might want to read the following article as a starting point, then post more targeted questions here based on your particular technology stack and your assumptions about which browsers you need to support.

http://techoctave.com/c7/posts/60-simple-long-polling-example-with-javascript-and-jquery