Programmatically measure performance of canvas animation

Question

I'm building a test platform for JavaScript performance contest. One of the tasks challenges contestants to optimize JavaScript code responsible for handling a canvas animation. After solution is submitted, server runs it using PhantomJS and reads average number of FPS after 20 seconds of animation. The problem is that I'm getting 3-4FPS for both optimized and unoptimized code. This makes it impossible to tell if code was improved.

Couple of facts:

• I'm 100% sure that phanotmjs is rendering animation correctly (done couple of screenshots)
• in a browser unoptimized code runs at 13FPS, optimized runs at 58FPS
• phantomjs doesn't support requestAnimationFrame so I had to use a polyfill
• I'm using this code below to test number of FPS

frameCounter.js

 var frameCounter = (function() {
    var frames = 0;
    var startTime = new Date();

    function bump() {
        frames++;
        window.requestAnimationFrame(bump);
    }

    bump();

    return {
        getFPS: function() {
            var time = (new Date() - startTime) / 1000;

            return (frames/time).toPrecision(4);
        }
    }
 })();

My question is: how can I programmatically measure performance of canvas animation?

Answer 1

Since phantomjs seems to be unable to produce more than 3-4 FPS on any animation, I ended up using 'real' browser for this task. I was able to automate it thanks to the Chrome's remote debugging protocol.

I made a node.js app that, each time there was new code to test, did the following steps:

• connected to a tab in the Chrome browser (the browser must be running with --remote-debugging-port=9222 flag)
• navigated tab to the test page
• evaluated code inside the tab that tried to render 300 frames of animation as quick as possible
• returned the execution time

Here is a snippet from my code:

//connect to a tab (you can find <tab-debug-id> on http://localhost:9222/json page)
var ws = new WebSocket("ws://localhost:9222/devtools/page/<tab-debug-id>");

ws.onerror = function() {
  //handle error
};

ws.onopen = function()
{
    //when connection is opened hard reload the page before we start
    ws.send(JSON.stringify({
        id: 1,
        method: "Page.reload",
        params: {
            ignoreCache: true
        }
    }));
};

ws.onmessage = function (evt)
{
    var data = JSON.parse(evt.data);

    if(data.id === 1) {
        //reload was successful - inject the test script
        setTimeout(function(){
           ws.send(JSON.stringify({
              id: 2,
              method: "Runtime.evaluate",
              params: {
                expression: '(' + injectedCode.toString() + '());'
              }
           }));
        }, 1000);
    } else if(data.id === 2) {
        //animation has finished - extract the result
        var result = data.result.result.value;
    }
};