Search code examples
testingwebdriverhtmlunitxvfb

What is the difference between using HtmlUnitDriver and writing headless tests using Xvfb in linux?

I am a novice in testing.

I am working on Linux. I was reading about testing in headless mode and came across two things. One was X virtual frame buffer which does graphical operations in memory. So, no output is displayed. The implementation details I found in this link http://www.seleniumtests.com/2012/04/headless-tests-with-firefox-webdriver.html.

The other one that I came across was HtmlUnitDriver. This also does not open any browser while running the test. I wrote a basic sample code using HtmlUnitDriver and the assertions seem to work fine.

I understand that HtmlUnitDriver doesn't work too well with javascript. But apart from this, are there any major differences to choose one over the other?

I am going to be testing a web application that does have some abount of javascript in it.

I am a novice in this field. So, any answers, suggestions, etc. will be appreciated. Thank you in advance

Solution

  • From my experience with both approaches:

    • HtmlUnit will in most practical cases be faster than a real browser with xvfb -- simply because it doesn't spend time rendering the pages. (A data point: 17 secs. HtmlDriver vs. 62 secs. FirefoxDriver for a specific test suite I'm using now).
    • It is easier to run several tests concurrently -- and it consumes a lot less resources -- using HtmlUnit. This can be very important if you have a large number of tests and you need them to finish fast (e.g. you want to follow the 10-minute-build rule).
    • As you said, HtmlUnit has its own quirks with JavaScript and the DOM. Not better or worse than any other browser (Firefox, Safari, IE, Chrome, ... -- they all have their own quirks), but one on which it is very questionable to spend time fixing bugs. I also find such bugs very difficult to diagnose, but that may be only my ignorance.
    • One advantage of real browsers + xvfb is that you can always use the exact same tests without xvfb and see what's going on -- possibly even use a console to run some JavaScript to diagnose issues. I sometimes feel quite blind when working with HtmlUnit, and because of above-mentioned quirks you can't always use the same exact tests code in both environments.

    So, in summary, unless total test duration is important and you're ready to spend some time fighting HtmlUnit, it's just easier to go with a regular browser + xvfb.

    I also like using xvnc, which has the added benefit of allowing you to connect to the screen of a running test and see what's going on (not sure whether you can do that with xvfb).