Download web page content using Selenium Webdriver and HtmlUnit

Question

For the following versions (JDK:8,JRE:8,HtmlUnit:2.17 and Selenium Webdriver:2.46), my this code works perfectly fine. When I say fine it means I am able to download the full web content (each and every line).

package  mypackage;
import org.openqa.selenium.By;      
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.htmlunit.HtmlUnitDriver;     
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

import com.gargoylesoftware.htmlunit.BrowserVersion;
public class HtmlUnitTest {             
public static void main(String[] args) {
WebDriver driver = new FirefoxDriver(); // Working fine
String baseUrl = "h t t p s : / / fourseasons . wd3 . myworkdayjobs . com / search / jobs";
driver.get(baseUrl);
WebDriverWait myWaitVar = new WebDriverWait(driver, 20);
try{
    myWaitVar.until(ExpectedConditions.visibilityOfElementLocated(By.id("wd-FacetedSearchResult-ResultsPnl-facetSearchResult")));
}catch(Exception ex){
    ex.printStackTrace();
}
String content=driver.getPageSource();
System.out.println(content);
driver.close();         
}      

}

But when i change the driver to HtmlUnitDriver, it doesn't download the full content.

WebDriver driver = new HtmlUnitDriver();

What i tried then:

1. WebDriver driver = new HtmlUnitDriver(BrowserVersion.FIREFOX_38);
2. WebDriver driver = new HtmlUnitDriver(BrowserVersion.CHROME);
3. WebDriver driver = new HtmlUnitDriver(BrowserVersion.INTERNET_EXPLORER_11);

Nothing works. What additional things I can add please suggest.

Answer 1

I got the answer guys. I enabled the javascript and it worked. So the new code is like:

public class HtmlUnitTest {             
    public static void main(String[] args) {
        HtmlUnitDriver driver = new HtmlUnitDriver(BrowserVersion.FIREFOX_38);
        driver.setJavascriptEnabled(true);
        String baseUrl = "h t t p s : / / fourseasons . wd3 . myworkdayjobs . com / search / jobs";
        driver.get(baseUrl);
        WebDriverWait myWaitVar = new WebDriverWait(driver, 20);
        try{
            myWaitVar.until(ExpectedConditions.visibilityOfElementLocated(By.id("wd-FacetedSearchResult-ResultsPnl-facetSearchResult")));
        }catch(Exception ex){
            ex.printStackTrace();
        }
        String content=driver.getPageSource();
        System.out.println(content);
        driver.close();         
    }      
}

Thanks all.