com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException: 404 Not Found for https://tax.ocgov.com/includes/jquery-2.2.3.min.js

Question

I am trying to look for valid Parcel Numbers on https://tax.ocgov.com/tcweb/search_page.asp via HtmlUnit, so I used the following for loop:

for (Integer pID=34056101; pID<34056111; pID++) {
    HtmlPage page = webClient.getPage("https://tax.ocgov.com/tcweb/search_page.asp");
    String i = pID.toString();
    final HtmlForm form = page.getFormByName("searchparcel");
    final HtmlTextInput taxIDInput = form.getInputByName("t_parcel_no");
    final HtmlImageInput button = form.getInputByName("s_parcel");
    taxIDInput.type(i);

    final HtmlPage page2 = (HtmlPage) button.click();

    //System.out.println(page2.getUrl().getPath());
    if (page2.getUrl().getPath() == "/tcweb/search_parcel.asp")
        {writer.println(pID);}
    }

However, I got this error:

com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException: 404 Not Found for https://tax.ocgov.com/includes/jquery-2.2.3.min.js

The exception is caused by this line of code:

final HtmlPage page2 = (HtmlPage) button.click();

What should I do to make my program work?

Answer 1

The page you are working with has a lot of problems - simply open the browser console and have a look yourself....

Out of the box HtmlUnit is written for page testing, because of this errors like this are handled more sensitive compared to real browsers. But you can change this.

For me this code works:

final String url = "https://tax.ocgov.com/tcweb/search_page.asp";

try (final WebClient webClient = new WebClient()) {
    webClient.getOptions().setThrowExceptionOnScriptError(false);
    webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);

    HtmlPage page = webClient.getPage(url);
    webClient.waitForBackgroundJavaScript(1000);

    final HtmlForm form = page.getFormByName("searchparcel");
    final HtmlTextInput taxIDInput = form.getInputByName("t_parcel_no");
    final HtmlImageInput button = form.getInputByName("s_parcel");
    taxIDInput.type("34056101");

    final HtmlPage page2 = (HtmlPage) button.click();
    webClient.waitForBackgroundJavaScript(1000);

    System.out.println("***********");
    System.out.println(page2.asNormalizedText());
    System.out.println("***********");

}