Search code examples
javaweb-crawlerimdbhttp-accept-language

set the Accept-Language header for crawler in java


I'd like to find the correct method to set the Accept-Language header for my crawler? I read other related answers like Getting imdb movie titles in a specific language and How to set Accept-Language header on request from applet but they didn't work for me (I get this error: "the method is undefined for type connection" Here is part of code:

String baseUrl = "http://www.imdb.com/search/title?at=0&count=250";

org.jsoup.Connection con = Jsoup.connect(baseUrl).userAgent("Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.21 (KHTML, like Gecko) Chrome/19.0.1042.0 Safari/535.21");

Please help me, I am really new to java.

Thanks


Solution

  • In JSoup, you use the header method to set request headers. So the last line of your code will become this. I've just added line breaks for readability.

    org.jsoup.Connection con = Jsoup
         .connect(baseUrl)
         .userAgent("Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.21 (KHTML, like Gecko) Chrome/19.0.1042.0 Safari/535.21")
         .header("Accept-Language", /* Put your language here */);
    

    For example, to accept English, you'd write "en" in place of that last comment.