why does Jsoup selector div#id work in netbeans but not in eclipse ADT?

Question

Following code works fine in netbeans, but in eclipse ADT, doc.select("div#tickertape_eng a").size() returns 0.

try {
            Document doc = Jsoup.connect("http://bdnews24.com").get();
            if (doc != null) {
                Elements links = doc.select("div#tickertape_eng a");
                for (Element link : links) {
                    headlines.add(link.text());
                    headlinks.add(link.absUrl("href"));
                }

            } else {
                Toast.makeText(getActivity(), "Doc not found",
                        Toast.LENGTH_LONG).show();
            }
        } catch (IOException e) {
            e.printStackTrace();
        }

In netbeans, I can print link.text() and link.absUrl("href"), but not in eclipse. Eclipse prints doc.title() in logcat fine, but why does doc.select("div#tickertape_eng a") not work? How to make it work?

Answer 1

Try this:

Document doc = Jsoup.connect("http://bdnews24.com")
  .userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20100101 Firefox/5.0")
  .get();

The user agent is what let the server guess which browser you are using. If it detects a mobile browser, it can redirect you to a mobile page. So forcing a desktop browser user agent is a way to make sure you get the standard page located at bdnews24.com.

You can find most of the available user agents here: http://www.useragentstring.com/pages/useragentstring.php