Search code examples
javaparsingjsoupgoogle-search-api

Parsing Google search result Error


I reference the answer to parse the google search result.

How can you search Google Programmatically Java API

However ,when I try the code .Error occurs .

How should I make the modifications?

import java.net.URLDecoder;
import java.net.URLEncoder;

import org.jsoup.Jsoup;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements ;

public class JavaApplication22 {
public static void main(String[] args) {
   String google = "http://www.google.com/search?q=";
   String search = "stackoverflow";
   String charset = "UTF-8";
    String userAgent = "ExampleBot 1.0 (+http://example.com/bot)"; // Change this to your company's name and bot homepage!

    Elements links = Jsoup.connect(google + URLEncoder.encode(search, charset)).userAgent(userAgent).get().select(".g>.r>a");

    for (Element link : links) {
        String title = link.text();
        String url = link.absUrl("href"); // Google returns URLs in format "http://www.google.com/url?q=<url>&sa=U&ei=<someKey>".
        url = URLDecoder.decode(url.substring(url.indexOf('=') + 1, url.indexOf('&')), "UTF-8");

        if (!url.startsWith("http")) {
            continue; // Ads/news/etc.
        }



        System.out.println("Title: " + title);
        System.out.println("URL: " + url);
    }
}

}

I guess it is because the libraries matters. But I tried ctrl +shift+i .It shows that nothing to fix in import statements.

Error

Exception in thread "main" java.lang.RuntimeException: Uncompilable
source code - unreported exception java.io.IOException; must be caught
or declared to be thrown    at
javaapplication22.JavaApplication22.main(JavaApplication22.java:32)

How should I modify the code so that I can parse the Google Search result ?


Solution

  • Please replace your main class with below code :

    public static void main(String[] args) throws UnsupportedEncodingException, IOException {
    
        String google = "http://www.google.com/search?q=";
    
        String search = "stackoverflow";
    
        String charset = "UTF-8";
    
        String userAgent = "ExampleBot 1.0 (+http://example.com/bot)"; // Change this to your company's name and bot homepage!
    
        Elements links = Jsoup.connect(google + URLEncoder.encode(search, charset)).userAgent(userAgent).get().select(".g>.r>a");
    
        for (Element link : links) {
            String title = link.text();
            String url = link.absUrl("href"); // Google returns URLs in format "http://www.google.com/url?q=<url>&sa=U&ei=<someKey>".
            url = URLDecoder.decode(url.substring(url.indexOf('=') + 1, url.indexOf('&')), "UTF-8");
    
            if (!url.startsWith("http")) {
                continue; // Ads/news/etc.
            }
            System.out.println("Title: " + title);
            System.out.println("URL: " + url);
        }
    }