Search code examples
javanlpsearch-enginebing-api

How to programmatically retrieve web pages with a query in java


what I need is to submit a query to the search engine and to retrieve the first 100 web pages to deal with their contents later on. Submitting the query to the search engine and dealing with the web pages resulted from the seach engine should be done within the programme. should I use search-engine api for example google-api or bing-api or there is another way?


Solution

  • To summarize what has been said in comments: yes, search engine API is the best way.

    Retrieving results as a web page and then parsing it is the worst approach imaginable(if you don't have any certain needs that make using API unbearable).