Search code examples
javascreen-scrapingbots

How to code an automated bot that can browse and do operations on a webpage

I need to code a bot that needs to do the following:

Go to a jsp page and search for something by:

  • 1: writing something on a search box
  • 2: clicking the search button(submit button)
  • 3: clicking one of the the resulting buttons/links(same jsp page with different output)
  • 4: get the entire html of the new page(same jsp page with different output)

The 4th one can be done with screen scraping and I do not think I need help with it. But I need some guidance to do the options from 1 to 3. Any links or just some keyword that will help me google to learn about it will be appreciated. I plan to do this with java.

Solution

  • Maybe it's not what you want, but you can try selenium : http://seleniumhq.org/

    It's a web application testing system.