Search code examples
pythonscreen-scraping

Python WWW macro

i need something like iMacros for Python. It would be great to have something like that:

browse_to('www.google.com')
type_in_input('search', 'query')
click_button('search')
list = get_all('<p>')

Do you know something like that?

Thanks in advance, Etam.

Solution

  • Almost a direct fulfillment of the wishes in the question - twill.

    twill is a simple language that allows users to browse the Web from a command-line interface. With twill, you can navigate through Web sites that use forms, cookies, and most standard Web features.

    twill supports automated Web testing and has a simple Python interface.

    (pyparsing, mechanize, and BeautifulSoup are included with twill for convenience.)

    A Python API example:

    from twill.commands import go, showforms, formclear, fv, submit

go('http://issola.caltech.edu/~t/qwsgi/qwsgi-demo.cgi/')
go('./widgets')
showforms()

formclear('1')
fv("1", "name", "test")
fv("1", "password", "testpass")
fv("1", "confirm", "yes")
showforms()

submit('0')