Search code examples
pythonweb-scrapingmechanize-python

Python transfer mechanize browser session

I'm having a little difficulty trying to navigate a website past the login screen. I've done this using mechanize. However once I navigate past the login page I want to interact with the page, click attributes, etc. which mechanize cannot do. I also want to do this all "behind the curtain" so the browser window is invisible (trying not to use selenium).

Here is the code I use to login. What can I do past this to start interacting with the page

import mechanize

br = mechanize.Browser()
#get computer browser

br.set_handle_robots(False)
#what robots?

br.open("www.website.com")
#open website

br.select_form(nr=0)
#get the main form    

br.set_all_readonly(False)

for control in br.form.controls:
    print control

user_control = br.form.controls[0]
user_control._value = 'username'

user_password = br.form.controls[1]
user_password._value = 'password'

br.submit()
Solution

  • One option would be to "transfer" cookies from mechanize to selenium and use selenium with a headless browser like PhantomJS or with a virtual display. Or, just switch to selenium+PhantomJS completely (including the authentication step).

    See also: