Search code examples
pythonmacosmechanize-python

Mechanize submit a form that creates a dynamic flie that needs to be downloaded


So far I have mechanize code that does this:

goes to a site
logs in
submits a form

heres where i hit problems. What I need it to do is to write the response (a file) to a local file. I am pretty clueless as far as python interacting with the file system.

Thanks in advance

EDIT: Here is some of the code i currently have

br = mechanize.Browser()
br.set_handle_robots(False)
br.set_handle_redirect(True)
br.set_handle_refresh(mechanize._http.HTTPRefreshProcessor(), max_time=1000)
br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')]

formcount=0
for frm in br.forms():  
  if str(frm.attrs["id"])=="id-of-form":
    break
  formcount=formcount+1
br.select_form(nr=formcount)

with open('a filename', 'wb') as f:
    shutil.copyfileobj(br.submit(name='submit', label='value of submit button'), f)

If it matters; I'm running mac OS X


Solution

  • The return value of submit is a file-like object. You can copy the data to a local file:

    import shutil
    with open('downloaded', 'wb') as f:
        shutil.copyfileobj(br.submit(), f)
    

    Unrelatedly, you can shorten the form selection bit like this:

    br.select_form(predicate=lambda form: form.attrs['id'] == 'id-of-form')
    

    Here's a full working example:

    import mechanize
    import shutil
    
    br = mechanize.Browser()
    br.open('http://stackoverflow.com/')
    br.select_form(predicate=lambda form: form.attrs.get('id') == 'search')
    br['q'] = '[python-mechanize]'
    with open('search results.html', 'wb') as f:
        shutil.copyfileobj(br.submit(), f)