The site that I'm trying to parse only has a single input box with no form. I'm having trouble just defining the single input box, passing it an address and then submitting.
What I'd like to do is input an address, submit, grab the info under id="A18" title="Click to get bulk trash pick up info" and load into JSON.
Python:
import mechanicalsoup
# URL that we authenticate against
map_url = "http://mapservices.phoenix.gov/gis/imap/iMap.html"
address = "<address>"
json_file = "/home/pi/bulk_pickup.json"
# Setup browser
browser = mechanicalsoup.StatefulBrowser(
soup_config={'features': 'lxml'},
user_agent='Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101206 Ubuntu/10.10 (maverick) Firefox/3.6.13',
)
# Open the login URL
map_page = browser.get(map_url)
# Similar to assert login_page.ok but with full status code in case of failure.
map_page.raise_for_status()
search_form = mechanicalsoup.Form(map_page.soup.select_one('input[id="search_input"]'))
search_form.input({'search_input': address})
Unfortunately, the page http://mapservices.phoenix.gov/gis/imap/iMap.html seems to do a heavy use of JavaScript. The <input ...> tag you're seeing is not even part of a <form>, and MechanicalSoup needs the action= attribute of the form to know where to submit it. Either you'll need to hack the low-level stuff yourself (but MechanicalSoup won't be very helpful compared to using the bare request library), or you'll need a more advanced solution like Selenium.
See http://mechanicalsoup.readthedocs.io/en/stable/faq.html#when-to-use-mechanicalsoup for more information.
Had the page been more "HTMLy" and less "JavaScripty", you could have written
browser.open(map_url)
browser.select_form(...)
browser["search_input"] = ...
browser.submit_selected()