I am trying to make an automated system that performs multiple search and download the results. I need to first get authentication for the website, the direct to the search page, set parameters for search, make the html req which includes some values sent by post, get response as html to parse what to download and download it. Can you help me and redirect me to some samples to how to do such a thing? What is the best packages to use?
Best
When I need to parse a bunch of HTML for the key pieces of information and I'm lucky enough to be using Python, I always reach for BeautifulSoup first.