Search code examples
web-scrapingscrapyscrapinghub

Login to a website then collect data with Scraping Hub


I've used scrapinghub for two days and am looking for how to log in into a website then scrape data. I see this topic but can't see how to apply it into the Dash.

http://blog.scrapinghub.com/2012/10/26/filling-login-forms-automatically/

Could you explain me how to achieve that?

When I was just only using Scrapy, I'll achieve it like this:

parsed = [FormRequest.from_response(
            response,
            formdata={
                'session[email]': 'email@gmail.com',
                'session[password]': 'password'
            },
            callback=self.after_login)]

But I did not see how to achieve it using Scraping Hub :/


Solution

  • Answer was more simple than expected.

    Perform login

    An other problem occur : on some website, authentication fail