Search code examples
rubypostrest-client

How to get webpage body using ruby after passing form data


Basically to get the body of webpage i used to do this.

require "rest-client"
url="example.com"
test=RestClient.get(url)

But now the webpage has setup an authentication step. On entering the correct password you go to PAGE A else you go to PAGE B. My question is how do i login using the id and password. You can think of the page as this one here

I enter the credentials using the script and find out if I logged in or not.

Edit: Simplifying I want to know how do i pass the user name and password and I want to know the URL of the page that opened after login button is clicked.

Please note that this is not at all a hijack attempt. I just want to know if this can be done or not. The website i gave is just an example.


Solution

  • You can do it in fact, but I would say this solution isn't very correct.

    On the link page you've provided you can find form tag with an action field action="/registration/chooseAuth.do;jsessionid=73f8323730d5704b00f173314aaeaad2de767c0c0c42.e34Nb38TbhaTbO0Tc34Sbx0Obhv0n6jAmljGr5XDqQLvpAe"

    This is the link to actual authorization, so you can do something like that:

    RestClient.post(url_from_the_form, { login: 'foo', password: 'bar' })
    

    And it might work, but. What you do is called web scraping or web crawling. There are tools which are more suitable for that:

    Mechanize (the simplest one) Capybara + Selenium WebDriver

    Mechanize works great for rather simple webpages. When it comes to more AJAX or SPA, you better chose Capybara + Selenium.