Search code examples
rubymechanizemechanize-ruby

Mechanize: Unable to redirect to final destination


I am using mechanize to try to login to a site. However when logging in with mechanize, I cant seem to get to final destination. Below is the example.

Mechanize go to (1) https://app.abc.com/users/login

If login successful, webpage will momentarily post to page link below

(2) https://paid.abc.com/Login.aspx?CID=123&XRALID=321&LanguageID=1&ExecuteLogin=1

From there automatically redirect to final destination

(3) https://paid.abc.com/ABC/ABCMainStrict.aspx?state=SearchQueriesStrict

My code is below:

require 'mechanize'

def say(msg)
  puts '- ' + msg
end

def exit_with(msg)
  say msg
  puts "...exiting..."
  exit
end

a = Mechanize.new

a.user_agent = "Friendly Mechanize Script"
a.redirect_ok = true

say "Logging in..."
a.get('https://app.abc.com/users/login') do |page|
  content_page = page.form_with(:action => "/users/login") do |f|
    username_field = f.field_with(name: "UserName")
    username_field.value = '[email protected]'
    password_field = f.field_with(name: "Password")
    password_field.value = 'Password123'
  end.submit

  say "Got page: " + content_page.title
  say "Show page: " + content_page.uri.to_s
  new_link = content_page.uri.to_s

  exit_with("Couldn't log in.") if content_page.uri.to_s =~ /login/
  puts a.page.inspect

end

Output of "a.page.inspect" is as below

#<Mechanize::Page
 {url
  #<URI::HTTPS:xxx URL:https://paid.abc.com/Login.aspx?CID=123&XRALID=321&LanguageID=1&ExecuteLogin=1>}
 {meta_refresh}
 {title "ABC RELATIONS"}
 {iframes}
 {frames}
 {links
  #<Mechanize::Page::Link
   "Internet Explorer 8.0 or higher"
   "http://www.microsoft.com/ie">
  #<Mechanize::Page::Link "Firefox 3.6 or higher" "http://www.getfirefox.com">
  #<Mechanize::Page::Link
   "Chrome 13.0 or higher"
   "http://www.google.com/chrome">
  #<Mechanize::Page::Link
   "Safari 5.0 or higher"
   "http://www.apple.com/safari/">}
 {forms}>

UPDATE

Thanks to @JonB, I realized that what I am trying to do is called follow meta refresh. I think this code below is helpful to trouble this problem.

<head>\r\n    
<meta http-equiv=\"Content-Type\" content=\"text/html; charset=UTF-8\" />\r\n    
<title>ABC RELATIONS</title>\r\n    
<script src=\"Login.js\" type=\"text/javascript\" />\r\n    
<link rel=\"shortcut icon\" href=\"favicon.ico\" type=\"image/x-icon\" />\r\n    
<script type=\"text/javascript\" src=\"//use.typekit.net/qsx7ypf.js\" />\r\n    
<script type=\"text/javascript\">try{Typekit.load();}catch(e){}</script>\r\n    
<style type=\"text/css\">\r\n\t\t\t\t\t.clearfix {display: inline-block;}  /* for IE/Mac */\r\n\t\t\t\t</style>\r\n    
<link href=\"Images/Login/styles4.css\" rel=\"stylesheet\" type=\"text/css\" />\r\n  
<script type=\"text/javascript\">var autoLogin = false;</script>
</head>

Solution

  • Looks like there is a meta refresh in there (per your description). Try adding this to your Mechanize object:

    a.follow_meta_refresh = true
    

    Also, you may want your user_agent to an accepted value instead of your custom one:

    require 'mechanize'
    Mechanize::AGENT_ALIASES.each { |k,v| puts k }
    => Mechanize
    => Linux Firefox
    => Linux Konqueror
    => Linux Mozilla
    => Mac Firefox
    => Mac Mozilla
    => Mac Safari 4
    => Mac Safari
    => Windows Chrome
    => Windows IE 6
    => Windows IE 7
    => Windows IE 8
    => Windows IE 9
    => Windows Mozilla
    => iPhone
    => iPad
    => Android
    => Mac FireFox
    => Linux FireFox