Search code examples
rubyauthenticationmechanize

How to login and crawl a site using Mechanize

I'm trying to use Mechanize to login and crawl a site.

For some reason, I can't seem to get the login function to work. Any ideas?

This is my code: 

require 'nokogiri'
require 'open-uri'
require 'mechanize'

a = Mechanize.new
a.get('https://jackthreads.com/')

form = a.page.form_with(:class => 'jt-form')
form.field_with(:name => "email").value = "email"
form.field_with(:name => "password21").value = "password"
page = a.submit(form, form.buttons.first)
Solution

  • The action on the form is set to "#", so your submit is being ignored. The POST call is actually being made to https://www.jackthreads.com/login?method=ajax via AJAX. Perhaps if you update the form's action attribute with Mechanize before submitting, it will do the trick.

    For what it's worth, I figured this out with the Chrome Web Inspector. After seeing the value was set to "#", I went to the network tab, filtered by XHR, then tried submitting something.