WWW::Mechanize and yellowpages.com

Question

I'm trying to search yellowpages.com via the Perl module WWW::Mechanize.

$mech->get( "http://www.yellowpages.com" );
$mech->form_name( "standard-searchform" );
$mech->field( "search-terms, "schneider" );
$mech->field( "search-location", "CA" );
$mech->submit();

I also tried $mech->submit_form( ... ) with the button value/type, but I get the following message all the time:

Error POSTing http://www.yellowpages.com/real_deals: Internal Server Error at /usr/lib/cgi-bin/index.pl line 39

Line 39 is

$mech->submit();

Is yp.com forwarding Mechanize to that site? How can I avoid that?

Answer 1

First you've missed a " after search-terms. Looking at the source code of yellowpages, there is no form with name "standard-searchform". The form is with an id "searchform-form". So that example should work:

my $mech = WWW::Mechanize->new;

$mech->get( "http://www.yellowpages.com" );
$mech->form_id( "searchform-form" );
$mech->field( "search-terms", "schneider" );
$mech->field( "search-location", "CA" );
$mech->submit();

EDIT:

also the search-terms and search-location are the input ids, where the documentation of the WWW::Mechanize says:

Given the name of a field, set its value to the value specified

That means you should change them with: search_terms and geo_location_terms.