Search code examples
perldownloadwww-mechanize

Trouble with downloading files


I am trying to download a file from a site using perl. I chose not to use wget so that I can learn how to do it this way. I am not sure if my page is not connecting or if something is wrong in my syntax somewhere. Also what is the best way to check if you are getting a connection to the page.

#!/usr/bin/perl -w
use strict;
use LWP;
use WWW::Mechanize;

my $mech = WWW::Mechanize->new();
$mech->credentials( '********' , '********'); # if you do need to supply server and realms use credentials like in [LWP doc][2]
$mech->get('http://datawww2.wxc.com/kml/echo/MESH_Max_180min/');
$mech->success();
if (!$mech->success()) {
    print "cannot connect to page\n";
    exit;
}
$mech->follow_link( n => 8);
$mech->save_content('C:/Users/********/Desktop/');

Solution

  • I'm sorry but almost everything is wrong.

    • You use a mix of LWP::UserAgent and WWW::Mechanize in a wrong way. You can't do $mech->follow_link() if you use $browser->get() as you mix function from 2 module. $mech don't know that you did a request.
    • Arguments to credentials are not good, see the doc

    You more probably want to do something like this:

    use WWW::Mechanize;
    my $mech = WWW::Mechanize->new();
    
    $mech->credentials( '************' , '*************'); # if you do need to supply server and realms use credentials like in LWP doc
    $mech->get('http://datawww2.wxc.com/kml/echo/MESH_Max_180min/');
    $mech->follow_link( n => 8);
    

    You can check result of get() and follow_link() by checking $mech->success() result if (!$mech->success()) { warn "error"; ... }
    After follow->link, data is available using $mech->content(), if you want to save it in a file use $mech->save_content('/path/to/a/file')

    A full code could be :

    use strict;
    use WWW::Mechanize;
    my $mech = WWW::Mechanize->new();
    
    $mech->credentials( '************' , '*************'); #
    $mech->get('http://datawww2.wxc.com/kml/echo/MESH_Max_180min/');
    die "Error: failled to load the web page" if (!$mech->success());
    $mech->follow_link( n => 8);
    die "Error: failled to download content" if (!$mech->success());
    $mech->save_content('/tmp/mydownloadedfile')