Search code examples
perlweb-scrapingwww-mechanizelwp-useragent

Perl Web Navigation through URL


i'm trying to navigate through this site http://irl.worldfootball.net/ to get to player's pages. I want to be able to just take the player name variables i have and insert them in the url to get to each player's respective page, but i can't quite figure it out. Can anyone help me?

As you can see from this link http://irl.worldfootball.net/spieler_profil/Wayne-Rooney/ the url is pretty simple, what i want to do is insert the player name of my choice where it says Wayne-Rooney and then navigate to that page. Any help would be greatly appreciated, thanks!


Solution

  • Basically it'll look like this:

    my $base_url = 'http://irl.worldfootball.net/spieler_profil/';
    my @players_data = ('Wayne Rooney', 'Lionel Messi', 'Thierry Henry');
    
    for (@players_data) {
      my $working_url = $base_url 
                      . ( join '-', split )
                      . '/';
      # processing $working_url now...
    }
    

    Processing itself may be done with either WWW::Mechanize (preferable, as for me, if you have some forms to fill and such) or just LWP::Simple (pretty simple to use even for a novice, yet gets the basic job done very well). I'd recommend using the latter if what you need is just collecting the data from pages with simple navigation. )