I want to make scrapy return a specific webpage with a table so that it can then "click into" each entry in that table to go to a new webpage and scrape data from there. The webpage that I want it to begin from is "https://www.fifaindex.com/players/?gender=0&league=13&order=desc".
In more specific terms I am after the spider to begin from the above URL, take the list of football (soccer) players, and follow the link to each individual players page where I can then scrape any data I want from that (although specifically I am after the name and weight data).
My code is cobbled together from having read the tutorial on the scrapy website and also other people's code who have come up on google searches for the issue I'm having (however, you'll have to forgive me as I can't remember exactly where the other bits of code came from). As mentioned above the spider will iterate through every page but not actually stop and take any data. A bit like going to the shops with a list and saying "yep, it's all there" without actually buying any of the items.
The code in question is:
class Test2Spider(scrapy.Spider):
name = "test2"
def start_requests(self):
home_url = "https://www.fifaindex.com/players/?gender=0&league=13&order=desc"
yield scrapy.Request(home_url, self.parse)
def parse(self, response):
next_page = response.css("li.ml-auto a::attr(href)").get()
print(next_page)
player_page = response.css("figure.player a::attr(href)").getall()
if player_page is not None:
for x in range(0, (len(player_page)-1)):
new_page = player_page[x]
yield response.follow(new_page, self.parse)
#name = response.css("h5.card-header::text").get()
#weight = response.css("span.data-units.data-units-metric::text").get()
#print(name)
for guy in response.css("div.card mb-5"):
yield {
"name": guy.css("h5.card-header::text").get(),
"weight": guy.css("span.data-units data-units-metric::text").get(),
}
if next_page is not None:
yield response.follow(next_page, self.parse)
When this code runs it will iterate through every individual page and correctly scrape and follow the links. However, for each of those URLs, the thing that is returned by the spider is None. My intention with this code is to save the scraped data as a csv file, but I haven't added in any code to do that yet as I wanted to first make sure that it was actually scraping the data.
I have tried altering the html classes I draw from in case it was a case of me choosing one that is too specific or general. I have also tried re-jigging the order that things occur in but that didn't help. The #-ed bits are where I tried to remove the 2nd for loop in case I had messed the loop up, but even on their own the requests didn't return anything.
If anybody is able to shine any light on this I would appreciate it, I'm sure it's something relatively simple I'm missing, but I've been stuck on this for a while now and it's not becoming any clearer. I'm happy to answer any questions should it be required.
You are on the right track, but what you need to do is to take the logic for extracting the players name and weight and put it in it's own parsing method and then use that method as the callback to response.follow.
For example:
import scrapy
class Test2Spider(scrapy.Spider):
name = "test2"
def start_requests(self):
home_url = "https://www.fifaindex.com/players/?gender=0&league=13&order=desc"
yield scrapy.Request(home_url, self.parse)
def parse(self, response):
for player_page in response.css("figure.player a::attr(href)").getall():
yield response.follow(player_page, self.parse_player)
if next_page := response.css("li.ml-auto a::attr(href)").get():
yield response.follow(next_page, self.parse)
def parse_player(self, response):
yield {
"name": response.css("h5.card-header::text").get(),
"weight": response.css("span.data-units.data-units-metric::text").getall()[1],
}
Partial Output
2024-02-06 16:52:44 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/226753/andr%C3%A9-onana/> (referer: https://www.fifaindex.com/players/?gender=0&league=13&order=desc)
2024-02-06 16:52:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/237692/phil-foden/>
{'name': 'Phil Foden', 'weight': '69 kg'}
2024-02-06 16:52:44 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/241096/sandro-tonali/> (referer: https://www.fifaindex.com/players/?gender=0&league=13&order=desc)
2024-02-06 16:52:44 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/234378/declan-rice/> (referer: https://www.fifaindex.com/players/?gender=0&league=13&order=desc)
2024-02-06 16:52:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/224458/diogo-jota/>
{'name': 'Diogo Jota', 'weight': '68 kg'}
2024-02-06 16:52:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/209658/leon-goretzka/>
{'name': 'Leon Goretzka', 'weight': '82 kg'}
2024-02-06 16:52:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/206517/jack-grealish/>
{'name': 'Jack Grealish', 'weight': '81 kg'}
2024-02-06 16:52:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/209989/thomas-partey/>
{'name': 'Thomas Partey', 'weight': '77 kg'}
2024-02-06 16:52:44 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/247851/bruno-guimar%C3%A3es/> (referer: https://www.fifaindex.com/players/?gender=0&league=13&order=desc)
2024-02-06 16:52:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/226753/andr%C3%A9-onana/>
{'name': 'André Onana', 'weight': '93 kg'}
2024-02-06 16:52:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/241096/sandro-tonali/>
{'name': 'Sandro Tonali', 'weight': '79 kg'}
2024-02-06 16:52:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/234378/declan-rice/>
{'name': 'Declan Rice', 'weight': '80 kg'}
2024-02-06 16:52:44 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/216267/andrew-robertson/> (referer: https://www.fifaindex.com/players/?gender=0&league=13&order=desc)
2024-02-06 16:52:44 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/231281/trent-alexander-arnold/> (referer: https://www.fifaindex.com/players/?gender=0&league=13&order=desc)
2024-02-06 16:52:44 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/232411/christopher-nkunku/> (referer: https://www.fifaindex.com/players/?gender=0&league=13&order=desc)
2024-02-06 16:52:44 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/246669/bukayo-saka/> (referer: https://www.fifaindex.com/players/?gender=0&league=13&order=desc)
2024-02-06 16:52:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/247851/bruno-guimar%C3%A3es/>
{'name': 'Bruno Guimarães', 'weight': '74 kg'}
2024-02-06 16:52:44 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/200104/heung-min-son/> (referer: https://www.fifaindex.com/players/?gender=0&league=13&order=desc)
2024-02-06 16:52:44 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/210257/ederson/> (referer: https://www.fifaindex.com/players/?gender=0&league=13&order=desc)
2024-02-06 16:52:44 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/222665/martin-%C3%B8degaard/> (referer: https://www.fifaindex.com/players/?gender=0&league=13&order=desc)
2024-02-06 16:52:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/216267/andrew-robertson/>
{'name': 'Andrew Robertson', 'weight': '64 kg'}
2024-02-06 16:52:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/231281/trent-alexander-arnold/>
{'name': 'Trent Alexander-Arnold', 'weight': '69 kg'}
2024-02-06 16:52:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/232411/christopher-nkunku/>
{'name': 'Christopher Nkunku', 'weight': '73 kg'}
2024-02-06 16:52:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/246669/bukayo-saka/>
{'name': 'Bukayo Saka', 'weight': '65 kg'}
2024-02-06 16:52:44 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/212198/bruno-fernandes/> (referer: https://www.fifaindex.com/players/?gender=0&league=13&order=desc)
2024-02-06 16:52:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/200104/heung-min-son/>
{'name': 'Heung Min Son', 'weight': '78 kg'}
2024-02-06 16:52:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/210257/ederson/>
{'name': 'Ederson', 'weight': '86 kg'}
2024-02-06 16:52:44 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/200145/casemiro/> (referer: https://www.fifaindex.com/players/?gender=0&league=13&order=desc)
2024-02-06 16:52:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/222665/martin-%C3%B8degaard/>
{'name': 'Martin Ødegaard', 'weight': '68 kg'}
2024-02-06 16:52:44 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/212831/alisson/> (referer: https://www.fifaindex.com/players/?gender=0&league=13&order=desc)
2024-02-06 16:52:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/212198/bruno-fernandes/>
{'name': 'Bruno Fernandes', 'weight': '69 kg'}
2024-02-06 16:52:44 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/231866/rodri/> (referer: https://www.fifaindex.com/players/?gender=0&league=13&order=desc)
2024-02-06 16:52:44 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/239818/r%C3%BAben-dias/> (referer: https://www.fifaindex.com/players/?gender=0&league=13&order=desc)
2024-02-06 16:52:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/200145/casemiro/>
{'name': 'Casemiro', 'weight': '84 kg'}
2024-02-06 16:52:44 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/251517/jo%C5%A1ko-gvardiol/> (referer: https://www.fifaindex.com/players/?page=2&gender=0&league=13&order=des
c)
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/players/?page=3&gender=0&league=13&order=desc> (referer: https://www.fifaindex.com/players/?page=2&gender=0&league=1
3&order=desc)
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/212831/alisson/>
{'name': 'Alisson', 'weight': '91 kg'}
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/231866/rodri/>
{'name': 'Rodri', 'weight': '82 kg'}
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/167948/hugo-lloris/> (referer: https://www.fifaindex.com/players/?page=2&gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/239837/alexis-mac-allister/> (referer: https://www.fifaindex.com/players/?page=2&gender=0&league=13&order=des
c)
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/239818/r%C3%BAben-dias/>
{'name': 'Rúben Dias', 'weight': '82 kg'}
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/218667/bernardo-silva/> (referer: https://www.fifaindex.com/players/?gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/203376/virgil-van-dijk/> (referer: https://www.fifaindex.com/players/?gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/251517/jo%C5%A1ko-gvardiol/>
{'name': 'Joško Gvardiol', 'weight': '80 kg'}
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/240091/guglielmo-vicario/> (referer: https://www.fifaindex.com/players/?page=2&gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/209331/mohamed-salah/> (referer: https://www.fifaindex.com/players/?gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/205988/luke-shaw/> (referer: https://www.fifaindex.com/players/?page=2&gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/167948/hugo-lloris/>
{'name': 'Hugo Lloris', 'weight': '82 kg'}
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/242516/cody-gakpo/> (referer: https://www.fifaindex.com/players/?page=2&gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/255475/antony/> (referer: https://www.fifaindex.com/players/?page=2&gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/253072/darwin-n%C3%BA%C3%B1ez/> (referer: https://www.fifaindex.com/players/?page=2&gender=0&league=13&order=
desc)
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/239837/alexis-mac-allister/>
{'name': 'Alexis Mac Allister', 'weight': '72 kg'}
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/218667/bernardo-silva/>
{'name': 'Bernardo Silva', 'weight': '64 kg'}
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/203376/virgil-van-dijk/>
{'name': 'Virgil van Dijk', 'weight': '92 kg'}
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/240091/guglielmo-vicario/>
{'name': 'Guglielmo Vicario', 'weight': '76 kg'}
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/209331/mohamed-salah/>
{'name': 'Mohamed Salah', 'weight': '71 kg'}
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/205988/luke-shaw/>
{'name': 'Luke Shaw', 'weight': '75 kg'}
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/190460/christian-eriksen/> (referer: https://www.fifaindex.com/players/?page=2&gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/202652/raheem-sterling/> (referer: https://www.fifaindex.com/players/?page=2&gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/242516/cody-gakpo/>
{'name': 'Cody Gakpo', 'weight': '76 kg'}
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/255475/antony/>
{'name': 'Antony', 'weight': '63 kg'}
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/205498/jorginho/> (referer: https://www.fifaindex.com/players/?page=2&gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/253072/darwin-n%C3%BA%C3%B1ez/>
{'name': 'Darwin Núñez', 'weight': '80 kg'}
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/229391/palhinha/> (referer: https://www.fifaindex.com/players/?page=2&gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/241464/pau-torres/> (referer: https://www.fifaindex.com/players/?page=2&gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/190460/christian-eriksen/>
{'name': 'Christian Eriksen', 'weight': '76 kg'}
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/247090/enzo-fern%C3%A1ndez/> (referer: https://www.fifaindex.com/players/?page=2&gender=0&league=13&order=des
c)
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/188377/kyle-walker/> (referer: https://www.fifaindex.com/players/?page=2&gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/202652/raheem-sterling/>
{'name': 'Raheem Sterling', 'weight': '69 kg'}
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/164240/thiago-silva/> (referer: https://www.fifaindex.com/players/?page=2&gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/205498/jorginho/>
{'name': 'Jorginho', 'weight': '68 kg'}
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/229391/palhinha/>
{'name': 'Palhinha', 'weight': '85 kg'}
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/219693/diego-carlos/> (referer: https://www.fifaindex.com/players/?page=3&gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/241464/pau-torres/>
{'name': 'Pau Torres', 'weight': '80 kg'}
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/247090/enzo-fern%C3%A1ndez/>
{'name': 'Enzo Fernández', 'weight': '76 kg'}
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/188377/kyle-walker/>
{'name': 'Kyle Walker', 'weight': '83 kg'}
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/164240/thiago-silva/>
{'name': 'Thiago Silva', 'weight': '79 kg'}
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/players/?page=4&gender=0&league=13&order=desc> (referer: https://www.fifaindex.com/players/?page=3&gender=0&league=1
3&order=desc)
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/208920/nathan-ak%C3%A9/> (referer: https://www.fifaindex.com/players/?page=3&gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/219693/diego-carlos/>
{'name': 'Diego Carlos', 'weight': '79 kg'}
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/211990/odisseas-vlachodimos/> (referer: https://www.fifaindex.com/players/?page=3&gender=0&league=13&order=de
sc)
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/226226/giovani-lo-celso/> (referer: https://www.fifaindex.com/players/?page=3&gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/189509/thiago/> (referer: https://www.fifaindex.com/players/?page=2&gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/208920/nathan-ak%C3%A9/>
{'name': 'Nathan Aké', 'weight': '75 kg'}
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/229984/ben-chilwell/> (referer: https://www.fifaindex.com/players/?page=3&gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/211990/odisseas-vlachodimos/>
{'name': 'Odisseas Vlachodimos', 'weight': '81 kg'}
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/203841/nick-pope/> (referer: https://www.fifaindex.com/players/?page=2&gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/236276/arnaut-danjuma/> (referer: https://www.fifaindex.com/players/?page=3&gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/226226/giovani-lo-celso/>
{'name': 'Giovani Lo Celso', 'weight': '68 kg'}
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/230977/miguel-almir%C3%B3n/> (referer: https://www.fifaindex.com/players/?page=3&gender=0&league=13&order=des
c)
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/233064/mason-mount/> (referer: https://www.fifaindex.com/players/?page=3&gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/213648/pierre-emile-h%C3%B8jbjerg/> (referer: https://www.fifaindex.com/players/?page=2&gender=0&league=13&or
der=desc)
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/189509/thiago/>
{'name': 'Thiago', 'weight': '70 kg'}
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/229984/ben-chilwell/>
{'name': 'Ben Chilwell', 'weight': '77 kg'}
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/235173/ibrahim-sangar%C3%A9/> (referer: https://www.fifaindex.com/players/?page=3&gender=0&league=13&order=de
sc)
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/233927/lucas-paquet%C3%A1/> (referer: https://www.fifaindex.com/players/?page=3&gender=0&league=13&order=desc
)
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/203841/nick-pope/>
{'name': 'Nick Pope', 'weight': '76 kg'}
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/236276/arnaut-danjuma/>
{'name': 'Arnaut Danjuma', 'weight': '74 kg'}
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/230977/miguel-almir%C3%B3n/>
{'name': 'Miguel Almirón', 'weight': '70 kg'}
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/235844/edson-%C3%A1lvarez/> (referer: https://www.fifaindex.com/players/?page=3&gender=0&league=13&order=desc
)
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/233064/mason-mount/>
{'name': 'Mason Mount', 'weight': '74 kg'}
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/213648/pierre-emile-h%C3%B8jbjerg/>
{'name': 'Pierre-Emile Højbjerg', 'weight': '84 kg'}
2024-02-06 16:52:45 [scrapy.crawler] INFO: Received SIGINT, shutting down gracefully. Send again to force
2024-02-06 16:52:45 [scrapy.core.engine] INFO: Closing spider (shutdown)
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/237678/ibrahima-konat%C3%A9/> (referer: https://www.fifaindex.com/players/?page=3&gender=0&league=13&order=de
sc)
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/235173/ibrahim-sangar%C3%A9/>
{'name': 'Ibrahim Sangaré', 'weight': '77 kg'}
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/243576/pedro-porro/> (referer: https://www.fifaindex.com/players/?page=3&gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/233927/lucas-paquet%C3%A1/>
{'name': 'Lucas Paquetá', 'weight': '72 kg'}
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/251892/beto/> (referer: https://www.fifaindex.com/players/?page=3&gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/263578/balde/> (referer: https://www.fifaindex.com/players/?page=3&gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/235844/edson-%C3%A1lvarez/>
{'name': 'Edson Álvarez', 'weight': '73 kg'}
2024-02-06 16:52:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.fifaindex.com/player/196978/callum-wilson/> (referer: https://www.fifaindex.com/players/?page=3&gender=0&league=13&order=desc)
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/237678/ibrahima-konat%C3%A9/>
{'name': 'Ibrahima Konaté', 'weight': '95 kg'}
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/243576/pedro-porro/>
{'name': 'Pedro Porro', 'weight': '69 kg'}
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/251892/beto/>
{'name': 'Beto', 'weight': '88 kg'}
2024-02-06 16:52:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/263578/balde/>
{'name': 'Balde', 'weight': '69 kg'}
2024-02-06 16:52:46 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.fifaindex.com/player/196978/callum-wilson/>
{'name': 'Callum Wilson', 'weight': '66 kg'}