Search code examples
node.jsrequesthrefcheeriosteam

Why I get wrong link from attribute href from site steam?


I use nodejs for get all links from site steam.

In site links (for example):

  1. https://steamcommunity.com/market/listings/730/Horizon%20Case
  2. https://steamcommunity.com/market/listings/730/Chroma%203%20Case
  3. etc.

I get:

  1. https://steamcommunity.com/market/listings/730/Horizon%20Case (good)
  2. https://steamcommunity.com/market/listings/730/Chroma%20Case (wrong)

My code:

var link_part1 = 'https://steamcommunity.com/market/search?category_730_ItemSet%5B%5D=any&category_730_ProPlayer%5B%5D=any&category_730_StickerCapsule%5B%5D=any&category_730_TournamentTeam%5B%5D=any&category_730_Weapon%5B%5D=any&category_730_Type%5B%5D=tag_CSGO_Type_WeaponCase&appid=730&q=case#p1'
var link_part2 = '_price_asc'
var number_page = 1

request(link_part1+number_page+link_part2, function(err, resp, html) {
        if (!err){
          const $ = cheerio.load(html);
          for (i=0; i<10; i++) {
            let box = $("a.market_listing_row_link").eq(i).attr('href')
            console.log(box);
          }
        } else {
          console.log("Error");
        }
});

What I do wrong???


Solution

  • This happens because initially this information is not in the page code. It is loaded in the process. If you use the puppeteer library, you can easily extract this information.