Search code examples
htmldatabasegoogle-chromexpathscreen-scraping

data repetition in webscraping using xpath

I have a problem when im trying scraping data, when im looking an specific information in google chrome browser console, this repeat it seven times and it goes to the next, here is my code

$x('//div[@class="cs"]/div/text()').map(x=>x.wholeText)

this code gives me this

['CS 35 (1.4)', 'CS 35 (1.4)', 'CS 35 (1.4)', 'CS 35 (1.4)', 
'CS 35 (1.4)', 'CS 35 (1.4)', 'CS 35 (1.4)', 'CS 269 (7.3)', 
'CS 269 (7.3)', 'CS 269 (7.3)', 'CS 269 (7.3)', 'CS 269 (7.3)', 
'CS 269 (7.3)', 'CS 269 (7.3)', 'CS 137 (8.5)', 'CS 137 (8.5)'
 ....................
'CS 241 (7.5)', 'CS 241 (7.5)', 'CS 241 (7.5)', 'CS 226 (9.2)', 
'CS 226 (9.2)', …]

Just i want this one time CS 35 (1.4) and then this CS 269 (7.3) and so.. i dont want it so many times

this web page im scraping https://www.op.gg/summoners/kr/Hide%20on%20bush

I want a code that helps me solve the problem that I put above

Solution

  • Try changing your xpath expression to

    $x('//div[@class="stats"]//div[@class="cs"]').map(x=>x.innerText)

    Output should be:

    [ "CS 35 (1.4)", "CS 269 (7.3)", "CS 137 (8.5)", "CS 226 (6.8)", 
"CS 224 (7.7)", "CS 262 (8.7)", "CS 218 (8.8)", "CS 160 (5.6)", 
"CS 252 (9.9)", "CS 239 (7)", … ]