I'm trying to scrape multiple tables on basketball reference player webpages (using the rvest package in R) but have been running into some issues. Let's take this webpage as an example: https://www.basketball-reference.com/players/w/whitede01/gamelog/2023. I want to scrape the tables for "2022-23 Regular Season" and for "2023 Playoffs." For some reason, the same type of rvest code works for the first but not the second. Here's the code that shows this:
# First load packages
library(tidyverse)
library(rvest)
# This code works
"https://www.basketball-reference.com/players/w/whitede01/gamelog/2023" %>%
read_html() %>%
html_nodes(xpath = '//*[@id="content"]/div[5]') %>%
html_table(fill = TRUE, header = FALSE) %>%
.[[1]]
# This code doesn't work
"https://www.basketball-reference.com/players/w/whitede01/gamelog/2023" %>%
read_html() %>%
html_nodes(xpath = '//*[@id="content"]/div[6]') %>%
html_table(fill = TRUE, header = FALSE) %>%
.[[1]]
# And this xpath doesn't make it work eitehr:
'//*[@id="all_pgl_basic_playoffs"]'
Is there any way to make this scraping of the second (playoffs) table work? I appreciate any help.
The post shared by margusl is spot on. But I'll add a bit more in hopes it is useful for those of us not familiar with webdev and/or are newer to scraping.
So you are right to try to access the playoff table by its id: all_pgl_basic_playoffs. Ideally the xml path //*[@id= "all_pgl_basic_playoffs"] should work.
However, if you go to the page that the table is on on a mobile device or on a browser window that is really small, you'll notice that the playoff table doesn't automatically populate. Instead, you have to click to expand the table.
When you tell rvest to go to the page for Derrick Whites' stats, it creates a web browser session that is happening behind the scenes. rvest will not be able to interact with the page to expand and to populate that table.
Without that table being populated by interactively expanding it, that table technically doesn't exist. HTML doesn't really do interactive stuff like that. So that table won't be found in the HTML text. Instead, once you click to expand the table, it kicks off an action to run some JavaScript to then load the table's contents and then bring it into existence in the HTML code.
However, since rvest can't interact with the webpage to load the table and to generate the HTML code to display the table and it's contents, it won't know where to look in that behind the scenes web browser. In these cases, you usually have one option: you can use selenium (either in R or in Python) that let's you activate a browser session that you can interact with and then access the HTML code from that activated browser session.
Luckily, it looks like the table and its contents are commented out in the <div id= "all_pgl_playoffs" element! Comments in HTML are started with <!-- and close with -->. What we can do, is we can instead access that comment node instead and parse its contents to grab the table and the data that should be in the table. The table element still has the same id, so we can access it by the same id label.
Here is the comment for reference:
<!--
<div class="table_container" id="div_pgl_basic_playoffs">
<table class="row_summable sortable stats_table" id="pgl_basic_playoffs" data-cols-to-freeze=",3">
<caption>2023 Playoffs Table</caption>
<colgroup><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col></colgroup>
<thead>
<tr>
<th aria-label="Rk" data-stat="ranker" scope="col" class="ranker poptip sort_default_asc show_partial_when_sorting center" data-tip="Rank" >Rk</th>
<th aria-label="G" data-stat="game_season" scope="col" class=" poptip center" data-tip="Season Game" >G</th>
<th aria-label="Date" data-stat="date_game" scope="col" class=" poptip sort_default_asc center" >Date</th>
<th aria-label="Age" data-stat="age" scope="col" class=" poptip sort_default_asc center" data-tip="Player's age on February 1 of the season" >Age</th>
<th aria-label="Tm" data-stat="team_id" scope="col" class=" poptip sort_default_asc center" data-tip="Team" >Tm</th>
<th aria-label=" " data-stat="game_location" scope="col" class=" poptip center" > </th>
<th aria-label="Opp" data-stat="opp_id" scope="col" class=" poptip sort_default_asc center" data-tip="Opponent" >Opp</th>
<th aria-label=" " data-stat="game_result" scope="col" class=" poptip center" > </th>
<th aria-label="Games Started" data-stat="gs" scope="col" class=" poptip center" data-tip="Games Started" >GS</th>
<th aria-label="Minutes Played" data-stat="mp" scope="col" class=" poptip center" data-tip="Minutes Played" >MP</th>
<th aria-label="Field Goals" data-stat="fg" scope="col" class=" poptip center" data-tip="Field Goals" >FG</th>
<th aria-label="FGA" data-stat="fga" scope="col" class=" poptip center" data-tip="Field Goal Attempts" >FGA</th>
<th aria-label="Field Goal Percentage" data-stat="fg_pct" scope="col" class=" poptip center" data-tip="Field Goal Percentage" >FG%</th>
<th aria-label="3-Point Field Goals" data-stat="fg3" scope="col" class=" poptip center" data-tip="3-Point Field Goals" >3P</th>
<th aria-label="3-Point Field Goal Attempts" data-stat="fg3a" scope="col" class=" poptip center" data-tip="3-Point Field Goal Attempts" >3PA</th>
<th aria-label="3-Point Field Goal Percentage" data-stat="fg3_pct" scope="col" class=" poptip center" data-tip="3-Point Field Goal Percentage" >3P%</th>
<th aria-label="Free Throws" data-stat="ft" scope="col" class=" poptip center" data-tip="Free Throws" >FT</th>
<th aria-label="Free Throw Attempts" data-stat="fta" scope="col" class=" poptip center" data-tip="Free Throw Attempts" >FTA</th>
<th aria-label="Free Throw Percentage" data-stat="ft_pct" scope="col" class=" poptip center" data-tip="Free Throw Percentage" >FT%</th>
<th aria-label="Offensive Rebounds" data-stat="orb" scope="col" class=" poptip center" data-tip="Offensive Rebounds" >ORB</th>
<th aria-label="Defensive Rebounds" data-stat="drb" scope="col" class=" poptip center" data-tip="Defensive Rebounds" >DRB</th>
<th aria-label="Total Rebounds" data-stat="trb" scope="col" class=" poptip center" data-tip="Total Rebounds" >TRB</th>
<th aria-label="Assists" data-stat="ast" scope="col" class=" poptip center" data-tip="Assists" >AST</th>
<th aria-label="Steals" data-stat="stl" scope="col" class=" poptip center" data-tip="Steals" >STL</th>
<th aria-label="Blocks" data-stat="blk" scope="col" class=" poptip center" data-tip="Blocks" >BLK</th>
<th aria-label="Turnovers" data-stat="tov" scope="col" class=" poptip center" data-tip="Turnovers" >TOV</th>
<th aria-label="Personal Fouls" data-stat="pf" scope="col" class=" poptip center" data-tip="Personal Fouls" >PF</th>
<th aria-label="Points" data-stat="pts" scope="col" class=" poptip center" data-tip="Points" >PTS</th>
<th aria-label="GmSc" data-stat="game_score" scope="col" class=" poptip center" data-tip="Game Score" >GmSc</th>
<th aria-label="Plus/Minus" data-stat="plus_minus" scope="col" class=" poptip right" data-tip="Plus/Minus" >+/-</th>
</tr>
</thead>
<tbody><tr id="pgl_basic_playoffs.34" ><th scope="row" class="right " data-stat="ranker" csk="1" >1</th><td class="right endpoint tooltip" data-endpoint="/players/pgl_cum_stats.cgi?player=whitede01&year=2023&date_game=2023-04-15&is_playoff_game=Y" data-stat="game_season" ><strong>1</strong></td><td class="left " data-stat="date_game" ><a href="/boxscores/202304150BOS.html">2023-04-15</a></td><td class="right " data-stat="age" >28-287</td><td class="left " data-stat="team_id" ><a href="/teams/BOS/2023.html">BOS</a></td><td class="center iz" data-stat="game_location" ></td><td class="left " data-stat="opp_id" ><a href="/teams/ATL/2023.html">ATL</a></td><td class="center " data-stat="game_result" csk="13" >W (+13)</td><td class="right " data-stat="gs" >1</td><td class="right " data-stat="mp" csk="2310" >38:30</td><td class="right " data-stat="fg" >7</td><td class="right " data-stat="fga" >13</td><td class="right " data-stat="fg_pct" >.538</td><td class="right " data-stat="fg3" >4</td><td class="right " data-stat="fg3a" >7</td><td class="right " data-stat="fg3_pct" >.571</td><td class="right " data-stat="ft" >6</td><td class="right " data-stat="fta" >7</td><td class="right " data-stat="ft_pct" >.857</td><td class="right " data-stat="orb" >1</td><td class="right " data-stat="drb" >4</td><td class="right " data-stat="trb" >5</td><td class="right " data-stat="ast" >7</td><td class="right iz" data-stat="stl" >0</td><td class="right " data-stat="blk" >2</td><td class="right " data-stat="tov" >2</td><td class="right " data-stat="pf" >1</td><td class="right " data-stat="pts" >24</td><td class="right " data-stat="game_score" >23.1</td><td class="right " data-stat="plus_minus" >+13</td></tr>
<tr id="pgl_basic_playoffs.35" ><th scope="row" class="right " data-stat="ranker" csk="2" >2</th><td class="right endpoint tooltip" data-endpoint="/players/pgl_cum_stats.cgi?player=whitede01&year=2023&date_game=2023-04-18&is_playoff_game=Y" data-stat="game_season" ><strong>2</strong></td><td class="left " data-stat="date_game" ><a href="/boxscores/202304180BOS.html">2023-04-18</a></td><td class="right " data-stat="age" >28-290</td><td class="left " data-stat="team_id" ><a href="/teams/BOS/2023.html">BOS</a></td><td class="center iz" data-stat="game_location" ></td><td class="left " data-stat="opp_id" ><a href="/teams/ATL/2023.html">ATL</a></td><td class="center " data-stat="game_result" csk="13" >W (+13)</td><td class="right " data-stat="gs" >1</td><td class="right " data-stat="mp" csk="2025" >33:45</td><td class="right " data-stat="fg" >11</td><td class="right " data-stat="fga" >16</td><td class="right " data-stat="fg_pct" >.688</td><td class="right " data-stat="fg3" >2</td><td class="right " data-stat="fg3a" >5</td><td class="right " data-stat="fg3_pct" >.400</td><td class="right " data-stat="ft" >2</td><td class="right " data-stat="fta" >2</td><td class="right " data-stat="ft_pct" >1.000</td><td class="right iz" data-stat="orb" >0</td><td class="right " data-stat="drb" >7</td><td class="right " data-stat="trb" >7</td><td class="right " data-stat="ast" >2</td><td class="right " data-stat="stl" >1</td><td class="right " data-stat="blk" >3</td><td class="right " data-stat="tov" >3</td><td class="right " data-stat="pf" >1</td><td class="right " data-stat="pts" >26</td><td class="right " data-stat="game_score" >22.4</td><td class="right " data-stat="plus_minus" >+2</td></tr>
<tr id="pgl_basic_playoffs.36" ><th scope="row" class="right " data-stat="ranker" csk="3" >3</th><td class="right endpoint tooltip" data-endpoint="/players/pgl_cum_stats.cgi?player=whitede01&year=2023&date_game=2023-04-21&is_playoff_game=Y" data-stat="game_season" ><strong>3</strong></td><td class="left " data-stat="date_game" ><a href="/boxscores/202304210ATL.html">2023-04-21</a></td><td class="right " data-stat="age" >28-293</td><td class="left " data-stat="team_id" ><a href="/teams/BOS/2023.html">BOS</a></td><td class="center " data-stat="game_location" >@</td><td class="left " data-stat="opp_id" ><a href="/teams/ATL/2023.html">ATL</a></td><td class="center " data-stat="game_result" csk="-8" >L (-8)</td><td class="right " data-stat="gs" >1</td><td class="right " data-stat="mp" csk="1808" >30:08</td><td class="right " data-stat="fg" >4</td><td class="right " data-stat="fga" >7</td><td class="right " data-stat="fg_pct" >.571</td><td class="right " data-stat="fg3" >3</td><td class="right " data-stat="fg3a" >4</td><td class="right " data-stat="fg3_pct" >.750</td><td class="right iz" data-stat="ft" >0</td><td class="right iz" data-stat="fta" >0</td><td class="right iz" data-stat="ft_pct" ></td><td class="right iz" data-stat="orb" >0</td><td class="right " data-stat="drb" >2</td><td class="right " data-stat="trb" >2</td><td class="right " data-stat="ast" >4</td><td class="right " data-stat="stl" >1</td><td class="right iz" data-stat="blk" >0</td><td class="right iz" data-stat="tov" >0</td><td class="right " data-stat="pf" >5</td><td class="right " data-stat="pts" >11</td><td class="right " data-stat="game_score" >10.1</td><td class="right " data-stat="plus_minus" >-14</td></tr>
<tr id="pgl_basic_playoffs.37" ><th scope="row" class="right " data-stat="ranker" csk="4" >4</th><td class="right endpoint tooltip" data-endpoint="/players/pgl_cum_stats.cgi?player=whitede01&year=2023&date_game=2023-04-23&is_playoff_game=Y" data-stat="game_season" ><strong>4</strong></td><td class="left " data-stat="date_game" ><a href="/boxscores/202304230ATL.html">2023-04-23</a></td><td class="right " data-stat="age" >28-295</td><td class="left " data-stat="team_id" ><a href="/teams/BOS/2023.html">BOS</a></td><td class="center " data-stat="game_location" >@</td><td class="left " data-stat="opp_id" ><a href="/teams/ATL/2023.html">ATL</a></td><td class="center " data-stat="game_result" csk="8" >W (+8)</td><td class="right " data-stat="gs" >1</td><td class="right " data-stat="mp" csk="1891" >31:31</td><td class="right " data-stat="fg" >7</td><td class="right " data-stat="fga" >14</td><td class="right " data-stat="fg_pct" >.500</td><td class="right " data-stat="fg3" >4</td><td class="right " data-stat="fg3a" >9</td><td class="right " data-stat="fg3_pct" >.444</td><td class="right iz" data-stat="ft" >0</td><td class="right iz" data-stat="fta" >0</td><td class="right iz" data-stat="ft_pct" ></td><td class="right iz" data-stat="orb" >0</td><td class="right " data-stat="drb" >2</td><td class="right " data-stat="trb" >2</td><td class="right " data-stat="ast" >2</td><td class="right iz" data-stat="stl" >0</td><td class="right iz" data-stat="blk" >0</td><td class="right " data-stat="tov" >2</td><td class="right " data-stat="pf" >3</td><td class="right " data-stat="pts" >18</td><td class="right " data-stat="game_score" >9.8</td><td class="right " data-stat="plus_minus" >+7</td></tr>
<tr id="pgl_basic_playoffs.38" ><th scope="row" class="right " data-stat="ranker" csk="5" >5</th><td class="right endpoint tooltip" data-endpoint="/players/pgl_cum_stats.cgi?player=whitede01&year=2023&date_game=2023-04-25&is_playoff_game=Y" data-stat="game_season" ><strong>5</strong></td><td class="left " data-stat="date_game" ><a href="/boxscores/202304250BOS.html">2023-04-25</a></td><td class="right " data-stat="age" >28-297</td><td class="left " data-stat="team_id" ><a href="/teams/BOS/2023.html">BOS</a></td><td class="center iz" data-stat="game_location" ></td><td class="left " data-stat="opp_id" ><a href="/teams/ATL/2023.html">ATL</a></td><td class="center " data-stat="game_result" csk="-2" >L (-2)</td><td class="right " data-stat="gs" >1</td><td class="right " data-stat="mp" csk="1882" >31:22</td><td class="right " data-stat="fg" >7</td><td class="right " data-stat="fga" >11</td><td class="right " data-stat="fg_pct" >.636</td><td class="right " data-stat="fg3" >2</td><td class="right " data-stat="fg3a" >5</td><td class="right " data-stat="fg3_pct" >.400</td><td class="right " data-stat="ft" >2</td><td class="right " data-stat="fta" >2</td><td class="right " data-stat="ft_pct" >1.000</td><td class="right iz" data-stat="orb" >0</td><td class="right " data-stat="drb" >5</td><td class="right " data-stat="trb" >5</td><td class="right " data-stat="ast" >4</td><td class="right iz" data-stat="stl" >0</td><td class="right " data-stat="blk" >2</td><td class="right iz" data-stat="tov" >0</td><td class="right " data-stat="pf" >1</td><td class="right " data-stat="pts" >18</td><td class="right " data-stat="game_score" >18.4</td><td class="right " data-stat="plus_minus" >+9</td></tr>
<tr id="pgl_basic_playoffs.39" ><th scope="row" class="right " data-stat="ranker" csk="6" >6</th><td class="right endpoint tooltip" data-endpoint="/players/pgl_cum_stats.cgi?player=whitede01&year=2023&date_game=2023-04-27&is_playoff_game=Y" data-stat="game_season" ><strong>6</strong></td><td class="left " data-stat="date_game" ><a href="/boxscores/202304270ATL.html">2023-04-27</a></td><td class="right " data-stat="age" >28-299</td><td class="left " data-stat="team_id" ><a href="/teams/BOS/2023.html">BOS</a></td><td class="center " data-stat="game_location" >@</td><td class="left " data-stat="opp_id" ><a href="/teams/ATL/2023.html">ATL</a></td><td class="center " data-stat="game_result" csk="8" >W (+8)</td><td class="right " data-stat="gs" >1</td><td class="right " data-stat="mp" csk="1786" >29:46</td><td class="right " data-stat="fg" >3</td><td class="right " data-stat="fga" >9</td><td class="right " data-stat="fg_pct" >.333</td><td class="right " data-stat="fg3" >1</td><td class="right " data-stat="fg3a" >5</td><td class="right " data-stat="fg3_pct" >.200</td><td class="right iz" data-stat="ft" >0</td><td class="right iz" data-stat="fta" >0</td><td class="right iz" data-stat="ft_pct" ></td><td class="right " data-stat="orb" >1</td><td class="right " data-stat="drb" >1</td><td class="right " data-stat="trb" >2</td><td class="right " data-stat="ast" >2</td><td class="right iz" data-stat="stl" >0</td><td class="right " data-stat="blk" >1</td><td class="right " data-stat="tov" >1</td><td class="right " data-stat="pf" >3</td><td class="right " data-stat="pts" >7</td><td class="right " data-stat="game_score" >2.8</td><td class="right " data-stat="plus_minus" >-1</td></tr>
<tr id="pgl_basic_playoffs.40" ><th scope="row" class="right " data-stat="ranker" csk="7" >7</th><td class="right endpoint tooltip" data-endpoint="/players/pgl_cum_stats.cgi?player=whitede01&year=2023&date_game=2023-05-01&is_playoff_game=Y" data-stat="game_season" ><strong>7</strong></td><td class="left " data-stat="date_game" ><a href="/boxscores/202305010BOS.html">2023-05-01</a></td><td class="right " data-stat="age" >28-303</td><td class="left " data-stat="team_id" ><a href="/teams/BOS/2023.html">BOS</a></td><td class="center iz" data-stat="game_location" ></td><td class="left " data-stat="opp_id" ><a href="/teams/PHI/2023.html">PHI</a></td><td class="center " data-stat="game_result" csk="-4" >L (-4)</td><td class="right " data-stat="gs" >1</td><td class="right " data-stat="mp" csk="1628" >27:08</td><td class="right " data-stat="fg" >1</td><td class="right " data-stat="fga" >5</td><td class="right " data-stat="fg_pct" >.200</td><td class="right iz" data-stat="fg3" >0</td><td class="right " data-stat="fg3a" >4</td><td class="right iz" data-stat="fg3_pct" >.000</td><td class="right " data-stat="ft" >2</td><td class="right " data-stat="fta" >2</td><td class="right " data-stat="ft_pct" >1.000</td><td class="right iz" data-stat="orb" >0</td><td class="right " data-stat="drb" >2</td><td class="right " data-stat="trb" >2</td><td class="right " data-stat="ast" >3</td><td class="right " data-stat="stl" >1</td><td class="right iz" data-stat="blk" >0</td><td class="right " data-stat="tov" >2</td><td class="right " data-stat="pf" >2</td><td class="right " data-stat="pts" >4</td><td class="right " data-stat="game_score" >1.8</td><td class="right " data-stat="plus_minus" >-15</td></tr>
<tr id="pgl_basic_playoffs.41" ><th scope="row" class="right " data-stat="ranker" csk="8" >8</th><td class="right endpoint tooltip" data-endpoint="/players/pgl_cum_stats.cgi?player=whitede01&year=2023&date_game=2023-05-03&is_playoff_game=Y" data-stat="game_season" ><strong>8</strong></td><td class="left " data-stat="date_game" ><a href="/boxscores/202305030BOS.html">2023-05-03</a></td><td class="right " data-stat="age" >28-305</td><td class="left " data-stat="team_id" ><a href="/teams/BOS/2023.html">BOS</a></td><td class="center iz" data-stat="game_location" ></td><td class="left " data-stat="opp_id" ><a href="/teams/PHI/2023.html">PHI</a></td><td class="center " data-stat="game_result" csk="34" >W (+34)</td><td class="right " data-stat="gs" >1</td><td class="right " data-stat="mp" csk="1793" >29:53</td><td class="right " data-stat="fg" >5</td><td class="right " data-stat="fga" >9</td><td class="right " data-stat="fg_pct" >.556</td><td class="right " data-stat="fg3" >3</td><td class="right " data-stat="fg3a" >6</td><td class="right " data-stat="fg3_pct" >.500</td><td class="right " data-stat="ft" >2</td><td class="right " data-stat="fta" >2</td><td class="right " data-stat="ft_pct" >1.000</td><td class="right " data-stat="orb" >1</td><td class="right iz" data-stat="drb" >0</td><td class="right " data-stat="trb" >1</td><td class="right " data-stat="ast" >1</td><td class="right iz" data-stat="stl" >0</td><td class="right iz" data-stat="blk" >0</td><td class="right iz" data-stat="tov" >0</td><td class="right " data-stat="pf" >1</td><td class="right " data-stat="pts" >15</td><td class="right " data-stat="game_score" >11.7</td><td class="right " data-stat="plus_minus" >+23</td></tr>
<tr id="pgl_basic_playoffs.42" ><th scope="row" class="right " data-stat="ranker" csk="9" >9</th><td class="right endpoint tooltip" data-endpoint="/players/pgl_cum_stats.cgi?player=whitede01&year=2023&date_game=2023-05-05&is_playoff_game=Y" data-stat="game_season" ><strong>9</strong></td><td class="left " data-stat="date_game" ><a href="/boxscores/202305050PHI.html">2023-05-05</a></td><td class="right " data-stat="age" >28-307</td><td class="left " data-stat="team_id" ><a href="/teams/BOS/2023.html">BOS</a></td><td class="center " data-stat="game_location" >@</td><td class="left " data-stat="opp_id" ><a href="/teams/PHI/2023.html">PHI</a></td><td class="center " data-stat="game_result" csk="12" >W (+12)</td><td class="right " data-stat="gs" >1</td><td class="right " data-stat="mp" csk="1948" >32:28</td><td class="right " data-stat="fg" >4</td><td class="right " data-stat="fga" >9</td><td class="right " data-stat="fg_pct" >.444</td><td class="right " data-stat="fg3" >2</td><td class="right " data-stat="fg3a" >5</td><td class="right " data-stat="fg3_pct" >.400</td><td class="right " data-stat="ft" >3</td><td class="right " data-stat="fta" >3</td><td class="right " data-stat="ft_pct" >1.000</td><td class="right " data-stat="orb" >1</td><td class="right " data-stat="drb" >5</td><td class="right " data-stat="trb" >6</td><td class="right " data-stat="ast" >2</td><td class="right iz" data-stat="stl" >0</td><td class="right iz" data-stat="blk" >0</td><td class="right " data-stat="tov" >1</td><td class="right " data-stat="pf" >1</td><td class="right " data-stat="pts" >13</td><td class="right " data-stat="game_score" >10.5</td><td class="right " data-stat="plus_minus" >+5</td></tr>
<tr id="pgl_basic_playoffs.43" ><th scope="row" class="right " data-stat="ranker" csk="10" >10</th><td class="right endpoint tooltip" data-endpoint="/players/pgl_cum_stats.cgi?player=whitede01&year=2023&date_game=2023-05-07&is_playoff_game=Y" data-stat="game_season" ><strong>10</strong></td><td class="left " data-stat="date_game" ><a href="/boxscores/202305070PHI.html">2023-05-07</a></td><td class="right " data-stat="age" >28-309</td><td class="left " data-stat="team_id" ><a href="/teams/BOS/2023.html">BOS</a></td><td class="center " data-stat="game_location" >@</td><td class="left " data-stat="opp_id" ><a href="/teams/PHI/2023.html">PHI</a></td><td class="center " data-stat="game_result" csk="-1" >L (-1)</td><td class="right " data-stat="gs" >1</td><td class="right " data-stat="mp" csk="1843" >30:43</td><td class="right " data-stat="fg" >5</td><td class="right " data-stat="fga" >11</td><td class="right " data-stat="fg_pct" >.455</td><td class="right " data-stat="fg3" >2</td><td class="right " data-stat="fg3a" >4</td><td class="right " data-stat="fg3_pct" >.500</td><td class="right iz" data-stat="ft" >0</td><td class="right iz" data-stat="fta" >0</td><td class="right iz" data-stat="ft_pct" ></td><td class="right " data-stat="orb" >1</td><td class="right " data-stat="drb" >1</td><td class="right " data-stat="trb" >2</td><td class="right iz" data-stat="ast" >0</td><td class="right iz" data-stat="stl" >0</td><td class="right " data-stat="blk" >1</td><td class="right iz" data-stat="tov" >0</td><td class="right " data-stat="pf" >1</td><td class="right " data-stat="pts" >12</td><td class="right " data-stat="game_score" >7.6</td><td class="right " data-stat="plus_minus" >-8</td></tr>
<tr id="pgl_basic_playoffs.44" ><th scope="row" class="right " data-stat="ranker" csk="11" >11</th><td class="right endpoint tooltip" data-endpoint="/players/pgl_cum_stats.cgi?player=whitede01&year=2023&date_game=2023-05-09&is_playoff_game=Y" data-stat="game_season" ><strong>11</strong></td><td class="left " data-stat="date_game" ><a href="/boxscores/202305090BOS.html">2023-05-09</a></td><td class="right " data-stat="age" >28-311</td><td class="left " data-stat="team_id" ><a href="/teams/BOS/2023.html">BOS</a></td><td class="center iz" data-stat="game_location" ></td><td class="left " data-stat="opp_id" ><a href="/teams/PHI/2023.html">PHI</a></td><td class="center " data-stat="game_result" csk="-12" >L (-12)</td><td class="right " data-stat="gs" >1</td><td class="right " data-stat="mp" csk="2018" >33:38</td><td class="right " data-stat="fg" >2</td><td class="right " data-stat="fga" >6</td><td class="right " data-stat="fg_pct" >.333</td><td class="right iz" data-stat="fg3" >0</td><td class="right " data-stat="fg3a" >2</td><td class="right iz" data-stat="fg3_pct" >.000</td><td class="right " data-stat="ft" >3</td><td class="right " data-stat="fta" >3</td><td class="right " data-stat="ft_pct" >1.000</td><td class="right " data-stat="orb" >2</td><td class="right " data-stat="drb" >3</td><td class="right " data-stat="trb" >5</td><td class="right iz" data-stat="ast" >0</td><td class="right iz" data-stat="stl" >0</td><td class="right iz" data-stat="blk" >0</td><td class="right iz" data-stat="tov" >0</td><td class="right " data-stat="pf" >1</td><td class="right " data-stat="pts" >7</td><td class="right " data-stat="game_score" >5.5</td><td class="right " data-stat="plus_minus" >-15</td></tr>
<tr id="pgl_basic_playoffs.45" ><th scope="row" class="right " data-stat="ranker" csk="12" >12</th><td class="right endpoint tooltip" data-endpoint="/players/pgl_cum_stats.cgi?player=whitede01&year=2023&date_game=2023-05-11&is_playoff_game=Y" data-stat="game_season" ><strong>12</strong></td><td class="left " data-stat="date_game" ><a href="/boxscores/202305110PHI.html">2023-05-11</a></td><td class="right " data-stat="age" >28-313</td><td class="left " data-stat="team_id" ><a href="/teams/BOS/2023.html">BOS</a></td><td class="center " data-stat="game_location" >@</td><td class="left " data-stat="opp_id" ><a href="/teams/PHI/2023.html">PHI</a></td><td class="center " data-stat="game_result" csk="9" >W (+9)</td><td class="right iz" data-stat="gs" >0</td><td class="right " data-stat="mp" csk="1152" >19:12</td><td class="right " data-stat="fg" >3</td><td class="right " data-stat="fga" >6</td><td class="right " data-stat="fg_pct" >.500</td><td class="right " data-stat="fg3" >3</td><td class="right " data-stat="fg3a" >5</td><td class="right " data-stat="fg3_pct" >.600</td><td class="right iz" data-stat="ft" >0</td><td class="right iz" data-stat="fta" >0</td><td class="right iz" data-stat="ft_pct" ></td><td class="right iz" data-stat="orb" >0</td><td class="right " data-stat="drb" >2</td><td class="right " data-stat="trb" >2</td><td class="right " data-stat="ast" >1</td><td class="right " data-stat="stl" >1</td><td class="right " data-stat="blk" >1</td><td class="right " data-stat="tov" >1</td><td class="right iz" data-stat="pf" >0</td><td class="right " data-stat="pts" >9</td><td class="right " data-stat="game_score" >8.0</td><td class="right " data-stat="plus_minus" >-7</td></tr>
<tr id="pgl_basic_playoffs.46" ><th scope="row" class="right " data-stat="ranker" csk="13" >13</th><td class="right endpoint tooltip" data-endpoint="/players/pgl_cum_stats.cgi?player=whitede01&year=2023&date_game=2023-05-14&is_playoff_game=Y" data-stat="game_season" ><strong>13</strong></td><td class="left " data-stat="date_game" ><a href="/boxscores/202305140BOS.html">2023-05-14</a></td><td class="right " data-stat="age" >28-316</td><td class="left " data-stat="team_id" ><a href="/teams/BOS/2023.html">BOS</a></td><td class="center iz" data-stat="game_location" ></td><td class="left " data-stat="opp_id" ><a href="/teams/PHI/2023.html">PHI</a></td><td class="center " data-stat="game_result" csk="24" >W (+24)</td><td class="right iz" data-stat="gs" >0</td><td class="right " data-stat="mp" csk="746" >12:26</td><td class="right " data-stat="fg" >1</td><td class="right " data-stat="fga" >4</td><td class="right " data-stat="fg_pct" >.250</td><td class="right " data-stat="fg3" >1</td><td class="right " data-stat="fg3a" >2</td><td class="right " data-stat="fg3_pct" >.500</td><td class="right iz" data-stat="ft" >0</td><td class="right iz" data-stat="fta" >0</td><td class="right iz" data-stat="ft_pct" ></td><td class="right iz" data-stat="orb" >0</td><td class="right iz" data-stat="drb" >0</td><td class="right iz" data-stat="trb" >0</td><td class="right iz" data-stat="ast" >0</td><td class="right iz" data-stat="stl" >0</td><td class="right " data-stat="blk" >1</td><td class="right iz" data-stat="tov" >0</td><td class="right " data-stat="pf" >3</td><td class="right " data-stat="pts" >3</td><td class="right " data-stat="game_score" >0.1</td><td class="right " data-stat="plus_minus" >-8</td></tr>
</table>
</div>
-->
And here is the code that worked for me to access the table.
box::use(
rvest[...],
magrittr[`%>%`]
readr[...]
)
"https://www.basketball-reference.com/players/w/whitede01/gamelog/2023" %>%
read_html() %>%
html_nodes(xpath = '//body//div[@id="wrap"]//div[@id="content"]/div[6]/div[@id="all_pgl_basic_playoffs"]//comment()') %>%
html_text() %>% # extract comment text
paste(collapse = '') %>% # collapse to single string
read_html() %>% # reread as HTML
html_node('table#pgl_basic_playoffs') %>% # select desired node
html_table() %>% # parse node to table
{ setNames(.[-1, ], paste0(names(.), .[1, ])) } %>% # extract names from first row
type_convert()
A side note: I tend to try to be a bit more specific, where possible when I am writing my xml paths to make sure that I am not accessing the wrong table or the wrong div. Some folks will give different elements the same id label which can cause some issues.