Search code examples
htmlxpathweb-inspector

Why is the web page html element (absolute xpath) dissapeared when downloaded as html file? what cause it?

First, let's set up a code for full-xpath:

(focus) for /html/body/div[2]/div/div[1]/div[3]/div/div

(sub-focus) for /html/body/div[2]/div/div[1]/div[3]/div/div[2]

Tool to use:

  1. Online Inspector: Chrome or Firefox (any of them will gives the same result)
  2. Offline Inspector: any code-editor, I personally use Sublime-text
  3. HTML prettifier to visually tidy up the downloaded html (optional)

Cases

piece of structure from non-downloaded html on the chrome browser web-inspecting tool

▼<div class="BrandPageWrapper-liveAlertAd BrandPageWrapper-contentWrapper" role="main" id="MainContent Container"> = $0
<p id="MainContent" tabindex="-1"></p>
  ▼<div class="QuotePageBuilder-container">
    ►<div class="BadgeGroup-badgeGroup">...</div> (focus)
    ►<div class="QuotePageBuilder-row">...</div> (sub-focus)
    </div>
  </div>
►<div id="MobileAdhesion-Homepage" class="MobileAdhesion-container" data-module="mps-slot">.</div>

piece of stucture from downloaded html on code-editor (cannot use chrome inspector, something prevent it to show the full content, preventing the user from scraping the data)

▼<div class="BrandPageWrapper-liveAlertAd BrandPageWrapper-contentWrapper" role="main" id="MainContentContainer">
<p id="MainContent" tabindex="-1"></p>
  ▼<div class="QuotePageBuilder-container">
    ▼<div class="QuotePageBuilder-row">...</div> (sub-focus)

As you can see, the (focus) is missing from the html, If I tried to search the BadgeGroup-badgeGroup class in the html document, I gone zero result

questions:

  1. Why is it gone?
  2. Where is it gone to?

bonus-question:

In chrome browser Inspector search element plugin both /html/body/div[2]/div/div[1]/div[3]/div/div[2]/div[1]/div[2]/div[3]/div/div[2]/span[1] and /html/body/div[2]/div/div[1]/div[3]/div/div/div[1]/div[2]/div[3]/div/div[2]/span[1] pointing at the same element inside the (sub-focus), why?

source

Solution

  • Almost certainly the HTML source of the web page does not contain the HTML concerned (the piece you describe as "gone" in the downloaded file), and it's actually generated in the browser by JavaScript.