Search code examples
web-scrapingreverse-engineeringweb-frontend

Where does 'SOURCE' file come from in Firefox Dev Tools Debugger?


Using Firefox's Web Developer Tools and loading up the homepage from say https://www.espn.com , I switch to the Debugger tab and see this:

1 - espn.com w/firefox dev tools on debugger tab

If I try to 'Find in Files' (Ctrl+Shift+F) in Dev Tools, say for 'data-id', I get results like this:

dev tools search results

My question is about the results like the two that list as '(1 match)' with no source file name. If I click a result from one of them, I jump to a 'SOURCE' file like this:

example SOURCE file

Generically asking (only using espn.com as an example) what is this 'SOURCE' file content of, what generates it, etc? I'm looking to better understand how particular websites supply dynamic content to their pages and find the information I'm interested in is found in these type of results/'files'. But without an understanding of them, it is unclear how I might be able to request/generate/scrape for them.

(Also, I notice that only Firefox's Dev Tools shows these type of generated (assumed) files, Chrome doesn't.)

Thanks


Solution

  • You may produce this by several means:

    new Function(content) does this (which is the one ESPN website uses)

    new Function(`const unique_string="_AAAAA_"; console.log(unique_string)`)();
    Go to Firefox's dev-tools' debugger and search for the unique string

    eval(content) does this too

    new Function(`const unique_string="_BBBBB_"; console.log(unique_string)`)();
    Go to Firefox's dev-tools' debugger and search for the unique string

    so does Element#append(Range#createContextualFragment("<script>"+content+"<\/script>")

    document.body.append(
      document.createRange().createContextualFragment(
        `<script>const unique_string="_CCCCC_"; console.log(unique_string)<\/script>`
      )
    );
    Go to Firefox's dev-tools' debugger and search for the unique string

    and maybe others I don't think of right now.
    (To say the truth I find it a bit surprising that something like setTimeout(content) doesn't produce one).