Search code examples
browserdevelopment-environment

Access browser console from a terminal?

Is it possible to access Developer Tools (Chrome/Firefox) console from a Linux terminal for scripting purposes?

I am trying to access an AngularJS application's variables from my browser so that I can quickly navigate across files. For example, I could ask for the state, and its related partial, controller etc; once I find the result, I would open that file in my editor.

The other alternatives I am exploring is to use some kind of headless browser. There is also devtools-protocol which is used by tools like puppeteer for programmatic access to Chrome Developer Tools.

A simple Node REPL would have been enough, but the problem is that I will have to figure a lot of stuff manually if I have load my site's JavaScript files in to REPL.

Does anyone have some good suggestions?

Solution

  • Unless you want to use or write a JavaScript parser, which is only fun for a very limited set of individuals, I suggest taking advantage of the thriving headless Chrome community. Grabbing JS variables with puppeteer is straightforward after some boilerplate node code. It's also shockingly (but not "blazingly") fast.

    Before running the code:

    • Have node js and npm working on your machine
    • Install jq for parsing JSON in the shell. It is available in most package managers, so brew install jq or sudo apt install jq etc should work.
    • Install Puppeteer in whichever directory these scripts are going to live in with npm i puppeteer

    A file like this is all you need to get started with Puppeteer. I added comments to the key areas.

    #!/usr/bin/env node

const puppeteer = require('puppeteer')

;(async () => {
  const browser = await puppeteer.launch()
  // Replace the line above with this statement a fun show
  // const browser = await puppeteer.launch({
  //   headless: false,
  //   devtools: true,
  // })
  const page = await browser.newPage()

  // Arbitrarily choosing SO for the demo, replace with your website
  await page.goto('https://stackoverflow.com/')
  // Or use an argument:
  // const uri = process.argv[2]
  // await page.goto(process.argv[0])

  const retrievedData = await page.evaluate(() => {
    // This block has the page context, which is almost identical to being in the console
    // except for some of the console's supplementary APIs.

    // Get the URL host name and path separately
    const { origin, pathname } = window.location;

    // Get the title in a silly way, for demonstration purposes only
    const title = document.querySelector('title').textContent

    // More interesting - save data from the `StackExchange` object from `window`
    const { options: soOptions } = window.StackExchange

    // Return an object literal with data for the shell script 
    return {
      origin,
      pathname,
      soOptions,
    }
  })

  // Convert the object from the browser eval to JSON to parse with with jq later
  const retrievedJSON = JSON.stringify(retrievedData, null, 4)

  // console.log writes to stdout in node
  console.log(retrievedJSON)

  await browser.close()
})()

    Note the shebang at the top, which makes the shell understand to run it with node.

    If we make this file executable and run it:

    chmod +x get-so-data.js
./get-so-data.js

    We have a CLI utility that will provide a JSON string of data from the JavaScript global execution context of the website. Here are some small generic shell examples.

    #!/bin/sh

# Confirm that jq understands the result (should pretty print with ANSI colors):
./get-so-data.js | jq
# {
#   Many data...
# }

# Check if user is logged in (the user is our node script in a sandboxed browser, so no):
./get-so-data.js | jq '.soOptions.user.isRegistered' 
# null

# Tell the time, according to StackExchange's server clock (linux only):
./get-so-data.js | jq '.soOptions.serverTime' | date -d $(echo -n '@' && cat --)
# Fri 11 Sep 2020 04:37:02 PM PDT

# Open a subset of the JSON payload returned by Puppeteer in the default editor:
./get-so-data.js | jq '.soOptions' | $EDITOR --
# or VS Code specifically
./get-so-data.js | jq '.soOptions' | code --

# ...

    As long as the JavaScript side of the equation is returning enough info to construct a file path, you can open files in your editor based on JavaScript in a browser.

    The shell date example takes about 1.5 seconds on a three-year-old Chromebook from within a Linux (Beta) container using 25mbps public wifi. Your mileage will vary depending on the performance of the site you're debugging and the steps in the script.

    $ time ./get-so-data.js | jq '.soOptions.serverTime' | date -d $(echo -ne '@' && cat --)
Fri 11 Sep 2020 04:43:24 PM PDT

real    0m1.515s
user    0m0.945s
sys     0m0.383s

$ time ./get-so-data.js | jq '.soOptions.serverTime' | date -d $(echo -ne '@' && cat --)
Fri 11 Sep 2020 04:43:30 PM PDT

real    0m1.755s
user    0m0.999s
sys     0m0.433s

$ time ./get-so-data.js | jq '.soOptions.serverTime' | date -d $(echo -ne '@' && cat --)
Fri 11 Sep 2020 04:43:33 PM PDT

real    0m1.422s
user    0m0.802s
sys     0m0.361s

    Resources