Search code examples
gitlabjekylllighthouse-ci

Lighthouse GitLab CI for a Jekyll website times out

I haven't found any complete examples for Jekyll + Lighthouse GitLab jobs. The Jekyll build job is based on the ruby:2.6 image. I added a report stage into my .gitlab-ci.yml where I install the required nodejs, npm, and chromium for the LHCI execution.

# requiring the environment of Ruby 2.6.x
image: ruby:2.6

# add bundle cache to 'vendor' for speeding up builds
cache:
  paths:
    - vendor/

variables:
  JEKYLL_ENV: production

stages:
  - deploy
  - report

pages:
  stage: deploy
  before_script:
  - apt-get update -qq && apt-get install -y
  - bundle install --path vendor
  script:
  - bundle exec jekyll build -d public/
  artifacts:
    paths:
    - public
  only:
  - master

static_review_report:
  stage: report
  before_script:
  - apt-get update -qq && apt-get install -y
  - apt-get install -y chromium
  - apt-get install -y nodejs
  - apt-get install -y npm
  - bundle install --path vendor
  - bundle exec jekyll build -d public/
  - npm install -g @lhci/[email protected]
  script:
  - lhci autorun || echo "LHCI failed!"
  artifacts:
    paths:
    - public
  allow_failure: true
  only:
  - master

My .lighouserc.json

{
  "ci": {
    "collect": {
      "url": [
        "https://csaba.page/",
        "https://csaba.page/about.html",
        "https://csaba.page/blog/",
        "https://csaba.page/blog/suunto-under-armour-integrations.html"
      ],
      "numberOfRuns": 1,
      "staticDistDir": "./public",
      "settings": {
        "preset": "desktop",
        "chromeFlags": "--no-sandbox --disable-dev-shm-usage --headless --in-process-gpu"
      }
    },
    "assert": {
      "preset": "lighthouse:recommended",
      "assertions": {
        "first-contentful-paint": [
          "warn",
          {
            "maxNumericValue": 2500,
            "aggregationMethod": "optimistic"
          }
        ],
        "interactive": [
          "warn",
          {
            "maxNumericValue": 5000,
            "aggregationMethod": "optimistic"
          }
        ],
        "uses-long-cache-ttl": "off",
        "uses-http2": "off"
      }
    },
    "upload": {
      "target": "temporary-public-storage"
    }
  }
}

The problem is that the LHCI times out:

$ lhci autorun || echo "LHCI failed!"
✅  .lighthouseci/ directory writable
✅  Configuration file found
✅  Chrome installation found
Healthcheck passed!
Started a web server on port 36651...
Running Lighthouse 1 time(s) on https://csaba.page:36651/
Run #1...failed!
Error: Lighthouse failed with exit code 1
    at ChildProcess.<anonymous> (/usr/local/lib/node_modules/@lhci/cli/src/collect/node-runner.js:120:21)
    at ChildProcess.emit (events.js:314:20)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:276:12)
Sun, 12 Dec 2021 21:46:08 GMT ChromeLauncher Waiting for browser.
Sun, 12 Dec 2021 21:46:08 GMT ChromeLauncher Waiting for browser...
Sun, 12 Dec 2021 21:46:09 GMT ChromeLauncher Waiting for browser.....
Sun, 12 Dec 2021 21:46:09 GMT ChromeLauncher Waiting for browser.....✓
Sun, 12 Dec 2021 21:46:09 GMT config:warn IFrameElements gatherer requested, however no audit requires it.
Sun, 12 Dec 2021 21:46:09 GMT config:warn FormElements gatherer requested, however no audit requires it.
Sun, 12 Dec 2021 21:46:09 GMT status Connecting to browser
Sun, 12 Dec 2021 21:46:09 GMT status Resetting state with about:blank
Sun, 12 Dec 2021 21:46:09 GMT status Benchmarking machine
Sun, 12 Dec 2021 21:46:10 GMT status Initializing…
Sun, 12 Dec 2021 21:46:10 GMT status Running defaultPass pass CSSUsage, JsUsage, ViewportDimensions, ConsoleMessages, AnchorElements, ImageElements, LinkElements, MetaElements, ScriptElements, IFrameElements, FormElements, MainDocumentContent, GlobalListeners, AppCacheManifest, Doctype, DOMStats, OptimizedImages, PasswordInputsWithPreventedPaste, ResponseCompression, TagsBlockingFirstPaint, FontSize, EmbeddedContent, RobotsTxt, TapTargets, Accessibility, TraceElements, InspectorIssues, SourceMaps, FullPageScreenshot
Sun, 12 Dec 2021 21:46:10 GMT status Resetting state with about:blank
Sun, 12 Dec 2021 21:46:11 GMT status Cleaning browser cache
Sun, 12 Dec 2021 21:46:11 GMT status Preparing network conditions
Sun, 12 Dec 2021 21:46:11 GMT status Beginning devtoolsLog and trace
Sun, 12 Dec 2021 21:46:11 GMT status Loading page & waiting for onload
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: CSSUsage
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: JsUsage
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: ViewportDimensions
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: ConsoleMessages
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: AnchorElements
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: ImageElements
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: LinkElements
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: MetaElements
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: ScriptElements
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: IFrameElements
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: FormElements
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: MainDocumentContent
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: GlobalListeners
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: AppCacheManifest
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: Doctype
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: DOMStats
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: OptimizedImages
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: PasswordInputsWithPreventedPaste
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: ResponseCompression
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: TagsBlockingFirstPaint
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: FontSize
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: EmbeddedContent
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: RobotsTxt
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: TapTargets
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: Accessibility
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: TraceElements
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: InspectorIssues
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: SourceMaps
Sun, 12 Dec 2021 21:46:41 GMT status Gathering in-page: FullPageScreenshot
Sun, 12 Dec 2021 21:46:41 GMT status Gathering trace
Sun, 12 Dec 2021 21:46:41 GMT status Gathering devtoolsLog & network records
Sun, 12 Dec 2021 21:47:11 GMT status Disconnecting from browser...
Sun, 12 Dec 2021 21:47:11 GMT ChromeLauncher Killing Chrome instance 5432
Runtime error encountered: Waiting for DevTools protocol response has exceeded the allotted time. (Method: Network.emulateNetworkConditions)
LHError: PROTOCOL_TIMEOUT
    at Timeout.<anonymous> (/usr/local/lib/node_modules/@lhci/cli/node_modules/lighthouse/lighthouse-core/gather/driver.js:322:21)
    at listOnTimeout (internal/timers.js:554:17)
    at processTimers (internal/timers.js:497:7)
LHCI failed!

I haven't found much about this combination but in this ticket https://github.com/GoogleChrome/lighthouse-ci/issues/504 it seems that there's no solution? I refuse to believe that noone with a Jekyll SSG website hosted on GitLab would do Lighthouse CI.

I went through this umbrella issue https://github.com/GoogleChrome/lighthouse/issues/6512 and added --disable-dev-shm-usage --headless --in-process-gpu Chromium options besides the --no-sandbox. I also added the desktop preset to the lighthouserc hoping that it may help. I hope someone here has a working set of .gitlab-ci.yml + .lighouserc.json or some tips.

Solution

  • This two lines in the log shed light on the real problem:

    Started a web server on port 36651...
Running Lighthouse 1 time(s) on https://csaba.page:36651/

    Now I understand why the test failed: https://csaba.page:36651/ is really not available, that port is closed. I realized that I need to point the testing in the .lighouserc.json to localhost instead of the real public domain, because that's where LHCI starts a server for the test.

    I'll see in the future if that random port could be just overridden with 443 (https) so the live site could be tested. If you refer localhost domain in the url section of your .lighouserc.json then the test will automatically insert that port into the URL (just like it did with the public domain). See https://github.com/GoogleChrome/lighthouse-ci/blob/main/docs/configuration.md

    "URLs will have their port replaced with the port of the local server that Lighthouse CI starts for you. This allows you to write URLs as http://localhost/my-static-page.html without worrying about the chosen temporary port."

    Now the job runs and doesn't time out.