Angular + Cypress code coverage reporting not working

Question

I have been trying to get Cypress code coverage working with my Angular production project to no avail.

To try and help diagnose it, I have created a minimal implementation project to make sure I wasn't introducing anything weird in the production version, which I don't think I am as the same issue is still happening. It's starting to drive me mad!

I have used a few references and as far as I can see I have the things in place I need to:

• Cypress Coverage Documentation
• Article 1
• Code Example (older dependencies)

As far as I can tell the Angular and Cypress side is all hooked up and am getting output in the .nyc_output folder and a coverage report. However the report is not indicating typescript line coverage or including those stats.

I have seen this but didn't seem to help.

Code Instrumentation (webpack extension + angular.json):

module.exports = {
  module: {
    rules: [
      {
        test: /\.(js|ts)$/,
        loader: "istanbul-instrumenter-loader",
        options: { esModules: true },
        enforce: "post",
        include: require("path").join(__dirname, "..", "src"),
        exclude: [
          /\.(e2e|spec)\.ts$/,
          /node_modules/,
          /(ngfactory|ngstyle)\.js/,
        ],
      },
    ],
  },
};

"serve": {
  "builder": "ngx-build-plus:dev-server",
  "options": {
    "browserTarget": "architecture-testing:build",
    "extraWebpackConfig": "./cypress/coverage.webpack.js",
    "sourceMap": true
  },
  "configurations": {
    "production": {
      "browserTarget": "architecture-testing:build:production"
    }
  }
}

Cypress appears to be recording and saving coverage:

const registerCodeCoverageTasks = require("@cypress/code-coverage/task");

module.exports = (on, config) => {
  registerCodeCoverageTasks(on, config);

  return config;
};

out.json appears to have the correct file and code mapping:

:

package.json (nyc config + deps):

{
  "name": "architecture-testing",
  "version": "0.0.0",
  "scripts": {
    "postinstall": "ngcc",
    "ng": "ng",
    "start": "ng serve",
    "build": "ng build",
    "test": "ng test",
    "lint": "ng lint",
    "e2e": "ng e2e",
    "precypress": "rimraf .nyc_output coverage",
    "cypress": "ng run architecture-testing:cypress-run",
    "cypress:open": "cypress open",
    "cypress:run": "cypress run"
  },
  "nyc": {
    "extends": "@istanbuljs/nyc-config-typescript",
    "all": true,
    "exclude": [
      "coverage/**",
      "cypress/**",
      "**/*.spec.ts"
    ]
  },
  "private": true,
  "dependencies": {
    "@angular/animations": "~9.1.9",
    "@angular/common": "~9.1.9",
    "@angular/compiler": "~9.1.9",
    "@angular/core": "~9.1.9",
    "@angular/forms": "~9.1.9",
    "@angular/platform-browser": "~9.1.9",
    "@angular/platform-browser-dynamic": "~9.1.9",
    "@angular/router": "~9.1.9",
    "rxjs": "~6.5.4",
    "tslib": "^1.10.0",
    "zone.js": "~0.10.2"
  },
  "devDependencies": {
    "@angular-devkit/build-angular": "~0.901.7",
    "@angular/cli": "~9.1.7",
    "@angular/compiler-cli": "~9.1.9",
    "@briebug/cypress-schematic": "^3.3.0",
    "@cypress/code-coverage": "^3.8.1",
    "@cypress/webpack-preprocessor": "5.4.1",
    "@istanbuljs/nyc-config-typescript": "^1.0.1",
    "@types/node": "^12.11.1",
    "codelyzer": "^5.1.2",
    "cypress": "^4.8.0",
    "istanbul-instrumenter-loader": "^3.0.1",
    "istanbul-lib-coverage": "^3.0.0",
    "ngx-build-plus": "^9.0.6",
    "nyc": "^15.1.0",
    "rimraf": "^3.0.2",
    "source-map-support": "^0.5.19",
    "ts-loader": "7.0.5",
    "ts-node": "^8.3.0",
    "tslint": "~6.1.0",
    "typescript": "~3.8.3"
  }
}

Spec file:

it('does something', () => {
  cy.visit('http://localhost:4200');
  cy.get('[data-cy=button-one]').click();
  cy.get('[data-cy=button-output]').should('have.text', 'you clicked button 1');
});

Sorry it's so long but I am stuck as to where to go next. Many thanks if you can point me in any direction.

Update based on answer investigation:

Looking at the past versions of the @cypress/code-coverage it appears the issue for me was introduced in v3.3.0 of the plugin. All the versions for v3.2.* were working for me when downgrading my minimal project. After looking at the documentation changes for v3.3.0 the key bit of information in the readme was:

**Note:** if you have `all: true` NYC option set, this plugin will check the produced `.nyc_output/out.json` before generating the final report. If the `out.json` file does not have information for some files that should be there according to `include` list, then an empty placeholder will be included, see [PR 208](https://github.com/cypress-io/code-coverage/pull/208).

My original nyc config was:

"nyc": {
  "extends": "@istanbuljs/nyc-config-typescript",
  "all": true,
  "exclude": [
    "coverage/**",
    "cypress/**",
    "**/*.spec.ts"
  ]
}

So for some reason even though I have metrics for the files that were being tested in out.json a second "empty placeholder" node was being created and overriding the subsequent report generation. I am guessing that this is potentially a bug or a problem with my setup again so will ask the creators.

I can now see coverage if I change my nyc config to:

"nyc": {
  "extends": "@istanbuljs/nyc-config-typescript",
  "all": true,
  "include": [
    "src/**/*.ts"
  ],
  "exclude": [
    "coverage/**",
    "cypress/**",
    "**/*.spec.ts"
  ]
}

This does mean that if I don't hit a file with testing it won't be included as an empty placeholder as "all": true is no longer present.

Looking at @briebug/[email protected] this doesn't appear to be causing any issues (same happens without using their builder) but has been raised as one here and here.

Answer 1

Comparing ang-cy-cov-example to your package.json, a major difference is that he uses @cypress/[email protected] where-as you have the latest v3.8.1.

Changing back to this v1.14.0 works ok with your setup. Since your info indicates data is appearing in .nyc_output/out.json, I tested with the command line ./node_modules/.bin/nyc report which gives a quick view in the console.

Comparing .nyc_output/out.json between the two versions, the individual nodes are structurally the same, i.e have the correct sections (path, statementMap, inputSourceMap etc).

There are two types of additional nodes

• additional files such as karma.conf.js, coverage.webpack.js, cy-ts-preprocessor.js, integration/spec.ts, support/commands.ts - which we are not interested in.

• the files we are interested in are duplicated at the end of the file but the duplicates have no coverage metrics.

e.g

First copy of main.ts with metrics

  "path-to\\src\\main.ts": {
    "path": "path-to\\src\\main.ts",
    "statementMap": {
         ...
      },
      "1": {
         ...
      },
      "2": {
         ...
      }
    },
    "fnMap": {},
    "branchMap": {
         ...
    },
    "s": {
      "0": 1,   // indicates one visit to this statement
      "1": 0,
      "2": 1
    },
    "f": {},
    "b": {
      "0": [
        0,
        1
      ]
    },
    "inputSourceMap": {
         ...
    },
    "_coverageSchema": "332fd63041d2c1bcb487cc26dd0d5f7d97098a6c",
    "hash": "5959c383a9744c99a600a28ff82b12f0a540c5e6"
  },

Second copy of main.ts with no metrics

  "path-to/src/main.ts": {
    "path": "path-to/src/main.ts",
    "statementMap": {},
    "fnMap": {},
    "branchMap": {},
    "s": {},          // no metrics recorded here
    "f": {},
    "b": {}
  },

So, conclusion is the NYC report is replacing the first nodes metrics with the empty metrics of the second node.

I skipped back through the versions, v3.2.0 was the latest I found working.

Also note this warning when adding node modules, but can't say if it is a contributing factor.

warning " > @briebug/[email protected]" has incorrect peer dependency "cypress@^3.6.1"

---

Point of failure

The basic problem is in task-utils.js.

ref getting allFiles

const allFiles = globby.sync(patterns, { absolute: true })

where globby returns paths with forward-slash even on Windows

and ref getting coveredPaths

const coveredPaths = coverageKeys.map(key => nycCoverage[key].path)

where the keys have be saved in out.json with back-slash in Windows.

A quick resolve would be to normalize the paths at this point

const coveredPaths = coverageKeys.map(key => nycCoverage[key].path)
  .map(path => path.replace(/\\/g, '/'))  // Compare "normalized" paths 

Patching '/node_modules/@cypress/code-coverage/task-utils.js' fixes the problem.