Search code examples
javascripttestingjasmineprotractorend-to-end

expect() with no actual expectations


The Problem:

Recently, while reviewing our existing test codebase, I've noticed a dangerous kind of typo/mistake when expect() was used without the "matching" part:

expect(page.filters.fromDateLabel.getText(), "After");

I'm pretty sure toEqual() was meant to be used here:

expect(page.filters.fromDateLabel.getText()).toEqual("After");

The problem with this is that jasmine would not fail the expectation in this case (well, obviously because nothing was actually expected). And this gets us to a more serious problem - nothing was actually tested in a test case - it was passing with no expectations made. We were getting a false sense of what was tested.

The Question:

I want to catch these mistakes as fast as possible. How do you think I should handle the problem?

Thoughts:

  • somehow fail a test case if there was no expectations made in it (not sure if jasmine has anything like this built-in)
  • "patch" the expect() and issue a warning/raise an error if nothing was called on the "expect" part
  • use static code analysis - define a custom eslint rule

Solution

  • The custom ESLint rule provided in the answer is now a part of eslint-plugin-jasmine 1.6.0:


    Old Answer:

    Here is a custom ESLint rule I've ended up with:

    module.exports = function (context) {
      return {
        // checking "expect()" arguments
        CallExpression: function (node) {
          if (node.callee.name === 'expect') {
            if (node.arguments.length > 1) {
              context.report(node, 'More than one argument passed to expect()')
            } else if (node.arguments.length === 0) {
              context.report(node, 'No arguments passed to expect()')
            }
          }
        },
    
        // nothing called on "expect()"
        'CallExpression:exit': function (node) {
          if (node.callee.name === 'expect' && node.parent.type === 'ExpressionStatement') {
            context.report(node, 'Nothing called on expect()')
          }
        }
      }
    }
    

    It checks for 3 things:

    • more than 1 argument passed to expect()
    • no arguments are passed to expect()
    • there was nothing called on expect()

    Here are the sample invalid expect() usages it currently catches:

    expect(page.filters.fromDateLabel.getText(), "After");
    expect("After");
    expect();
    

    As for the option #1, there is actually a quite related and useful ESLint rule being already implemented and open-sourced by [eslint-plugin-jasmine]: