Search code examples
c#testingspecflow

Analysing SpecFlow duration metrics


I have some SpecFlow tests that provide test output like this:

Given my location is set to secret
-> done: Location.GivenMyLocationIsSetTo("secret") (0.0s)
And the date today is 03 Jun 2017 15:00
-> done: Date.GiveThatTheEffectiveDateIs(2017-06-03 3:00:0...) (0.0s)
And I have selected a client with the following properties
  --- table step argument ---
  | property                  | value      |
  | SomeAttr                  | someval    |
  | SomeAttr2                 | someval2   |
-> done: Client.GivenIHaveSelectedAClientWithTheFollowingProperties(<table>) (0.0s)
And I select a currency of CAD
-> done: Currency.WhenISetCurrency("CAD") (0.0s)
Then I see the following messages
  --- table step argument ---
  | classification | message       |
  | Class1         | Hello world   |
-> done: Message.ThenISeeTheFollowingMessages(<table>) (2.9s)

I have noticed that for all of my tests, the final step is the one that takes all the time (i.e. 2.9s in above - see the bottom line - all previous steps add up to zero).

I have many thousands of test cases and they take many hours to run, which has become a problem.

  • Is this normal or to be expected?
  • Does this duration metric include tear-down of the test harness itself (or is it purely the code-undertest)?
  • Is there any tips on how I could go about analysing that duration and breaking it down further?

Note:

  • I am not using Selenium
  • The entire test takes 2.9s - 100% of which appears to be in that last step

Solution

  • The closest I could come to to getting metrics within the step was to step through the code in debug mode.

    I found I was able to put breakpoints in the .feature file and Visual Studio would break there. Also was able to put breakpoints in my code using the usual methods. Visual Studio PerfTips help indicate slow parts of the code.