Search code examples
c#.netcommand-line-interfacecommand-promptspecflow

How to run Specflow feature/scenarios in cmd using vstest.console.exe?


I have a feature file defined like

Feature: Portal Sign in
    In order to login to my portal
    As a User
    I have input user id and password

Scenario: User Login
    Given I have entered username and password 
    When The user clicks on Login button
    Then logged in successfully pop up message should be displayed

I tried this cmd:

vstest.console.exe tests.dll /TestCaseFilter:"FullyQualifiedName~PortalSigninFeature"

but it returns with no test matches for the given test case filter. Am I missing anything?

Also how do you suggest to run scenarios using cmd?

Edit:

In case of scenario Outline like the one below

Scenario Outline: Multiple User Login
    Given I have entered <username> and <password>
    When The user clicks on Login button
    Then logged in successfully pop up message should be displayed

Examples:
|username|password|
|User1   |pwd1    |
|User2   |pwd2    |

When I run the below cmd

SpecRun.exe run D:\SpecFlow\bin\Debug\MySpecFlowTests.dll --filter testpath:"Scenario:Multiple+User+Login"

Discovered tests appear as 0.

Should I change anything to make it work for scenario outline?


Solution

  • When I ran this in jenkins from a command line I had the below. I am using SpecRun for a runner. This did require a RunSettings file.

    cd "C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\CommonExtensions\Microsoft\TestWindow" vstest.console.exe /settings:E\MyPath\To\RunSettings\ E:\MyPathTo\Test.dll /ResultsDirectory:E:\MyPath\ /TestCaseFilter: "(TestCategory=MyTag)"
    

    I created a folder called Jenkins and the RunSettings and .srprofile both resided in that folder.

    RunSettings file pointed to the .srprofile

    <RunSettings>
      <!-- Configurations for SpecFlow+ Runner -->
      <SpecRun>
       <Profile>Jenkins/MyTest.srprofile</Profile>
       <GenerateSpecRunTrait>false</GenerateSpecRunTrait>
      <GenerateFeatureTrait>false</GenerateFeatureTrait>
      </SpecRun>
    </RunSettings>
    

    If you are using SpecRun as a runner, you can also run this from a cmd line. Update the runner version accordingly.

    cd E:\Path\to\packages\SpecRun.Runner.3.3.*\tools\net461
    SpecRun.exe run PathTo/My.srprofile --baseFolder E:\Path\To\bin\Debug --filter "@TagL" --log specrun.log
    

    To run by a Feature Name, you would use:

    SpecRun.exe run D:\Path\Jenkins\My.srprofile --baseFolder D:\Path\bin\Debug --filter testpath:"Feature:MyFeature*" --log specrun.log
    

    wild card will match anything starting with "MyFeature".

    https://docs.specflow.org/projects/specflow-runner/en/latest/Profile/Filter.html