Search code examples
specflowmicrosoft-test-managermtm

Select test cases by SpecFlow tag in Microsoft Test Manager

Is it possible to select test cases by SpecFlow tag in Microsoft Test Manager? And if so, how?

Solution

  • Tags in SpecFlow get translated into TestCategory attributes in the generated code. As far as I know (worked with MTM until a year ago) you can:

    1. Execute tests with MSTest filtered on TestCategory by Categories
    2. Execute tests with VSTest.Console filtered on TestCategory by TestCaseFilter
    3. Import test cases into MTM filtered on TestCategory

    With the last option, you can create with a bit of creativity a set of testplans with different tests sorted by test category. And I am afraid that is the best you can do without writing your own wrapper around MTM. The TestCategory information is available in TFS, but not exposed to the user in MTM.

    EDIT

    To clear things up after the comment.
    Given this feature file:

    @timtim
Feature: MSTest Tags Test
    In order to get feedback at the right time
    As a test automation specialist
    I want to categorize my tests and run them accordingly

@Jabberwocky @manxome
Scenario: A test with tags
    Given 'twas brillig
    When gyre and gimble in the wabe
    Then all mimsy were the borogoves

    It gets generated to this code:

    [Microsoft.VisualStudio.TestTools.UnitTesting.TestMethodAttribute()]
[Microsoft.VisualStudio.TestTools.UnitTesting.DescriptionAttribute("A test with tags")]  
[Microsoft.VisualStudio.TestTools.UnitTesting.TestPropertyAttribute("FeatureTitle", "MSTest Tags Test")]
[Microsoft.VisualStudio.TestTools.UnitTesting.TestCategoryAttribute("timtim")]
[Microsoft.VisualStudio.TestTools.UnitTesting.TestCategoryAttribute("Jabberwocky")]
[Microsoft.VisualStudio.TestTools.UnitTesting.TestCategoryAttribute("manxome")]
public virtual void ATestWithTags()
{
    TechTalk.SpecFlow.ScenarioInfo scenarioInfo = new TechTalk.SpecFlow.ScenarioInfo("A test with tags", new string[] {
                "Jabberwocky",
                "manxome"});
    #line 8
    this.ScenarioSetup(scenarioInfo);
    #line 9
    testRunner.Given("\'twas brillig", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "Given ");
    #line 10
    testRunner.When("gyre and gimble in the wabe", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "When ");
    #line 11
     testRunner.Then("all mimsy were the borogoves", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "Then ");
    #line hidden
    this.ScenarioCleanup();
}

    The scenario becomes a (by MSTest.exe) executable testmethod with three TestCategories: timtim, Jabberwocky and manxome. These are the same testcategories as mentioned in the articles. Coded UI does have a Test Category property that can be used to order the tests, but this category boiles down to using the same TestCategory attribute.