Search code examples
c#tfsazure-devopsmstestmicrosoft-test-manager

functional testing with TFS

i'm looking to perform functional tests with a special app we build. that app operates all sorts of embedded functionality and i need to be able to build test cases that perform these actions as a scenario.

we thought we could simply use the TFS API to get info and just write back test runs and their results but it proved to be difficult task to do.

so we researched the "associated automation" feature inside test cases, but it seems that i need a special framework for this. i was told only unit testing frameworks such as xunit nunit and mstest can be integrated. i need functional testing, scenarios that are more complicated than a unit test.

do u have any ideas? on how can i simply run my own tests and update the TFS with runs that i created?

Solution

  • If your test tool can generate a JUnit, XUnit or TRX compatible file that contains the test results, then the data can be ingested using the "Publish Test Results" task in your build and/or release pipeline.

    If you want a wrapper around any executable, then the "Generic Tests" feature of MsTest may also be an option. These configure how to run your executable and then point to a result file for reporting purposes. A sample result file is shown here in the docs.

    <?xml version="1.0" encoding="utf-8" ?>
<SummaryResult>
    <TestName>ParentTest</TestName>
    <TestResult>Passed</TestResult>
    <InnerTests>
        <InnerTest>
            <TestName>InnerTest1</TestName>
            <TestResult>Passed</TestResult>
            <ErrorMessage>Everything is fine.</ErrorMessage>
            <DetailedResultsFile>D:\Documents and Settings\Results.txt</DetailedResultsFile>
        </InnerTest>
        <InnerTest>
            <TestName>InnerTest2</TestName>
            <TestResult>Failed</TestResult>
            <ErrorMessage>Something went wrong.</ErrorMessage>
            <DetailedResultsFile>D:\Documents and Settings\Results.txt</DetailedResultsFile>
        </InnerTest>
    </InnerTests>
</SummaryResult>

    Alternatively, test results can be created directly through the REST API: