Search code examples
flutterazureunit-testingazure-pipelines

Flutter-Azure Deviops pipelineThe discrepancy between the number of tests reported as passed or failed in the pipeline output and the test report

I am trying to build Flutter Android pipeline using Azure Devops using classic editors. I have a task for Flutter test which gives coverage report and publish it as well. Coverage report is made correctly, but my Test Cases passed/failed report seems wrong.

Attached are the snapshots Test Cases report after running task and Test Report

enter image description here

Why there is discrepancy in the number of tasks. I have 96 passed and one failed but report shows 134 passed.

Kindly help me with this. I might be doing something wrong.

Link referred- https://medium.com/bina-nusantara-it-division/integrate-flutter-code-coverage-report-to-azure-devops-b5d55f457be4

Updated : junit.xml

<testsuite name="auth_model_test.dart" timestamp="2024-02-05T05:11:54.823Z" errors="0" skipped="0" failures="0" tests="7">
    <testcase name="loading D:/a/1/s/ISFV2/ericsson_mobile_isf/test/features/authentication/datamodel/auth_model_test.dart" classname="loading D:/a/1/s/ISFV2/ericsson_mobile_isf/test/features/authentication/datamodel/auth_model_test.dart" time="14.657"/>
    <testcase name="AuthModelNotifier Tests Set AuthModel" classname="AuthModelNotifier Tests Set AuthModel" time="0.029"/>
    <testcase name="AuthModelNotifier Tests Clear AuthModel" classname="AuthModelNotifier Tests Clear AuthModel" time="0"/>
    <testcase name="AuthModel Tests AuthModel toMap" classname="AuthModel Tests AuthModel toMap" time="0"/>
    <testcase name="AuthModel Tests AuthModel fromMap" classname="AuthModel Tests AuthModel fromMap" time="0"/>
    <testcase name="AuthModel Tests AuthModel toJson" classname="AuthModel Tests AuthModel toJson" time="0"/>
    <testcase name="AuthModel Tests AuthModel fromJson" classname="AuthModel Tests AuthModel fromJson" time="0"/>
  </testsuite>
  <testsuite timestamp="2024-02-05T05:11:54.823Z" errors="0" skipped="0" failures="0" tests="1">
    <testcase name="Some tests failed." classname="Some tests failed." time="0"/>
  </testsuite>

above is some data from junit.xml file. I see there are test cases added like

testcase name with loading... or some test failed are increasing the counts I guess. But I don't know why this is happening

Solution

  • When I use the Flutter extension published by Hey24sheep, I will have the same issue sometime. In my test, there are 8 tests in total, but there are only 6 in the test result. enter image description here

    When I use the Flutter extension published by Aloïs, everything is working fine so far. enter image description here