how to upload generic test execution report in a sonarqube server?

On an application compound by a set of micro services, I used JMeter to perform API tests and users scenario testing (see here).

I would like to manage these tests statuses and durations in sonarqube and for that to upload these tests results in sonarqube.

Producing a generic test execution report is easy, the format is quite simple.( see here )

<testExecutions version="1">
  <file path="testx/ClassOneTest.xoo">
    <testCase name="test1" duration="5"/>
    <testCase name="test2" duration="500">
      <skipped/>
    </testCase>
    <testCase name="test3" duration="100">
      <failure/>
    </testCase>
    <testCase name="test4" duration="500">
      <error/>
    </testCase>
  </file>
</testExecutions>

But the documentation explains "Unlike for coverage reports, the files present in the report must be test file names, not source code files covered by tests.".

But in the context of generic test execution report what is a test file name?

JMX files are not JUnit files, if I set the element like <file path="jmx/api/scenario1.jmx"> will the sonarqube server accept it as a test file name ?

The version of Sonarqube is sonarqube-10.2.1.78527.

The version of the sonar-scanner is 5.0.1 .

Solution

Short answer - referencing jmx in the report won't work, but workaround exists

Now, answering your questions...

what is a test file name?

Practically, SonarQube differentiates files by two scopes - sources and tests. Any file which is in 'tests' scope (defined by sonar.tests parameter) is considered as a test file. In the context of generic test execution report 'a test file name' means the same - it is a file defined in the 'tests' scope. See more detales on scopes

will the sonarqube server accept it as a test file name ?

The type of the referenced file matters. SonarQube tries to detect file language and it will fail to do so for '.jmx' (at least in SonarQube CE 9.9). So your "jmx/api/scenario1.jmx" will most likely end up with an error Skipping file ... in the generic test execution report because it doesn't have a known language even if you define this file as part of the 'tests' scope. One possible workaround would be to rename .jmx to .xml. That way language check will pass.