Search code examples
testingamazon-s3junitcloudreporting

How to display a directory full of junit test result files


As part of testing in the cloud, I plan to copy my test results (I happen to be using gradle) as xml out of a short lived container to an s3 bucket.

I am wondering what is the best approach for making use of the bucket full of test results and if anyone has ideas for how to get the most of these results with the least custom code, etc.

In the past I have handled this with XSLT to make the XML into HTML but that approach feels like more work than I'd like to have to do.

Has anyone solved this issue for their own purposes that can share your approach?

As I side note... I could also copy out to my s3 bucket the HTML reports from gradle if someone has an idea about how to use the HTML instead.

Ideally, I would be able to see a sorted list of results and click on the most recent to see the test run details.


Solution

  • Disclaimer: I'm not a Devops guy, I'll present a "developer's" point of view here

    The answer IMO depends on what exactly would like to do with all these reports.

    In a nutshell, you can have text/xml reports from the test execution (I can't say for gradle but in maven surefire plugin creates these reports, I believe gradle does the same).

    You can also generate a good looking site with the reports information, there are tools for this as well, for example Allure Reports. Bottom line, some stuff to be shown can be generated in the build directory.

    Copying the reports is as simple as copying a file, but the question is what would like to do with them when they're in S3?

    In my understanding, its a Job for CI tools (like Jenkins) to run the tests during the build, or maybe a suit of automation tests against a deployed environment (in the cloud). These CI Tools run gradle to build the project, and run tests, then they show test results per build memorizing last N builds. It can also be integrated with tools like aforementioned allure and show the HTML report per build as long as the build is stored in CI tool.

    So I'm not sure I understand why would you like to "run tests in the cloud" (from which I assume you're running tests suite when the artifact is being deployed).

    As for possible ideas of what can be done with all these results:

    1. You can create a site that will show the results across builds (again, this is already solved by CI tool like Jenkins)

    2. You can store the results of running the tests in some database and provide some clever statistics (I personally don't see much benefit if usually the build is green and you're not in the process of "recuperation" of project from flaky tests or something)