For running automated tests in a C++ application, I would like the application to dump an image and compare it against a baseline image. I saw several examples of this on various CDash dashboards, e.g. this one (link might not be valid for long). https://open.cdash.org/testDetails.php?test=660365465&build=5407474
My google-fu has failed me on this one, what is the correct way to get this functionality?
The easiest way to attach ordinary files to test results is by listing those files in either the ATTACHED_FILES
or ATTACHED_FILES_ON_FAIL
test properties. This is not the mechanism being used here though.
According to this mailing list post, you can output special contents like that shown below to the test's stdout and it results in the named files being uploaded. The sample CDash results page you linked to follows a similar pattern as the example from the mailing list, which I've reproduced here for reference (I've made one small correction to change DifferenceImage
to DifferenceImage2
):
<DartMeasurement name="BaselineImage" type="text/string">Standard</DartMeasurement>
<DartMeasurementFile name="TestImage" type="image/png">C:/Users/.../Testing/Temporary/BoxWidget.png</DartMeasurementFile>
<DartMeasurementFile name="DifferenceImage2" type="image/png">C:/Users/.../Testing/Temporary/BoxWidget.diff.png</DartMeasurementFile>
<DartMeasurementFile name="ValidImage" type="image/png">C:/Users/.../VTKData/Baseline/Widgets/BoxWidget.png</DartMeasurementFile>
I've checked through the CTest source code and it scans the test output looking for <DartMeasurement>
and <DartMeasurementFile>
tags here and here. These are uploaded as discrete measurement items to CDash, which also looks for these particular names and presents them specially as in the example CDash links in the question.