Search code examples
iosswiftperformanceperformance-testing

How to make a performance test fail if it's too slow?

I'd like my test to fail if it runs slower than 0.5 seconds but the average time is merely printed in the console and I cannot find a way to access it. Is there a way to access this data?

Code

//Measures the time it takes to parse the participant codes from the first 100 events in our test data.
func testParticipantCodeParsingPerformance()
{
    var increment = 0
    self.measureBlock
    {
        increment = 0
        while increment < 100
        {
            Parser.parseParticipantCode(self.fields[increment], hostCodes: MasterCalendarArray.getHostCodeArray()[increment])
            increment++
        }
    }
    print("Events measured: \(increment)")
}

Test Data

[Tests.ParserTest testParticipantCodeParsingPerformance]' measured [Time, seconds] average: 0.203, relative standard deviation: 19.951%, values: [0.186405, 0.182292, 0.179966, 0.177797, 0.175820, 0.205763, 0.315636, 0.223014, 0.200362, 0.178165]

Solution

  • You need to set a baseline for your performance test. Head to the Report Navigator:

    Report Navigator

    and select your recent test run. You'll see a list of all your tests, but the performance ones will have times associated with them. Click the time to bring up the Performance Result popover:

    Performance Result

    The "Baseline" value is what you're looking for--set it to 0.5s and that will inform Xcode that this test should complete in half a second. If your test is more than 10% slower than the baseline, it'll fail!