If I am able to get the filename of the failed test, I could run the test a second time to check if it really is a error or just a coincidence. Until now I couldnt find a sulution.
You get the instance of test in _failed hook, so you can do the same thing as RunFailed extension:
use Codeception\Test\Descriptor;
public function _failed(\Codeception\TestInterface $test, $fail)
{
$fullName = Descriptor::getTestFullName($test);
}
Original answer:
Codeception has RunFailed extension, which is enabled by default.
It saves the names of all failed tests to tests/_output/failed
file, and failed tests can be executed by running codecept run -g failed
command.
In order to make CI pass if failed tests pass on second run you can use this command:
codecept run || codecept run -g failed