I need to use pytest-stress to run tests for a period of time. Previously, I was using pytest-repeat which is based on the number of iterations. I am using pytest-html for reporting results.
When I switched to stress from repeat, all iterations of the test would all appear inside 1 test row. Repeat was able to separate the tests into different iterations. I am trying to get stress to just list the test and the current iteration count in a separate row for pytest-html.
I know that pytest-stress is overriding pytest_runtestloop so it should be running at the session scope level.
I did try adding in some of the functionality from pytest_generate_tests that pytest-repeat overrides because it runs at the function scope level.
I would like to for the results to report each iteration separately (I removed the preceding path for readability here)
ex:
test_report.py::TestReport::test_fail[1]
test_report.py::TestReport::test_fail[2]
test_report.py::TestReport::test_fail[3]
pytest-repeat.png pytest-stress.png
Example Code:
import logging
class TestReport:
def test_pass(self):
logging.info("assert(True)")
assert(True)
def test_fail(self):
logging.info("assert(False)")
assert(False)
Conftest.py
def pytest_html_results_table_header(cells):
cells.insert(2, html.th("Time", class_="sortable time", col="time"))
cells.pop()
def pytest_html_results_table_row(report, cells):
cells.insert(2, html.td(datetime.utcnow(), class_="col-time"))
cells.pop()
@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item, call):
outcome = yield
report = outcome.get_result()
setattr(report, "duration_formatter", "%H:%M:%S.%f")
@pytest.hookimpl(tryfirst=True)
def pytest_configure(config):
if not os.path.exists('results'):
os.makedirs('results')
config.option.htmlpath = 'results/results_'+datetime.now().strftime("%Y-%m-%d_T%H-%M-%S")+".html"
#change name based on iteration
def pytest_itemcollected(item):
cls = item.getparent(pytest.Class)
iter = 0
if hasattr(cls.obj, "iteration"):
iter = int(getattr(cls.obj, "iteration"))
item._nodeid = item._nodeid + f"[{iter}]"
I do not know if this is possible through a minor edit to conftest.py or if I would have to create my own plugin for pytest to run based on a period of time.
Created my own pytest plugin called pytest-loop. It merges pytest-stress and pytest-repeat with a fix to clear previous reports.