pytest `--last-failed` does not seem to remember failed subtests

[chingc]

I'm experiencing an issue where it appears that --last-failed cannot remember a failed test if the failure occurs in a subtest.

As a result, instead of only re-running the failed test --last-failed will re-run all tests.

Additionally, no tests will run if --last-failed is used with --last-failed-no-failure=none.

Here is my test:

def test_fails(subtests):
    for i in range(0, 2):
        with subtests.test(msg="error", i=i):
            assert i == 1

def test_passes():
    assert True

Result with pytest -q

Looks good so far. This is expected output.

============================ FAILURES ============================
____________________ test_fails [error] (i=0) ____________________

    def test_fails(subtests):
        for i in range(0, 2):
            with subtests.test(msg="error", i=i):
>               assert i == 1
E               assert 0 == 1

test_subtest.py:4: AssertionError
==================== short test summary info =====================
FAILED test_subtest.py::test_fails - assert 0 == 1
1 failed, 2 passed, 2 warnings in 0.21s

Result with pytest -q --lf

It's re-running all tests including test_passes which didn't fail. This is unexpected.

============================ FAILURES ============================
____________________ test_fails [error] (i=0) ____________________

    def test_fails(subtests):
        for i in range(0, 2):
            with subtests.test(msg="error", i=i):
>               assert i == 1
E               assert 0 == 1

test_subtest.py:4: AssertionError
==================== short test summary info =====================
FAILED test_subtest.py::test_fails - assert 0 == 1
1 failed, 2 passed, 2 warnings in 0.25s

Result with pytest -q --lf --lfnf=none

This is the worst case because I know something failed and it didn't re-run anything!

2 deselected in 0.01s

[nicoddemus]

Thanks for the report @chingc!

[RonnyPfannschmidt]

i jsut hit this while trying to ease debugging in a codebase

[nicoddemus]

Yeah currently, if only sub-tests failed, then the parent test passes, so it is not considered by the last-failed plugin.

[bluetech]

When integrating into pytest core (pytest-dev/pytest#13738) we are intending to change the behavior to fail the parent test if a subtest fails. I verified that it makes --lf work.