Add --no-subtest-reports CLI opt

[giampaolo]

See #198.

[nicoddemus]

Great work @giampaolo!

Could you please also add a test case using the option, and ensuring that the subtests do not show up in the output? It should also show subtests if they fail or skip, regardless of the option. Feel free to add more than one test.

Thanks!

[giampaolo]

I've tried by installing my local branch with python3 -m pip install -e . (editable / dev mode) but it keeps using the module installed at system level. :-\

The way I run the new test is:

 python3 -m pytest --tb=short --no-header -v -s  tests/test_subtests.py::TestFixture::test_xxx

...and the draft of the new test looks like:

diff --git a/tests/test_subtests.py b/tests/test_subtests.py
index bd5cef9..307953d 100644
--- a/tests/test_subtests.py
+++ b/tests/test_subtests.py
@@ -156,6 +156,31 @@ class TestFixture:
         expected_lines += ["* 1 passed *"]
         result.stdout.fnmatch_lines(expected_lines)
 
+    def test_xxx(self, pytester: pytest.Pytester, mode: Literal["normal", "xdist"]):
+        pytester.makepyfile(
+            """
+            import pytest
+
+            def test_foo(subtests):
+                for i in range(5):
+                    with subtests.test(msg="custom", i=i):
+                        if i % 2 == 0:
+                            pytest.fail('even number')
+        """
+        )
+        expected_lines = [
+            "[custom] (i=0) SUBFAIL test_xxx.py::test_foo - Failed: even number",
+            "[custom] (i=2) SUBFAIL test_xxx.py::test_foo - Failed: even number",
+            "[custom] (i=4) SUBFAIL test_xxx.py::test_foo - Failed: even number",
+        ]
+        # result = pytester.runpytest("-v")
+        # result.stdout.fnmatch_lines(expected_lines)
+
+        result = pytester.runpytest("--no-subtests-reports")
+        print(result.stdout.str())
+        # breakpoint()  # XXX
+        # result.stdout.fnmatch_lines(expected_lines)
+
 
 class TestSubTest:
     """

[nicoddemus]

Good practice is to always use a virtual env:

python3 -m venv .venv
source .venv/bin/activate
pip install -e .
pytest --tb=short --no-header -v -s  tests/test_subtests.py::TestFixture::test_xxx

This way you never install anything in your system Python.

[giampaolo]

Sorry for the delay. I added a test case. Could you please check if it makes sense?

[nicoddemus]

Hi @giampaolo, I will get to your PR this week!

[nicoddemus]

Sorry for the delay. I added a test case. Could you please check if it makes sense?

Thanks! The problem with the test you added is that the test still passed even without your modifications. Tests for a certain functionality should fail if you remove the code that implements the functionality, otherwise they don't serve the purpose to prevent regressions.

I took the liberty of improving it and pushing. 👍