Beginner woes - test run of single test executable does not finish?

[bilbothebaggins]

I have a little project with one Test executable, and wanted to try out the test adapter.

First, I had a hard time getting the discovery to work. My Test executables are commonly named ".Test.exe" and only after fiddling around I discovered (ha!) that I need to specify <FilenameFilter>\.Test$</FilenameFilter> with the $ end marker, or otherwise it wouldn't find my test executable.

After that, discovery works, with the .runsettings file co-located with the solution, after I do a full solution rebuild.

Test discovery finished: 3 Tests found in 417ms

When I run this test executable, it finishes instantly and reports 76 successful assertions.

When I try to run the tests in th test explorer via Run, it hangs forever.

The command line as reported by Process Explorer is:

"D:\myprj\bin\Debug\x64-vc143\mytst.Test.exe" --reporter xml --durations yes --input-file "D:\myprj\bin\Debug\x64-vc143\mytst.Test.exe.testcaselist.638864573819662252" --out "D:\myprj\bin\Debug\x64-vc143\mytst.Test.exe.report.638864573819662252.xml"

the content of the input file is like this:

# testcase list generated by Test Adpater for Catch2
"mytsts priorities"
"mytsts Default Blackbox Tests"
"mytsts Priority Tests"

... this matches my test names.

Output File hangs at line 18:

<?xml version="1.0" encoding="UTF-8"?>
<Catch2TestRun ... ... 
  <TestCase name="-redact- Default Blackbox Tests" filename="D:\-redact-\prj\mytst.Test\-redact-.Test.cpp" line="6">
    <Section name="ActivateAdd" filename="D:\-redact-\prj\mytst.Test\-redact-.Test.cpp" line="10">
      <Section name="Deactivate/Remove after add" filename="D:\-redact-\prj\mytst.Test\-redact-.Test.cpp" line="31">
XXX ... it is only written up to here.

Now, when I stop the test in the Test Runner GUI with the Stop 🟥 Button, I noticed that after refreshing the result file in Notepad++ it is actually now written to completion, and contains successfully run 76 test cases:

... ...
  </TestCase>
  <OverallResults successes="76" failures="0" expectedFailures="0" skips="0"/>
  <OverallResultsCases successes="3" failures="0" expectedFailures="0" skips="0"/>
</Catch2TestRun>

When I run that command line manually in cmd.exe in runs to completion successfully.

[bilbothebaggins]

I should note that these tests have quite a lot Console interaction (output / colors) but since they run to completion without problem when using the equivalent command line that the Test Runner uses, there still must be some peculiarity from within VS?

I have now disabled SetConsoleTextAttributeso no color control is done anymore.

I have narrowed it down to a for loop that prints a bunch of numbers in a large-ish table.
The printing is simply done via printf, so special shenanigans here.
When I disable the innermost printf, the test runs to completion. (it's just a printf("%03d ", 0)

So it seems there is something going out with the stdout interaction, but unfortunately I have to leave it at this for now.

[bilbothebaggins]

OK. I have now managed to reproduce this the following rather trivial test case:

TEST_CASE("WTF_3") {
	const char* const ten_char = "0123456789";
	// 341 = ok
	// 342 = hangs
	for (int i=0; i != 342; ++i) {
		printf("%s\n", ten_char);
	}
}

If I print 10 digits 341 times (or less), the test case runs through in test explorer.
Printing them 342 times (or more) hangs the text explorer. (runs fine on command line)

I can reproduce it with my VS 2022 17.8.5 on both Windows 11 and Windows 10.
Test Adapter for Catch2 Version is 2.0.0