Enhance JUnit test run visibility for method/platform/configuration

[vellotis]

As I was trying to figure out why WASI and C platform tests failed on my Mac, I felt a crying need to visualize which method+platform+configuration exactly failed. I changed the code for myself just enough so it would make my life easier. Decided to turn this into a PR presentable enhancement.

C platform tests are disabled in the screenshot test run.

Not sure if you plan to have another 0.12.x version or this PR should only be introduced into master.

Changes

• Refactor TeaVMTestRunner to maintain test descriptions and test results visibility across multiple platforms and configurations.

  New suite/test descriptors structure:

  TestClass(Suite)
  ├─  "Compile Platform+Configuration"(Tests)
  └─  Method(Suites)
      └─  Platform+Configuration(Tests)
  

• Introduced JVM as org.jvm.junit.TestPlatform value. It makes creating test Descriptiors more unified.

• Introduce visibility for compilation per platform/configuration

  • remove redundant failAllClasses. No point for running/failing/ignoring all other tests. Just skip. Compilation failure is already enough.
  • skip (even ignoring) tests that are related to TestClass, if whole class compilation fails

• If JVM test run is not skipped, but fails, all test method related tests are ignored.

• Changed generateName method to use [] instead of (), so it could be used also for test Description. () are already preserved for other means in test Descriptor display name. Change affects tests.json test name format.

[konsoletyper]

Does this work properly in IDEA?

[konsoletyper]

I also mean: if I click "run" next to a certain test method or class, will this approach work? If I do this in test result? AFAIK, usually such custom things aren't supported by IDEs well

[vellotis]

I digged areound a bit and found that

• IDEA walks the tree of getDescription() and collects just the leaf nodes, and structures the result as TEST-CLASS->LEAF-NODE hierarchy. When selecting to run a single test method it submits this specific method name pattern. eg. for org.teavm.classlib.java.io.FileTest selecting mkdir[JS,optimized] in IDEA it creates a gradle run configuration :tests:test --tests "org.teavm.classlib.java.io.FileTest.getName[JS,optimized]". There is no such method. But this will fix it. https://github.com/konsoletyper/teavm/pull/1058/files#diff-63e7c8b10e26c2912ba11e348b666b7177500743e924d1192e23057ead2cb073R982-R998

• In VSCode it captures the tests results under the TEST-CLASS->METHOD hierarchy. When selecting to run a specific method it uses a Descriptor tree matcher - if there is no method for the test Descriptor present it falls back to the parent Descriptor. So it actually does not need the fix defined for IDEA because it falls back to test suite descriptor that is a combination of CLASS+METHOD.

  

  I don't have the Mac related code changes in this branch and previously discussed (Running test suite on MacOS in TestContainers: "Segmentation fault" #1049) test fails. Hence, the FileTest shows as red.

To be said, currently it runs all of the enabled platform tests when selecting to run a specific <test>[<platform> ... ].