Skip to content

Conversation

@csutherl
Copy link
Member

Summary

This change introduces a Maven-style test profile system to our build.xml to allow for convenient, selective execution of test subsets without knowing all the class patterns to do the needful. It does this without creating any additional Ant targets that would pollute the -projecthelp output too.

Usage

$ ant test -Dtest.profile=smoke    # Run smoke tests
....

Profiles Introduced

  • Smoke: uses the pattern list from ci.yml which is used as the 'smoke tests' for each PR
  • Catalina and Coyote: these are are not all of the components, but listed to show that it's easy to do them. If we think they're not useful they can be dropped, or if we want to cover more components we can add them.
  • Performance: run all performance tests
  • Tribes: run all tribes tests, including the old ones that are known to be flaky that are excluded by default
  • Buildutil: This one I only included for completeness because it is always excluded, now there's an easy way to run it if you needed to.

I'd love to get some feedback on what other profiles might be useful here, or if the ones we have should be modified or eliminated.

Changes

  • The build.xml has been updated with a new macrodef and two internal targets to handle the profile logic, which were added to the test* targets as a dependency.
  • A new build.properties.test-profiles file was created to store the profile pattern definitions and keep the build.xml easier to read.
  • I updated the .github/workflows/ci.yml to use the new profile rather than the pattern list since I used the ci.yml pattern to create the smoke test profile.
  • And lastly, added a changelog entry documenting the change to the Other subsection.

The changes are fully backwards compatible by using test.entry takes precedence, its easy to add new profiles with a one condition needed in the build.xml and the profile pattern in the properties file, and the CI workflow is simplified and easier to maintain.

@csutherl
Copy link
Member Author

I forgot to mention that I tried using $$ expansion to eliminate the need to have the sequential logic in the build.xml and define everything in the properties file but that didn't work out and needed additional libraries to do it the way I wanted (antlib's property expansion library, specifically). Any thoughts on how to make the macro a bit nicer and more extendable would be great.

@rmaucher
Copy link
Contributor

The properties file with the profiles look really good to me.

@markt-asf
Copy link
Contributor

LGTM.

…ing -Dtest.profile=<name> without polluting the ant -projecthelp output.

Profile test patterns are in the a newly introduced build.properties.test-profiles file.
GitHub Actions CI configuration is also updated to use the 'smoke' profile since it was created using the ci.yml list as it's source.
@csutherl csutherl force-pushed the introduce-test-profile-system branch from ba832e7 to aea1647 Compare November 25, 2025 16:50
@csutherl
Copy link
Member Author

I updated the commit to change the build.properties.test-profiles to test-profiles.properties and fixed all the relevant references. I also add a few more property file settings so that the behavior of test-profiles.properties matches build.properties and build.properties.default.

The PR has been open for long enough without objection, so I'll go ahead and commit when I can.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants