-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Prune org.junit.start from exceptions #5158
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
When using `JUnit.start` and creating a failing test, users will be confronted with a large stacktrace with mostly irrelevant information. Even after #5158 is merged, the stacktrace will contain several internal frames: ``` org.opentest4j.AssertionFailedError: expected: <11> but was: <12> at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:158) at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:139) at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:201) at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:152) at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:147) at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:558) at com.examp.project/com.example.project.HelloTest.stringLength(HelloTest.java:14) ``` By pruning these internal frames, the stacktrace can be reduced to a much more readable: ``` org.opentest4j.AssertionFailedError: expected: <11> but was: <12> at com.examp.project/com.example.project.HelloTest.stringLength(HelloTest.java:14) ``` Comparable behaviour can be found in AssertJ[1] and IDEA which folds internal frames in the console using `<6 internal line>`. The pruning functionality is intentionally added to the `AssertionFailureBuilder` rather than the `ExceptionUtils` to enable other users to also prune the internal frames from their own assertions. 1. https://github.com/assertj/assertj/blob/79bdebf1817692e5e0ff5ee3ab097dcd104d47ae/assertj-core/src/main/java/org/assertj/core/util/Throwables.java#L117-L148
bc68cc8 to
db66072
Compare
When using `JUnit.start` the `pruneStackTrace` algorithm immediately sees the `TestClass.main` frame and assumes that this is the test method because the test class name matches. ``` org.opentest4j.AssertionFailedError: expected: <11> but was: <12> at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:158) at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:139) at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:201) at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:152) at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:147) at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:558) at com.examp.project/com.example.project.HelloTest.stringLength(HelloTest.java:14) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) ... at org.junit.platform.launcher@6.1.0-SNAPSHOT/org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:81) at org.junit.start@6.1.0-SNAPSHOT/org.junit.start.JUnit.run(JUnit.java:63) at org.junit.start@6.1.0-SNAPSHOT/org.junit.start.JUnit.run(JUnit.java:37) at com.examp.project/com.example.project.HelloTest.main(HelloTest.java:9) ``` By checking if `org.junit.start` is involved further down the stack we exclude this scenario.
db66072 to
a75d16f
Compare
When using `JUnit.start` and creating a failing test, users will be confronted with a large stacktrace with mostly irrelevant information. Even after #5158 is merged, the stacktrace will contain several internal frames: ``` org.opentest4j.AssertionFailedError: expected: <11> but was: <12> at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:158) at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:139) at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:201) at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:152) at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:147) at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:558) at com.examp.project/com.example.project.HelloTest.stringLength(HelloTest.java:14) ``` By pruning these internal frames, the stacktrace can be reduced to a much more readable: ``` org.opentest4j.AssertionFailedError: expected: <11> but was: <12> at com.examp.project/com.example.project.HelloTest.stringLength(HelloTest.java:14) ``` Comparable behaviour can be found in AssertJ[1] and IDEA which folds internal frames in the console using `<6 internal line>`. The pruning functionality is intentionally added to the `AssertionFailureBuilder` rather than the `ExceptionUtils` to enable other users of the builder to also prune the internal frames from their own assertions. 1. https://github.com/assertj/assertj/blob/79bdebf1817692e5e0ff5ee3ab097dcd104d47ae/assertj-core/src/main/java/org/assertj/core/util/Throwables.java#L117-L148
When using `JUnit.start` and creating a failing test, users will be confronted with a large stacktrace with mostly irrelevant information. Even after #5158 is merged, the stacktrace will contain several internal frames: ``` org.opentest4j.AssertionFailedError: expected: <11> but was: <12> at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:158) at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:139) at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:201) at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:152) at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:147) at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:558) at com.examp.project/com.example.project.HelloTest.stringLength(HelloTest.java:14) ``` By pruning these internal frames, the stacktrace can be reduced to a much more readable: ``` org.opentest4j.AssertionFailedError: expected: <11> but was: <12> at com.examp.project/com.example.project.HelloTest.stringLength(HelloTest.java:14) ``` Comparable behaviour can be found in AssertJ[1] and IDEA which folds internal frames in the console using `<6 internal line>`. The pruning functionality is intentionally added to the `AssertionFailureBuilder` rather than the `ExceptionUtils` to enable other users of the builder to also prune the internal frames from their own assertions. 1. https://github.com/assertj/assertj/blob/79bdebf1817692e5e0ff5ee3ab097dcd104d47ae/assertj-core/src/main/java/org/assertj/core/util/Throwables.java#L117-L148
sormuras
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copied from junit-team/junit-examples#656 (comment)
Here, I'd like to keep some detailed information about where the control flow was and where the exception was generated. Hiding too much details hinders to understand which part of the software was in control.
What does the stacktrace shown in the initial description look like when this pruning is applied?
When using
JUnit.startthepruneStackTracealgorithm immediately sees theHelloTest.mainframe and assumes that this is the test method because the test class name matches.By checking if
org.junit.startis involved further down the stack we exclude this scenario.I hereby agree to the terms of the JUnit Contributor License Agreement.
Definition of Done
@APIannotations