Skip to content

Conversation

@mpkorstanje
Copy link
Contributor

@mpkorstanje mpkorstanje commented Nov 17, 2025

When using JUnit.start the pruneStackTrace algorithm immediately sees the HelloTest.main frame and assumes that this is the test method because the test class name matches.

org.opentest4j.AssertionFailedError: expected: <11> but was: <12>
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:158)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:139)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:201)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:152)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:147)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:558)
	at com.examp.project/com.example.project.HelloTest.stringLength(HelloTest.java:14)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
...
	at org.junit.platform.launcher@6.1.0-SNAPSHOT/org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:81)
	at org.junit.start@6.1.0-SNAPSHOT/org.junit.start.JUnit.run(JUnit.java:63)
	at org.junit.start@6.1.0-SNAPSHOT/org.junit.start.JUnit.run(JUnit.java:37)
	at com.examp.project/com.example.project.HelloTest.main(HelloTest.java:9)

By checking if org.junit.start is involved further down the stack we exclude this scenario.


I hereby agree to the terms of the JUnit Contributor License Agreement.


Definition of Done

mpkorstanje added a commit that referenced this pull request Nov 17, 2025
When using `JUnit.start` and creating a failing test, users
will be confronted with a large stacktrace with mostly irrelevant
information. Even after #5158 is merged, the stacktrace will contain
several internal frames:

```
org.opentest4j.AssertionFailedError: expected: <11> but was: <12>
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:158)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:139)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:201)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:152)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:147)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:558)
	at com.examp.project/com.example.project.HelloTest.stringLength(HelloTest.java:14)
```

By pruning these internal frames, the stacktrace can be reduced to a
much more readable:

```
org.opentest4j.AssertionFailedError: expected: <11> but was: <12>
	at com.examp.project/com.example.project.HelloTest.stringLength(HelloTest.java:14)
```

Comparable behaviour can be found in AssertJ[1] and IDEA which folds
internal frames in the console using `<6 internal line>`.

The pruning functionality is intentionally added to the
`AssertionFailureBuilder` rather than the `ExceptionUtils` to enable
other users to also prune the internal frames from their own assertions.

1. https://github.com/assertj/assertj/blob/79bdebf1817692e5e0ff5ee3ab097dcd104d47ae/assertj-core/src/main/java/org/assertj/core/util/Throwables.java#L117-L148
@mpkorstanje mpkorstanje force-pushed the fix/prune-junit-start branch from bc68cc8 to db66072 Compare November 18, 2025 00:01
When using `JUnit.start` the `pruneStackTrace` algorithm immediately
sees the `TestClass.main` frame and assumes that this is the test
method because the test class name matches.

```
org.opentest4j.AssertionFailedError: expected: <11> but was: <12>
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:158)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:139)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:201)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:152)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:147)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:558)
	at com.examp.project/com.example.project.HelloTest.stringLength(HelloTest.java:14)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
...
	at org.junit.platform.launcher@6.1.0-SNAPSHOT/org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:81)
	at org.junit.start@6.1.0-SNAPSHOT/org.junit.start.JUnit.run(JUnit.java:63)
	at org.junit.start@6.1.0-SNAPSHOT/org.junit.start.JUnit.run(JUnit.java:37)
	at com.examp.project/com.example.project.HelloTest.main(HelloTest.java:9)
```

By checking if `org.junit.start` is involved further down the stack we
exclude this scenario.
@mpkorstanje mpkorstanje force-pushed the fix/prune-junit-start branch from db66072 to a75d16f Compare November 18, 2025 00:04
@mpkorstanje mpkorstanje added this to the 6.1.0-M2 milestone Nov 18, 2025
mpkorstanje added a commit that referenced this pull request Nov 18, 2025
When using `JUnit.start` and creating a failing test, users
will be confronted with a large stacktrace with mostly irrelevant
information. Even after #5158 is merged, the stacktrace will contain
several internal frames:

```
org.opentest4j.AssertionFailedError: expected: <11> but was: <12>
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:158)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:139)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:201)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:152)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:147)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:558)
	at com.examp.project/com.example.project.HelloTest.stringLength(HelloTest.java:14)
```

By pruning these internal frames, the stacktrace can be reduced to a
much more readable:

```
org.opentest4j.AssertionFailedError: expected: <11> but was: <12>
	at com.examp.project/com.example.project.HelloTest.stringLength(HelloTest.java:14)
```

Comparable behaviour can be found in AssertJ[1] and IDEA which folds
internal frames in the console using `<6 internal line>`.

The pruning functionality is intentionally added to the
`AssertionFailureBuilder` rather than the `ExceptionUtils` to enable
other users of the builder to also prune the internal frames from their
own assertions.

1. https://github.com/assertj/assertj/blob/79bdebf1817692e5e0ff5ee3ab097dcd104d47ae/assertj-core/src/main/java/org/assertj/core/util/Throwables.java#L117-L148
mpkorstanje added a commit that referenced this pull request Nov 18, 2025
When using `JUnit.start` and creating a failing test, users
will be confronted with a large stacktrace with mostly irrelevant
information. Even after #5158 is merged, the stacktrace will contain
several internal frames:

```
org.opentest4j.AssertionFailedError: expected: <11> but was: <12>
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:158)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:139)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:201)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:152)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:147)
	at org.junit.jupiter.api@6.1.0-SNAPSHOT/org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:558)
	at com.examp.project/com.example.project.HelloTest.stringLength(HelloTest.java:14)
```

By pruning these internal frames, the stacktrace can be reduced to a
much more readable:

```
org.opentest4j.AssertionFailedError: expected: <11> but was: <12>
	at com.examp.project/com.example.project.HelloTest.stringLength(HelloTest.java:14)
```

Comparable behaviour can be found in AssertJ[1] and IDEA which folds
internal frames in the console using `<6 internal line>`.

The pruning functionality is intentionally added to the
`AssertionFailureBuilder` rather than the `ExceptionUtils` to enable
other users of the builder to also prune the internal frames from their
own assertions.

1. https://github.com/assertj/assertj/blob/79bdebf1817692e5e0ff5ee3ab097dcd104d47ae/assertj-core/src/main/java/org/assertj/core/util/Throwables.java#L117-L148
@marcphilipp marcphilipp requested a review from sormuras November 18, 2025 18:44
Copy link
Member

@sormuras sormuras left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copied from junit-team/junit-examples#656 (comment)

Here, I'd like to keep some detailed information about where the control flow was and where the exception was generated. Hiding too much details hinders to understand which part of the software was in control.

What does the stacktrace shown in the initial description look like when this pruning is applied?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants