chore: Device/integration tests for mobile on .net10 by jamescrosswell · Pull Request #4750 · getsentry/sentry-dotnet

jamescrosswell · 2025-11-19T02:25:53Z

Resolves #4744

#skip-changelog

codecov · 2025-11-19T02:41:29Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.85%. Comparing base (dc9a036) to head (30cd0f6).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #4750   +/-   ##
=======================================
  Coverage   73.85%   73.85%           
=======================================
  Files         483      483           
  Lines       17578    17578           
  Branches     3464     3464           
=======================================
+ Hits        12982    12983    +1     
+ Misses       3742     3741    -1     
  Partials      854      854

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

jamescrosswell · 2025-11-21T00:07:06Z

Tricky... locally everything passes for me:

However in CI we've got 2 tests failing in Debug and 2 tests failing in Release:

...
2025-11-20T03:50:51.4905577Z ##[error][-] MAUI app (net10.0, Release).Native crash (Release) 21.13s (21.12s|3ms)�[0m
2025-11-20T03:50:51.4916597Z ##[group]Message�[0m
2025-11-20T03:50:52.4056556Z Expected a collection with size 1, but got collection with size 25
2025-11-20T03:50:52.7693005Z at <ScriptBlock>, /home/runner/work/sentry-dotnet/sentry-dotnet/integration-test/android.Tests.ps1:170�[0m
...
025-11-20T03:53:44.9710461Z ##[error][-] MAUI app (net10.0, Debug).Java crash (Debug) 10.9s (10.9s|1ms)�[0m
2025-11-20T03:53:44.9723309Z ##[group]Message�[0m,
2025-11-20T03:53:45.0186641Z Expected a collection with size 1, but got collection with size 2
2025-11-20T04:01:08.0804390Z at $result.Envelopes() | Should -HaveCount 1, /home/runner/work/sentry-dotnet/sentry-dotnet/integration-test/android.Tests.ps1:156�[0m
...
2025-11-20T03:58:20.3428226Z ##[error][-] MAUI app (net10.0, Release).Native crash (Release) 22.23s (22.23s|3ms)�[0m
2025-11-20T03:58:20.3429986Z ##[group]Message�[0m
2025-11-20T03:58:21.0057104Z Expected a collection with size 1, but got collection with size 28
2025-11-20T03:50:52.7693005Z at <ScriptBlock>, /home/runner/work/sentry-dotnet/sentry-dotnet/integration-test/android.Tests.ps1:170�[0m
...
2025-11-20T04:01:08.0369087Z ##[error][-] MAUI app (net10.0, Debug).Java crash (Debug) 9.43s (9.43s|1ms)�[0m
2025-11-20T04:01:08.0387771Z ##[group]Message�[0m
2025-11-20T04:01:08.0683949Z Expected a collection with size 1, but got collection with size 2
2025-11-20T04:01:08.0804390Z at $result.Envelopes() | Should -HaveCount 1, /home/runner/work/sentry-dotnet/sentry-dotnet/integration-test/android.Tests.ps1:156�[0m
...

Looking at the 2 envelopes received for the Java crashes (java_received.json) the only difference is the timestamp:

... so it looks like we're getting duplicate envelopes being sent for some reason
(but only on CI of course) 😕

jamescrosswell · 2025-11-26T02:57:59Z

@supervacuus wondering if you might have some insights. When bumping to .NET 10, our Android integration tests fail when creating a native crash...

2025-11-26T00:27:34.1540247Z Expected a collection with size 1, but got collection with size 16

Basically we have a single crash, so we're expecting a single envelope... but instead we get loads of envelopes (from sentry.native). We don't see the same behaviour in .net 9 (those tests pass fine)... so maybe a change in behaviour in terms of how .net 10 wraps signals 🤷🏻 although, if so, not one that's documented in the release notes. It's not always 16 envelopes - in previous runs it's been 26 or 28 etc. so basically just "way more than expected".

I can't reproduce this locally - it only happens to us in CI (which makes it pretty tricky to analyse).

It looks like maybe there are some tests that are already relevant in the native repo, so I created a PR to bump these to .NET 10 to see if we see the same issue there... although those are run on Linux Alpine (not Android) so not necessarily expecting the same results:

Bump dotnet signal tests to .net 10 sentry-native#1457

supervacuus · 2025-11-26T11:11:19Z

Hi @jamescrosswell 👋

Basically we have a single crash, so we're expecting a single envelope... but instead we get loads of envelopes (from sentry.native). We don't see the same behaviour in .net 9 (those tests pass fine)... so maybe a change in behaviour in terms of how .net 10 wraps signals 🤷🏻 although, if so, not one that's documented in the release notes. It's not always 16 envelopes - in previous runs it's been 26 or 28 etc. so basically just "way more than expected".

From what I have seen over the last year in the .NET runtime's "documentation" on signal usage, I would be surprised if a change in behavior in that area made its way into the release notes.

I don't have an immediate idea and it is rather hard to follow for me, because i don't know what your test does and which of those outputs are relevant.

However, intuitively, a variable number of envelopes rather than one looks like a signal loop (typically caused by an abort(), because it resets the signal mask before it raises a SIGABRT), where some function after serializing the envelope causes a crash in the handler.

But it's really hard to say for me from what I know about your tests. It would be interesting to see all envelopes rather than just their count.

I can't reproduce this locally - it only happens to us in CI (which makes it pretty tricky to analyse).

But does it repro on CI stably? Would it be possible to actions/upload-artifact the entire envelope cache? The JSON printed in the logs only shows a stackrace with a SIGILL and a single frame from the crash() function libsentrysupplemental.so.

It looks like maybe there are some tests that are already relevant in the native repo, so I created a PR to bump these to .NET 10 to see if we see the same issue there... although those are run on Linux Alpine (not Android) so not necessarily expecting the same results:

getsentry/sentry-native#1457

Yes, the .NET tests fail with that bump, but likely due to .NET 10 being an unsupported target (see my response).

I would also ping @jpnurmi on this topic.

This reverts commit 344c80a.

jamescrosswell · 2025-11-27T01:16:36Z

I would also ping @jpnurmi on this topic.

Yes normally. He's unfortunately unavailable until next year 😢

I don't have an immediate idea and it is rather hard to follow for me, because i don't know what your test does and which of those outputs are relevant.

That's fair... JP actually wrote the tests. They're supposed to be high level simple integration tests in pester/powershell that compile a .net app and run it, passing in an argument that determines what the app should do. For example, in the test that's failing here, the argument is "Native", which just means the app should throw a native exception. That will crash the app, so the test then runs the app (which crashes) then runs the app again (to ensure any crashes dumped to disk get handled) then checks to make sure an envelope containing the expected event was sent/received by the fake sentry server.

a variable number of envelopes rather than one looks like a signal loop (typically caused by an abort(), because it resets the signal mask before it raises a SIGABRT), where some function after serializing the envelope causes a crash in the handler.

Hm, that gives me some clues. After each test, the app gets shut down with this. That should shut down Sentry before calling KillProcess, but I think I see an issue with the implementation for cascading shutdown to other SDKs here... in the case of Android and iOS there is no cascading so the Java SDK and it's embedded Native SDK don't get explicitly shutdown.

I'll dig into that and report back - thanks @supervacuus 🙏🏻

jamescrosswell · 2025-11-27T23:58:45Z

So I tried cascading the SDK shutdown to the Java SDK:

sentry-dotnet/src/Sentry/Internal/Hub.cs

Line 874 in 278c04c

JavaSdk.Sentry.Close();

Unfortunately the Java SDK also doesn't cascade this to the instance of the native SDK 😢
https://github.com/getsentry/sentry-java/blob/fc5ccaf5844ae029d93c566b6fd1b4f5c2e4da83/sentry/src/main/java/io/sentry/Sentry.java#L715-L723

I guess it would be possible to rectify that in the Java SDK but that would require a series of new releases etc. I'll see if I can find an alternative workaround.

… tests

jamescrosswell · 2025-12-03T23:56:08Z

@Flash0ver in the latest test runs we're seeing:

Error: [-] MAUI app (net10.0, Release).Null reference exception (Release) 7.58s (7.57s|4ms)
Expected string '"type":"SIGSEGV"' to match no elements in collection

However when I run the same code in a sample application, I don't see the same behaviour, so I'm pretty sure this is as a result of the difficulty in finding a way to shut the app down in a way that does't create a signal loop and lots of other unrelated exceptions that get captured by the native client.

One possible "proper fix" would be to cascades the SDK shutdown logic, but I don't want to delay the version 6 release in order to build that so I think we park this PR for the time being and circle back post v6 release (perhaps in the new year when JP is back on deck).

jpnurmi · 2026-01-05T16:40:48Z

Hey, I have a fix coming up for duplicate Java events:

feat: add InvokeSentryResult::Events() to extract events from envelopes github-workflows#137

The SIGILL loop is still a mystery. It doesn't seem to be related to Process.KillProcess() - the same happens even with Process.KillProcess() removed.

jpnurmi · 2026-01-09T13:18:18Z

Good news. Both issues are in the test infrastructure, and fixes are on the way.

The SIGILL issue is luckily not a signal loop. Notice how the envelopes are sent at roughly 1s intervals. It is caused by the test app getting automatically relaunched because we trigger crashes a bit too early:

test(integration): prevent auto-relaunch on Android #4833

jamescrosswell · 2026-01-18T20:51:15Z

Good news. Both issues are in the test infrastructure, and fixes are on the way.

The SIGILL issue is luckily not a signal loop. Notice how the envelopes are sent at roughly 1s intervals. It is caused by the test app getting automatically relaunched because we trigger crashes a bit too early:

test(integration): prevent auto-relaunch on Android #4833

Awesome, thanks @jpnurmi - I'm back on deck now as well.

So once those two issues have been resolved, we could reopen this PR?

jpnurmi

what about ios integration tests?

integration-test/android.Tests.ps1

jamescrosswell · 2026-01-22T20:57:10Z

what about ios integration tests?

That was going to be next yeah... wanted to get Android going first though, since it's typically easier.

github-actions · 2026-02-03T00:17:25Z

Semver Impact of This PR

⚪ None (no version bump detected)

📋 Changelog Preview

This is how your changes will appear in the changelog.
Entries from this PR are highlighted with a left border (blockquote style).

This PR will not appear in the changelog.

_{🤖 This preview updates automatically when you update the PR.}

jamescrosswell · 2026-02-04T01:34:17Z

test/Sentry.Maui.Device.TestApp/Sentry.Maui.Device.TestApp.csproj

    <SupportedOSPlatformVersion Condition="$([MSBuild]::GetTargetPlatformIdentifier('$(TargetFramework)')) == 'android'">21.0</SupportedOSPlatformVersion>
-    <!-- Pin target iOS version so that our tests don't break when new versions of Xcode are released.
-      'netX.0-ios' resolves the latest version of the iOS SDK otherwise. -->
-    <TargetPlatformVersion Condition="$([MSBuild]::GetTargetPlatformIdentifier('$(TargetFramework)')) == 'ios'">18.0</TargetPlatformVersion>


We could leave this in there (bump to 26.2 of course)... that would be consistent with what we've done for the IntegrationTestApp

follow-up: #4895

cursor · 2026-02-05T10:26:31Z

test/Sentry.Maui.Device.TestApp/Sentry.Maui.Device.TestApp.csproj

+  <ItemGroup Condition="'$(TargetFramework)' == 'net9.0'">
+    <!-- https://github.com/advisories/GHSA-5f2m-466j-3848 -->
+    <PackageReference Include="System.Private.Uri" Version="4.3.2"/>
+  </ItemGroup>


Security package condition will never match target framework

Medium Severity

The ItemGroup condition '$(TargetFramework)' == 'net9.0' will never match because the project's target frameworks are net9.0-android, net10.0-android, and net10.0-ios. The exact string net9.0 never appears as a target framework. This means the security fix for GHSA-5f2m-466j-3848 (System.Private.Uri package reference) will never be included. The condition should use $(TargetFramework.StartsWith('net9')) to match platform-specific TFMs, consistent with patterns used elsewhere in the codebase like Sentry.Bindings.Android.csproj.

Seems this section is redundant now... when I tweaked the condition per the suggestion above I got:

error NU1510: Warning As Error: PackageReference System.Private.Uri will not be pruned. Consider removing this package from your dependencies, as it is likely unnecessary.

I've removed it altogether.

…ill not be pruned

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.}

integration-test/android.Tests.ps1

Flash0ver

🤘

Flash0ver · 2026-02-05T21:08:57Z

integration-test/net9-maui/Sentry.Maui.Device.IntegrationTestApp.csproj

+    <!-- Pin target iOS version so that our tests don't break when new versions of Xcode are released.
+         'netX.0-ios' resolves the latest version of the iOS SDK otherwise.
+    -->
+    <TargetFrameworks Condition="$([MSBuild]::IsOSPlatform('OSX'))">$(TargetFrameworks);net10.0-ios26.2</TargetFrameworks>


follow-up: #4895

integration-test/android.Tests.ps1

Flash0ver · 2026-02-05T21:18:59Z

integration-test/ios.Tests.ps1

+    # Note: we can't run against net10 and net9 becaus .NET 10 requires Xcode 26.2 and .NET 9 requires Xcode 26.0.
+    # The macOS GitHub Actions runners only have Xcode 26.1+ installed and no support for Xcode 26.2 is planned for
+    # net9.0-ios: https://github.com/dotnet/macios/issues/24199#issuecomment-3819021247


follow-up: #4895

Flash0ver · 2026-02-05T21:26:59Z

test/Sentry.Maui.Device.TestApp/Sentry.Maui.Device.TestApp.csproj

+<!-- Note: we can't run against net9-ios because .NET 10 requires Xcode 26.2 and .NET 9 requires Xcode 26.0.
+     The macOS GitHub Actions runners only have Xcode 26.1+ installed and no support for Xcode 26.2 is planned for
+     net9.0-ios: https://github.com/dotnet/macios/issues/24199#issuecomment-3819021247
+-->
+    <TargetFrameworks Condition="$([MSBuild]::IsOSPlatform('OSX'))">$(TargetFrameworks);net10.0-ios</TargetFrameworks>


follow-up: #4895

Flash0ver · 2026-02-05T21:27:07Z

test/Sentry.Maui.Device.TestApp/Sentry.Maui.Device.TestApp.csproj

    <SupportedOSPlatformVersion Condition="$([MSBuild]::GetTargetPlatformIdentifier('$(TargetFramework)')) == 'android'">21.0</SupportedOSPlatformVersion>
-    <!-- Pin target iOS version so that our tests don't break when new versions of Xcode are released.
-      'netX.0-ios' resolves the latest version of the iOS SDK otherwise. -->
-    <TargetPlatformVersion Condition="$([MSBuild]::GetTargetPlatformIdentifier('$(TargetFramework)')) == 'ios'">18.0</TargetPlatformVersion>


follow-up: #4895

.net10 tests for Android

d2c4358

jamescrosswell added 3 commits November 20, 2025 12:37

.

5e9fdcb

.

f99a711

.

5b7fbc3

Testing a theory...

d40c030

jamescrosswell added 2 commits November 27, 2025 14:03

Try to prevent double crash

344c80a

Revert "Try to prevent double crash"

b8d6c02

This reverts commit 344c80a.

jamescrosswell added 4 commits November 27, 2025 14:40

Ensure JavaSDK is shutdown when closing the Hub

241a908

Merge branch 'main' into net10_integration_tests

a773aa5

Remove todo from previously failing test

278c04c

Try System.Environment.Exit(0) instead of process.kill

ad9c671

jamescrosswell added 7 commits November 28, 2025 15:21

roll back unrelated change

429801b

reinstate null reference workaround for net9 tfm

cf16076

Revert change from kill to System.Environment.Exit

0889735

Skip the Native Crash integration test on net10

c1b3e7c

.

0adaddb

Revert fix to close Java SDK until we can work out why it breaks unit…

1361614

… tests

.

ed8b558

jpnurmi mentioned this pull request Jan 2, 2026

feat: add InvokeSentryResult::Events() to extract events from envelopes getsentry/github-workflows#137

Merged

jpnurmi mentioned this pull request Jan 9, 2026

test(integration): prevent auto-relaunch on Android #4833

Merged

Flash0ver mentioned this pull request Jan 21, 2026

build(deps): bump getsentry/github-workflows/sentry-cli/integration-test from 2.14.1 to 3.2.0 #4858

Merged

jpnurmi reviewed Jan 22, 2026

View reviewed changes

integration-test/android.Tests.ps1 Outdated Show resolved Hide resolved

integration-test/android.Tests.ps1 Outdated Show resolved Hide resolved

jamescrosswell added 2 commits January 23, 2026 15:08

Review feedback

461ff9b

Merge remote-tracking branch 'origin/main' into net10_integration_tests

03cdd74

jpnurmi mentioned this pull request Jan 26, 2026

feat: native signal handler strategy on Android #4676

Draft

jamescrosswell added 2 commits February 2, 2026 10:08

Merge remote-tracking branch 'origin/main' into net10_integration_tests

c8d7051

Merge branch 'main' into net10_integration_tests

91da55f

jamescrosswell added 3 commits February 4, 2026 10:05

Unpin the target platform version for ios in the device test app

130e51a

.

1309078

Drop net9.0 device/integration tests

b38c497

jamescrosswell commented Feb 4, 2026

View reviewed changes

jamescrosswell added 2 commits February 5, 2026 14:05

Remove side effects from common.ps1

edcc792

Revert behaviour change to cli and runtime integration tests

68db72f

jamescrosswell changed the title ~~Device/integration tests for mobile on .net10~~ chore: Device/integration tests for mobile on .net10 Feb 5, 2026

jamescrosswell added 5 commits February 5, 2026 15:39

Spinning in circles now...

3b5ea8d

.

4580fe2

msbuild

7f66cda

.

af39bb7

.

8ce560c

jamescrosswell marked this pull request as ready for review February 5, 2026 10:19

jamescrosswell requested a review from Flash0ver as a code owner February 5, 2026 10:19

cursor bot reviewed Feb 5, 2026

View reviewed changes

error NU1510: Warning As Error: PackageReference System.Private.Uri w…

30cd0f6

…ill not be pruned

cursor bot reviewed Feb 5, 2026

View reviewed changes

integration-test/android.Tests.ps1 Show resolved Hide resolved

Flash0ver approved these changes Feb 5, 2026

View reviewed changes

Flash0ver mentioned this pull request Feb 5, 2026

ci: use Cirrus Runners for iOS Integration Tests #4895

Open

jamescrosswell merged commit 863ee6a into main Feb 6, 2026
56 of 58 checks passed

jamescrosswell deleted the net10_integration_tests branch February 6, 2026 04:01

Uh oh!

Conversation

jamescrosswell commented Nov 19, 2025

Uh oh!

codecov bot commented Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

jamescrosswell commented Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jamescrosswell commented Nov 26, 2025

Uh oh!

supervacuus commented Nov 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jamescrosswell commented Nov 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jamescrosswell commented Nov 27, 2025

Uh oh!

jamescrosswell commented Dec 3, 2025

Uh oh!

jpnurmi commented Jan 5, 2026

Uh oh!

jpnurmi commented Jan 9, 2026

Uh oh!

jamescrosswell commented Jan 18, 2026

Uh oh!

jpnurmi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

jamescrosswell commented Jan 22, 2026

Uh oh!

github-actions bot commented Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Semver Impact of This PR

Uh oh!

jamescrosswell Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Flash0ver Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

cursor bot Feb 5, 2026

Choose a reason for hiding this comment

Security package condition will never match target framework

Uh oh!

jamescrosswell Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Flash0ver left a comment

Choose a reason for hiding this comment

Uh oh!

Flash0ver Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Flash0ver Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

Flash0ver Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

Flash0ver Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

codecov bot commented Nov 19, 2025 •

edited

Loading

jamescrosswell commented Nov 21, 2025 •

edited

Loading

supervacuus commented Nov 26, 2025 •

edited

Loading

jamescrosswell commented Nov 27, 2025 •

edited

Loading

github-actions bot commented Feb 3, 2026 •

edited

Loading