GenerateBundle Task fails writing to singlefilehost.exe

[dsplaisted]

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=1146095
Build error leg or test failing: Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution
Pull request: #50648

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "The process cannot access the file '.*singlefilehost\\.exe' because it is being used by another process",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=1146095
Error message validated: [The process cannot access the file '.*singlefilehost\.exe' because it is being used by another process]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 9/12/2025 3:27:55 PM UTC

Report

Build	Definition	Test	Pull Request
1167156	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution	#51036
1167157	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51074
1167138	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51036
1167106	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51143
1166889	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51138
1166893	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50980
1166850	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51088
1166699	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51074
1166526	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51135
1166457	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51119
1166108	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51134
1165607	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51127
1165384	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution	#51074
1165036	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#48918
1165008	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51075
1165014	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51036
1164810	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51120
1164821	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51108
1164627	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51118
1164637	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51119
1164469	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution	#51117
1164366	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution	#51074
1163714	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51050
1163547	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51052
1163599	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51108
1163532	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51050
1163331	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51036
1163197	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution	#51054
1163200	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51094
1163206	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51095
1162428	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51052
1163029	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51073
1159610	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51020
1163119	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51071
1162767	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51054
1162774	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51080
1162493	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51078
1162091	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51072
1162483	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51063
1162584	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51072
1162252	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51075
1162215	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51074
1162101	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution	#51071
1162125	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51068
2805635	dotnet-sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#53833
1161981	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51068
1161907	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51069
1161906	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50999
1160239	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51042
1161644	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51063
1160876	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51054
1161099	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50864
1161516	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51039
1161431	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51031
1161295	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51058
1161249	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51036
1161229	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50999
1161230	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution	#51050
1161015	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50999
1160940	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50999
1160906	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51031
1160756	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51053
1160740	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51052
1160652	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51050
1159928	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51039
1160074	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51032
1160469	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50999
1160285	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51031
1160278	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51036
1160283	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51044
1160277	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51043
1159681	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51031
1160093	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution	#51017
1160083	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution	#51019
1160022	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51036
1159860	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51039
1159639	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50999
1159565	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50992
1158694	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50901
1159023	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51020
1159428	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50992
1159285	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50992
1159255	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50999
1159211	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51027
1158830	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50983
1158684	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51014
1158808	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51019
1157762	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50901
1157823	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50974
1158448	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50992
1157799	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50975
1158291	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50992
1158265	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50983
1157845	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51004
1157923	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51006
1157881	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50999
1157814	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50496
1157558	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50974
1157804	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#49617
1157775	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50977
Displaying 100 of 223 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
10	53	223

[dsplaisted]

@dotnet/illink @agocke We are seeing failures where the GenerateBundle task is failing because it cannot access the singlnefilehost.exe file. We added retry logic because we thought maybe the issues were caused by defender. I thought that helped, but now the issue is happening pretty consistently, so it may be a product issue.

Interestingly, it seems to only fail on Mac OS and Linux when bundling a Windows app. I can't remember if it was failing on Windows before we added the retry logic or not.

Can someone investigate this?

I'm also wondering whether we should merge this codeflow PR which is blocked by the failure. That's where I've seen the failure most consistently recently, so it's possible there's a code change coming in that made the problem worse.

[agocke]

@jtschuster can you take a look?

[dsplaisted]

Ping @jtschuster, any progress on this? I just noticed it in #50876

[agocke]

I'm going to load balance a bit. @sbomer can you look instead?

[jtschuster]

Still investigating, but I am able to repro locally on a mac, but only on the first run of the test. Subsequent runs use the same path for the tmp project and don't repro. It's also interesting that it's failing on a read and not a write.

[dsplaisted]

@jtschuster Any update on this? This is impacting a bunch of our CI runs.

[jtschuster]

Still investigating exactly what's going on, but my hunch is that there's an async memory map flush that keeps the apphost mapped from CreateAppHost. In particular it looks like this todo might fix the issue: https://github.com/dotnet/runtime/blob/b8c39cc4d102518f8608df70e6e3cb7a5c10d48d/src/installer/managed/Microsoft.NET.HostModel/AppHost/HostWriter.cs#L172
The ResourceUpdater.Update() maps the destination apphost back into memory, and if there's async flushing after Dispose(), we could run into issues.

As a workaround, it looks like adding --disable-build-servers to the PublishCommand.Execute() method does something to make the issue fail to repro for both me and @sbomer. I'm not sure why, though.