Skip to content

Add API stubs for Go binary emulation#270

Merged
mr-tz merged 8 commits intomasterfrom
worktree-fix-go-emulation-223
Mar 25, 2026
Merged

Add API stubs for Go binary emulation#270
mr-tz merged 8 commits intomasterfrom
worktree-fix-go-emulation-223

Conversation

@williballenthin
Copy link
Copy Markdown
Collaborator

Summary

  • Add winmm.dll and bcryptprimitives.dll to the default module config so the Go runtime can load them via LoadLibraryExW
  • Add bcryptprimitives.ProcessPrng stub (Go 1.20+ requires this for CSPRNG)
  • Add kernel32 stubs: AddVectoredContinueHandler, CreateWaitableTimerExW, GetProcessAffinityMask, SetConsoleCtrlHandler, GetErrorMode, WerGetFlags, WerSetFlags
  • Add ntdll stubs: RtlGetNtVersionNumbers, RtlGetCurrentPeb, RtlGetVersion
  • Add winmm stubs: timeBeginPeriod, timeEndPeriod

Closes #223

Before

Go binaries crash immediately with unsupported_api after ~7 API calls during runtime initialization.

After

Go runtime initializes fully — loads libraries, allocates heap, spawns threads, reads environment (188+ API calls). The runtime still panics (exit code 2) due to deeper emulation gaps (thread scheduling, std handle initialization), but the unsupported_api crashes are resolved.

Test plan

  • Existing test suite passes (50 pass, 13 skipped — unchanged)
  • Ruff lint passes on all changed files
  • Verified with cross-compiled Go 1.25 hello world binary (GOOS=windows GOARCH=amd64)

🤖 Generated with Claude Code

williballenthin and others added 5 commits March 9, 2026 11:34
Co-authored-by: Moritz <mr-tz@users.noreply.github.com>
The Go runtime dynamically loads and calls several Windows APIs that
speakeasy did not previously emulate, causing Go binaries to crash
during initialization (issue #223).

- Add winmm.dll to default module config so LoadLibraryExA succeeds
- Add kernel32 stubs: AddVectoredContinueHandler, CreateWaitableTimerExW,
  GetProcessAffinityMask, SetConsoleCtrlHandler
- Add ntdll stub: RtlGetNtVersionNumbers (returns Windows 10 version)
- Add winmm stubs: timeBeginPeriod, timeEndPeriod

Refs: #223
Follow-up to the initial Go emulation fix. After testing with a real
Go binary (Go 1.25), additional missing APIs were discovered:

- Add bcryptprimitives.dll module + ProcessPrng stub (Go 1.20+ uses
  this instead of advapi32!SystemFunction036 for CSPRNG)
- Add kernel32 stubs: GetErrorMode, WerGetFlags, WerSetFlags
- Add ntdll stubs: RtlGetCurrentPeb, RtlGetVersion

With these changes, Go runtime initialization proceeds through heap
setup, thread creation, and environment loading. The runtime still
panics (exit code 2) due to deeper emulation gaps (thread scheduling,
timer resolution), but the unsupported_api crashes from #223 are
fully resolved.

Refs: #223
@google-cla
Copy link
Copy Markdown

google-cla bot commented Mar 9, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@williballenthin williballenthin force-pushed the worktree-fix-go-emulation-223 branch from 807f7f2 to 9768a3c Compare March 9, 2026 14:21
Root cause of Go runtime panic "nanotime returning zero": Go reads
time directly from KUSER_SHARED_DATA (0x7FFE0000) without calling
QueryPerformanceCounter. Speakeasy mapped this page but left it
zero-filled.

- Populate KUSER_SHARED_DATA with realistic values: InterruptTime,
  SystemTime, QpcFrequency, NtMajorVersion, TickCount
- Fix GetStdHandle on amd64: mask the DWORD argument to 32 bits so
  STD_ERROR_HANDLE (0xFFFFFFF4) matches when sign-extended to 64-bit
  (0xFFFFFFFFFFFFFFF4)
- Add SwitchToThread stub

With these fixes, Go runtime initialization proceeds past nanotime
into the goroutine scheduler, where it hits "stoplockedm: not
runnable" — a fundamental threading limitation (Go requires concurrent
thread execution, speakeasy executes threads serially).

Refs: #223
@williballenthin
Copy link
Copy Markdown
Collaborator Author

williballenthin commented Mar 9, 2026

Investigation: What it takes to fully emulate Go binaries

After getting the initial API stubs in place, "I" (read: Opus) did a deeper investigation into what's needed for a Go hello-world to actually print output. Here's the full picture.

Root cause chain

  1. fatal error: nanotime returning zero — Go reads time directly from KUSER_SHARED_DATA (0x7FFE0000) without calling QueryPerformanceCounter. Speakeasy mapped this page but left it zero-filled. Fixed in latest push — populated with QpcFrequency, InterruptTime, SystemTime, etc.

  2. GetStdHandle returns 0 on amd64 — The STD_ERROR_HANDLE constant 0xFFFFFFF4 (32-bit DWORD) doesn't match 0xFFFFFFFFFFFFFFF4 (64-bit sign-extended value from RCX). Fixed — added dev & 0xFFFFFFFF mask in get_std_handle().

  3. fatal error: stoplockedm: not runnable — Go's M:N scheduler requires concurrent thread execution. The sysmon thread (created via CreateThread) must run simultaneously with the main goroutine. Not fixable with stubs — this is an architectural limitation of speakeasy's execution model. See also CreateThread and WaitForXXX APi #86

Thread model analysis

Speakeasy's start() loop (winemu.py:507-571) breaks after the first run completes successfully. Created threads are queued in run_queue but only execute if the primary run raises an exception. Go's sysmon thread never gets CPU time, so the scheduler enters an invalid state when the main goroutine tries to yield.

Two potential approaches to fix this (both non-trivial):

  • Run-queue draining: After ExitProcess, continue executing queued thread runs. Simple but insufficient — Go needs threads concurrent with main, not sequential after it.
  • Cooperative scheduling: On blocking APIs (WaitForSingleObject, Sleep, SwitchToThread), save thread state and switch to the next queued run. This could work for Go but requires significant changes to the core emulation loop.

Other findings

Area Status
TlsAlloc returns index 0 Not a problem — Go checks against TLS_OUT_OF_INDEXES
VirtualQuery on reserved memory Returns ERROR_INVALID_PARAMETER (only committed regions are visible). Secondary issue — not the current crash point
powrprof.dll missing Go handles gracefully
KUSER_SHARED_DATA unpopulated Fixed
GetStdHandle 32/64-bit mismatch Fixed

Current state

With all three commits, Go runtime initialization proceeds through: library loading → heap allocation (including 0xC000000000 region) → thread creation → environment loading → 627+ API calls. The runtime reaches the goroutine scheduler before hitting the threading wall.

Copy link
Copy Markdown
Collaborator Author

@williballenthin williballenthin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link
Copy Markdown
Contributor

@mr-tz mr-tz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very very nice, just one minor question

@williballenthin williballenthin requested a review from mr-tz March 25, 2026 08:04
@mr-tz mr-tz merged commit 531726f into master Mar 25, 2026
6 checks passed
@mr-tz mr-tz deleted the worktree-fix-go-emulation-223 branch March 26, 2026 11:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

"Hello World" in Golang

2 participants