fix: compiled gbrain binary can open PGLite brains (v0.10.3)#180
Open
adamv99-eng wants to merge 3 commits intogarrytan:masterfrom
Open
fix: compiled gbrain binary can open PGLite brains (v0.10.3)#180adamv99-eng wants to merge 3 commits intogarrytan:masterfrom
adamv99-eng wants to merge 3 commits intogarrytan:masterfrom
Conversation
bun build --compile bundles JavaScript but does not auto-embed data
files referenced via new URL("...", import.meta.url) inside node_modules.
Every compiled gbrain binary crashed on PGLite startup with "Extension
bundle not found: file:///.../vector.tar.gz" because five PGLite assets
were missing from the embed: vector.tar.gz, pg_trgm.tar.gz, pglite.wasm,
initdb.wasm, pglite.data.
Import each via `with { type: 'file' }` so Bun embeds them. Materialize
the extension tarballs to $TMPDIR/gbrain-pglite-ext/ because PGLite's
fs.createReadStream + zlib loader cannot traverse Bun's virtual FS. Hand
the WASM modules and pglite.data Blob directly to PGlite.create()'s
pgliteWasmModule / initdbWasmModule / fsBundle options so URL resolution
is bypassed entirely.
Compiles a fresh gbrain binary, runs init + list + search against a scratch HOME dir, and asserts stderr contains no "Extension bundle not found" or "Failed to fetch extension". Catches any future change that silently drops a PGLite asset from the bun --compile output.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix a bug where every compiled
gbrainbinary (the shipping artifact) crashed on any PGLite command withExtension bundle not found: file:///.../vector.tar.gz. Source mode (bun run src/cli.ts) worked, so the gap hid in plain sight until someone actually ran the binary.Root cause:
bun build --compilebundles JavaScript modules into its virtual FS but does not auto-embed arbitrary data files referenced vianew URL("...", import.meta.url)from insidenode_modules. PGLite needs five such assets:vector.tar.gz,pg_trgm.tar.gz,pglite.wasm,initdb.wasm,pglite.data. All five were missing from the compiled binary.Fix:
src/core/pglite-engine.tswith { type: 'file' }so Bun embeds them.$TMPDIR/gbrain-pglite-ext/because PGLite'sfs.createReadStream + zlibloader cannot traverse Bun's virtual FS.pglite.dataBlob directly toPGlite.create()viapgliteWasmModule/initdbWasmModule/fsBundleoptions, bypassing URL resolution entirely for the core assets.Works in both source mode and compiled mode. ~80 lines of glue.
Relates to the standing TODO in
TODOS.mdabout fixing this upstream in Bun (issue oven-sh/bun#15032). A workaround here lets us ship today; the upstream fix would eventually let us delete this code.Test Coverage
New E2E regression guard in
test/e2e/compiled-binary.test.ts:gbrain-compiled-test.exeinit,list,searchagainst a scratchHOMEdirExtension bundle not foundorFailed to fetch extensionVerified locally on Windows (the canary platform that surfaced the bug):
bun test test/pglite-engine.test.ts test/engine-factory.test.ts test/e2e/compiled-binary.test.ts— 48/48 pass.~/.local/bin/gbrain.exe—gbrain list,gbrain search,gbrain doctor --fastall run clean.Pre-Landing Review
withEmbeddedBundleis a thin wrapper around the upstream extensions; preserves theirsetup()contract, only overridesbundlePath.materializeExtensions()writes to$TMPDIRonce per process, idempotently.loadCoreAssets()caches the compiled WASM modules + data Blob.pathToFileURL().Platform Note
The project's release matrix ships
darwin-arm64+linux-x64only, so the bug was only visible to people building locally (typically Windows users since there's no prebuilt Windows artifact). The fix applies to all three targets, but Windows was the canary.Test plan
bun test test/pglite-engine.test.ts— 41/41 passbun test test/engine-factory.test.ts— passbun test test/e2e/compiled-binary.test.ts— 2/2 pass (compiles a fresh binary, runs init/list/search, asserts clean)gbrain list/gbrain search/gbrain doctor --fastagainst real brain in compiled mode — clean🤖 Generated with Claude Code