feat(scala-sbt): warm-cache plugin command; add ADR-015#21
Merged
Conversation
Closes #18. PR #20 raised virtiofsd's NOFILE ceiling 8x but virtiofsd --cache=auto accumulates backing-file FDs monotonically; any session long enough hits the new ceiling. The durable fix is to keep churning caches off virtiofs entirely. ADR-015 codifies the rule: plugins must not virtiofs-mount package-manager caches, build caches, or other LRU-shaped directories. Cache state lives on root.img (which persists per workspace), at the tool's default location or via env var redirect from a setup script. scala-sbt migrated as the reference: dropping the ~/.cache/coursier and ~/.ivy2 mounts is sufficient since both defaults are under \$HOME on root.img. Trade-off: first sbt update per workspace re-fetches; cache then stays warm for the workspace lifetime. Future work (called out in the ADR): VM snapshotting via qcow2 backing or cloud-hypervisor save/restore would obviate the warmup cost. Separately scoped. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Credentials are not virtiofs-mounted — they're written to a one-shot ext4 disk that the guest reads at boot and unmounts (ADR-003). Listing ~/.env as a virtiofs example contradicts the architecture and the explicit-credentials philosophy (ADR-013). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
~/.ssh is not a recommended virtiofs target — the architecture explicitly avoids sharing host SSH identity. ADR-014 injects a per-VM key for VM SSH identity, and host SSH keys (GitHub, AWS, deploy keys) deliberately do not cross the sandbox boundary. Reframe the appropriate-virtiofs-uses paragraph to state the principle (moderate file count, in-place cross-boundary semantics) and point at dedicated channels for sensitive data: ADR-003 (credentials disk), ADR-014 (SSH key injection). Plugins mounting additional host state (claude-code/~/.claude) own that tradeoff in their own ADR (ADR-010). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ADR-015 distinguishes steady-state runtime virtiofs use (forbidden for churning caches) from bounded one-shot bootstrap reads (permitted). scala-sbt adopts the bootstrap pattern: - RO virtiofs mounts at /mnt/host-cache/coursier and /mnt/host-cache/ivy2 (only if host paths exist) - Setup script rsyncs into ~/.cache/coursier and ~/.ivy2 on first boot, guarded by a sentinel file - Subsequent boots: rsync is a no-op; sbt's runtime I/O lives entirely on root.img and never crosses virtiofs - rsync added to nix.packages Trade-off: virtiofsd holds backing-file FDs accumulated during the rsync pass for the VM's lifetime (~50-100k for typical coursier caches), well under the 524288 ceiling from PR #20. No further accumulation occurs since nothing reads the mount after warmup. ADR-015 updated to codify the bootstrap exception with explicit MUST requirements (readonly, side path, sentinel-guarded, no post-warmup access). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
razvanz
changed the title
Remove cruft that doesn't drive the decision: - toolchain reference table (belongs in a plugin-authoring guide) - "what stays on virtiofs" sub-section (scope creep into ADR-003/014/010) - atomic-ops rationale (collapsed; ADR-001 owns it) - host-cache-sharing trade-off discussion (bootstrap addresses it) - "log directories" example (logs are append-mostly, not LRU) - future-work detail (one-line callout) Keep what matters for a plugin author reading this in a year: the steady-state prohibition, the bootstrap MUSTs, the FD rationale, and scala-sbt as concrete grounding. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bootstrap mounts (RO virtiofs at /mnt/host-cache/*) were left attached
for the VM's lifetime, leaving virtiofsd processes idle but holding the
backing-file FDs accumulated during the rsync. ADR-015's whole pitch is
"no steady-state virtiofs"; leaving virtiofsd idling walks back from
that.
Changes:
- bin/nixbox: do_unmount is idempotent — if no active mount matches the
target, log and return 0 instead of dying. Lets post-up hooks run
unconditionally on every up (warmup is sentinel-guarded; mount may
not have been (re)created on subsequent boots).
- plugins/scala-sbt: post-up hook releases /mnt/host-cache/{coursier,ivy2}
via nixbox unmount. nixbox unmount detaches the virtio device and
kills virtiofsd, freeing the accumulated FDs.
- ADR-015: split the bootstrap rules into MUSTs (readonly, side path,
sentinel, no post-warmup access) and SHOULDs (release after warmup).
In-tree plugins MUST release; third-party plugins are encouraged but
not required, since a single bootstrap's FDs are well under the
524288 ceiling.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the virtiofs-bootstrap pattern with a host-side plugin command following the established 'nixbox aws login' / 'nixbox claude-code sync-config' pattern. - New: plugins/scala-sbt/commands/warm-cache.sh — pipes host's ~/.cache/coursier and ~/.ivy2 into the guest via 'tar | nixbox run "tar -x"' over the existing SSH channel. Sentinel-guarded for idempotency. - Plugin: post-up hook calls 'nixbox scala-sbt warm-cache'. No virtiofs mounts at all. No rsync in nix.packages. Setup script reverts to credentials-only. - ADR-015: drops the bootstrap-exception entirely. The decision is now strictly "no virtiofs for churning caches"; warmup is delivered by a host-side plugin command, not a transient mount. - bin/nixbox: revert the do_unmount idempotency change — it was scaffolding for the previous post-up unmount approach, no longer needed. Trade-off vs. the previous virtiofs-bootstrap approach: tar over SSH is slightly slower than virtiofs+rsync (~1.5x for local vsock) but eliminates the virtiofsd FD legacy entirely — no idle processes, no held FDs, no exception in the ADR. The plugin command is also user-invokable for manual refreshes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
razvanz
changed the title
Reframe the ADR so it documents trade-offs rather than dictating to third-party plugin authors: - Title: "No virtiofs ..." → "Avoid virtiofs ..." - Decision: "Plugins MUST NOT ..." → "In-tree plugins do not ..." - Add an explicit paragraph stating that third-party plugins are not bound; authors who virtiofs-mount churning caches accept the FD-pressure cost documented in #18 - Plugin-command pattern: presented as one clean alternative, not "the right primitive" - Consequences scoped to in-tree plugins The substance — what the project does and why, and what it costs to do otherwise — is unchanged. Only the framing softens. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cut the Rationale section entirely — its FD-monotonic argument is already in Problem; the plugin-command motivation is conveyed by the "established pattern" cue in Decision. Compress Consequences to two bullets. 36 → 27 lines. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
"Future work" implied roadmap commitment. Reframe as a possible direction the trade-off could be revisited from, with explicit "not committed work" framing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ions - scala-sbt provides line: include scala-cli (added in #15) and the warm-cache plugin command. Mention root.img persistence + ADR-015 link in the trailing paragraph. - Known limitations: add "No live host↔guest cache sharing" entry pointing at ADR-015 — useful for users who'd otherwise expect virtiofs-style mount semantics for caches. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Implements a new approach for Scala/SBT caches that avoids virtiofs for churn-heavy directories by seeding guest-local caches from the host via a plugin command, and documents the rationale in a new ADR.
Changes:
- Removes virtiofs mounts for
~/.cache/coursierand~/.ivy2from thescala-sbtplugin and adds apost-uphook to warm caches. - Adds
nixbox scala-sbt warm-cacheto stream host caches into the guest (sentinel-guarded to be idempotent). - Adds ADR-015 documenting the “no virtiofs for hot caches” decision and updates READMEs accordingly.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| plugins/scala-sbt/default.nix | Drops cache mounts; adds hooks.post-up to invoke warm-cache automatically. |
| plugins/scala-sbt/commands/warm-cache.sh | New plugin command that streams coursier/ivy2 caches into the guest via `tar |
| plugins/scala-sbt/README.md | Updates plugin docs for new cache behavior and new warm-cache command. |
| docs/decisions/015-no-virtiofs-hot-caches.md | New ADR capturing the motivation/decision/trade-offs. |
| README.md | Updates scala-sbt plugin overview and adds a “no live cache sharing” limitation note. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #18.
ADR-015: avoid virtiofs for churning caches.
virtiofsd --cache=autoaccumulates FDs monotonically; PR #20 raised the ceiling but didn't change the trajectory.In-tree plugins seed warm host caches via a host-side plugin command invoked from
post-up, mirroringnixbox aws login/nixbox claude-code sync-config. Third-party plugins aren't bound — the FD trade-off is documented.plugins/scala-sbt: newnixbox scala-sbt warm-cachestreams~/.cache/coursierand~/.ivy2from host into the guest viatar | nixbox run "tar -x", sentinel-guarded. No virtiofs mounts.Test plan
🤖 Generated with Claude Code