feat: lazy context loading and mops source caching for faster startup#456
Draft
feat: lazy context loading and mops source caching for faster startup#456
Conversation
Defer per-project moc.js loading and package resolution until a file in that project is actually opened, instead of eagerly loading all projects at startup. Cache mops source resolution results to disk (keyed by hash of mops.toml + mops.lock) to avoid repeated npx invocations across sessions. For workspaces with many projects (~20 mops.toml files), this reduces startup from spawning 40+ child processes and loading 20 compiler instances to a fast glob scan with near-instant per-project loading on first file open. Made-with: Cursor
…dling Made-with: Cursor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
In workspaces with many Motoko projects (~20
mops.tomlfiles), the extension startup is very slow. The language server eagerly creates a compiler context for every discovered project in parallel — spawning 40+ child processes (npx --no ic-mops -- --version,npx --no ic-mops sources, possiblydfx cache show+moc --versionper project), downloading/loading moc.js for each unique version, and resolving all packages before the server is usable. With 20 projects this can take over a minute.Root cause
notifyPackageConfigChange()runsPromise.all(directories.map(...))over all discovered project directories, where each iteration spawns multiplenpxchild processes, downloads moc.js from GitHub releases,require()s a multi-MB JS compiler module, and runsmops sources. All of this runs eagerly at server initialization before the user opens any file.Fix
Lazy context creation: The glob scan for
mops.toml/vessel.dhall/dfx.jsonstill runs at startup (it's cheap), but the expensive per-project work is deferred until a file in that project is actually opened.ensureContextLoaded(uri)is called in all LSP request handlers (onDidOpen,onCompletion,onHover,onDefinition,onCodeAction,onDocumentSymbol,onSignatureHelp,onReferences,onPrepareRename,onRenameRequest, andcheckImmediate). Loading promises are deduplicated so concurrent requests for the same project share a single load.Mops source caching: The output of
npx --no ic-mops sourcesis cached to.mops/.sources-cache.json, keyed by SHA-256 hash ofmops.toml+mops.lock. On cache hit, the npx invocations are skipped entirely.Caveats
notifyWorkspace()still runs at startup and writes all workspace files to the default context's virtual FS. When a project context is later lazily loaded, files are re-read from disk to populate the new context. This duplication is acceptable since the expensive operations (child process spawns, moc.js loading) are what dominated startup time.Test plan
enhancedMigrationfailure only)requestMocJstest updated to trigger lazy loading before asserting context stateextraFlagstest passes — default context gets moc flags applied at discovery timemops.tomlprojects, verify only the opened project's context loads.mops/.sources-cache.jsonis written and reused on subsequent startupsmops.tomlormops.lockand verify cache is invalidated and packages reload