Skip to content

Read only theme directories and their direct children when preloading files#1157

Open
elia wants to merge 1 commit intoShopify:mainfrom
elia:elia/known-files
Open

Read only theme directories and their direct children when preloading files#1157
elia wants to merge 1 commit intoShopify:mainfrom
elia:elia/known-files

Conversation

@elia
Copy link

@elia elia commented Mar 23, 2026

What are you adding in this PR?

Fixes #1151, this is an alternative to #1152, after having another look at the problem following @aswamy's comment.

When preloading files at startup, the language server was recursively loading all .json files under the theme root — including unrelated data files (e.g. data/orders.json). A single 175 MB JSON file parsed into an AST can consume 1–2 GB of heap, easily exhausting even an 8 GB limit on repos that happen to have a data/ directory alongside the theme.

This change restricts preloading to Shopify theme directories only: assets, blocks, config, layout, locales, sections, snippets, templates, templates/customers, and templates/metaobjects. It will only read the direct children of these directories, not their subdirectories, which is consistent with the theme architecture specification and should be sufficient for language features.

What's next? Any followup issues?

None.

What did you learn?

Nothing surprising in retrospect, but worth noting: recursiveReadDirectory starts from the workspace root, not the theme root — so any large files anywhere in the repo are fair game without an explicit allowlist.

Before you deploy

  • I included a patch bump changeset

Copy link
Contributor

@aswamy aswamy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggested one minor change, but looks great!

@elia elia force-pushed the elia/known-files branch from cad9c96 to dfc0a54 Compare March 23, 2026 13:59
Copy link
Contributor

@aswamy aswamy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@elia Forgot one more thing - create a changeset of the libraries affect by this change. You can run it via yarn changeset on the root repo, make it a patch, and push it up to this commit.

Restrict preload() to Shopify theme directories (assets, blocks,
config, layout, locales, sections, snippets, templates). Previously
all .json files under the root were loaded, including unrelated data
files. A single 175 MB JSON file parsed into an AST can consume
1-2 GB of heap, easily exhausting even an 8 GB limit on large repos.

Co-Authored-By: Claude <noreply@anthropic.com>
@elia elia force-pushed the elia/known-files branch from dfc0a54 to fff4551 Compare March 23, 2026 16:14
@elia
Copy link
Author

elia commented Mar 23, 2026

@elia Forgot one more thing - create a changeset of the libraries affect by this change. You can run it via yarn changeset on the root repo, make it a patch, and push it up to this commit.

Done ✅

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test failures look real:

-             "uri": "mock-fs:/templates/index.json",
+             "uri": "mock-fs:sections/header.json",

I'm not sure what's going on there.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's just missing a leading slash, the diff algorithm got confused, a few lines above you have its counterpart:

image

I'll look into that, my Claude initially added a check on slashes, I kinda convinced it it was not necessary, but it was probably right 🤖

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

theme-language-server OOM crash on large themes (~600 files) — Electron's 4GB heap limit

3 participants