[Pending internal discussions] tweak the parsing of _redirects files

[vicb]

Follow up for #12467

The main idea of this PR is that the maximum line length of a _redirects should not take comments into account.

There are also a few other changes.

There was a single MAX_LINE_LENGTH used for both the _headers and _redirects while the first one should be 2000 and second one 1000 (See opened questions below for that one). The new constants are MAX_HEADER_LINE_LENGTH and MAX_REDIRECT_LINE_LENGTH.

Note that other constants have been renamed to clearly indicated when they relates to either HEADER or REDIRECT.

Also a bit of cleanup in the tests to make them consistent.

question

• Pages doc has no explicit max line length. Workers has 1000. EWC has 1024. What is the right value? I have used 1000 in this PR (from 2000 before). But we should align EWC should not error on something this code allow and vice versa
• Should we bump the _redirects parser version, not sure how it is used - it is defined both here and EWC
• Should we add a global file size limit i.e. to prevent OOM? (or do we already have something in place?)

future work

• Update EWC to also trailing comments (draft MR pending)

Usage of _redirects (by Claude and I)

1. Workers Assets deploy (wrangler deploy / wrangler versions upload)

• Read: packages/wrangler/src/assets.ts:469 — maybeGetFile() reads raw text
• Passed through: packages/wrangler/src/deploy/deploy.ts:867 (deploy),
  packages/wrangler/src/versions/upload.ts:726 (versions upload)
• Uploaded: packages/wrangler/src/deployment-bundle/create-worker-upload-form.ts:90 —
  raw string embedded in AssetConfigMetadata JSON inside the FormData metadata field
• Parsed by parseRedirects? No — the Cloudflare API parses it server-side

2. Workers Assets remote dev (wrangler dev --remote)

• Passed through: packages/wrangler/src/dev/remote.ts:197
• Same as deploy — raw text sent to the API
• Parsed by parseRedirects? No

3. Pages deploy (wrangler pages deploy)

• Read: packages/wrangler/src/api/pages/deploy.ts:136 — readFileSync()
• Uploaded: packages/wrangler/src/api/pages/deploy.ts:297-299 — appended as a File
  object in FormData (not inside JSON)
• Parsed by parseRedirects? No — the Pages API parses it server-side

4. Miniflare local dev (Workers Assets, wrangler dev --local)

• Read & parsed: packages/miniflare/src/plugins/assets/index.ts:99-124
• parseRedirects() → constructRedirects() → RedirectsSchema.parse() → injected as
  JSON CONFIG binding into the local workerd asset-worker
• Parsed by parseRedirects? Yes

5. Pages local dev (wrangler pages dev)

• Read & parsed: packages/wrangler/src/miniflare-cli/assets.ts:150-161
• parseRedirects() → createMetadataObject() → constructRedirects()
• Also watches for file changes and re-parses (lines 177-200)
• Parsed by parseRedirects? Yes

6. Vite plugin dev (experimental)

• Read & parsed: packages/vite-plugin-cloudflare/src/asset-config.ts:105-117
• parseRedirects() → constructRedirects() → RedirectsSchema.parse() → passed to
  Miniflare options
• Gated behind experimental.headersAndRedirectsDevModeSupport
• Also detects changes via hasAssetsConfigChanged() (line 27-46)
• Parsed by parseRedirects? Yes

7. Runtime: Workers asset-worker

• File: packages/workers-shared/asset-worker/src/handler.ts:48,999-1050
• Matching engine: packages/workers-shared/asset-worker/src/utils/rules-engine.ts:116-160
• Receives pre-parsed structured configuration.redirects via CONFIG binding
• Applies staticRedirectsMatcher() (O(1) dict lookup) then generateRedirectsMatcher()
  (regex-based dynamic matching) on each request
• Status 200 → rewrite (proxy), 301/302/303/307/308 → HTTP redirect response

8. Runtime: Pages asset-server

• File: packages/pages-shared/asset-server/handler.ts:202-243
• Same two-phase matching as Workers asset-worker, but using metadata.redirects
• Metadata generated by packages/pages-shared/metadata-generator/createMetadataObject.ts
• Receives both the parsedRedirects and the raw file

---

• Tests
  • Tests included/updated
  • Automated tests not possible - manual testing has been completed as follows:
  • Additional testing not necessary because:
• Public documentation
  • Cloudflare docs PR(s):
  • Documentation not necessary because: fix

A picture of a cute animal (not mandatory, but encouraged)

---

[changeset-bot]

🦋 Changeset detected

Latest commit: 09e3bb1

The changes in this PR will be included in the next version bump.

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[devin-ai-integration]

Devin Review found 2 potential issues.

View 5 additional findings in Devin Review.

[pkg-pr-new]

create-cloudflare

npm i https://pkg.pr.new/create-cloudflare@12490

@cloudflare/kv-asset-handler

npm i https://pkg.pr.new/@cloudflare/kv-asset-handler@12490

miniflare

npm i https://pkg.pr.new/miniflare@12490

@cloudflare/pages-shared

npm i https://pkg.pr.new/@cloudflare/pages-shared@12490

@cloudflare/unenv-preset

npm i https://pkg.pr.new/@cloudflare/unenv-preset@12490

@cloudflare/vite-plugin

npm i https://pkg.pr.new/@cloudflare/vite-plugin@12490

@cloudflare/vitest-pool-workers

npm i https://pkg.pr.new/@cloudflare/vitest-pool-workers@12490

@cloudflare/workers-editor-shared

npm i https://pkg.pr.new/@cloudflare/workers-editor-shared@12490

@cloudflare/workers-utils

npm i https://pkg.pr.new/@cloudflare/workers-utils@12490

wrangler

npm i https://pkg.pr.new/wrangler@12490

commit: 09e3bb1

[devin-ai-integration]

Devin Review found 1 new potential issue.

View 7 additional findings in Devin Review.

[petebacondarwin]

The "valid" test needs fixing.

[devin-ai-integration]

Devin Review found 1 new potential issue.

View 10 additional findings in Devin Review.

[claude]

Claude finished @vicb's task —— View job

---

Changeset Review

Reviewing changeset files...

• Read changeset guidelines
• Review .changeset/sweet-lizards-cough.md
• Validate changeset content
• Check for cute animals

---

⚠️ Issues found

The changeset .changeset/sweet-lizards-cough.md has the following issues:

1. Title needs improvement: "tweak the parsing of _redirects file" is too vague. It should clearly describe what changed, such as:

   • "Exclude comments from _redirects line length validation"
   • "Fix _redirects max line length to exclude comments"

2. Body lacks context: The description doesn't explain:

   • Why this change matters (what problem it solves)
   • What the previous behavior was
   • What the new behavior is
   • The significance of changing from 2000 to 1000 characters

   A better body would be something like:

   Previously, comments in `_redirects` files counted toward the maximum line length of 2000 characters, which could cause valid redirects to be rejected. Now comments are excluded from length validation, and the maximum line length has been corrected to 1000 characters (matching the Workers platform limit) for redirect directives only.
   

3. Escaped underscore: The \_redirects in the title/body should just be _redirects - the backslash escape isn't necessary in markdown here and makes it harder to read.

Suggestions for improvement:

---
"@cloudflare/workers-shared": patch
---

Exclude comments from `_redirects` line length validation

Previously, comments in `_redirects` files counted toward the maximum line length, which could cause valid redirects with comments to be rejected. Comments are now excluded from length validation. Additionally, the maximum line length for redirect directives has been corrected to 1000 characters (matching the Workers platform limit), down from 2000.

---