[#964] Add IPFS response size limit

[realproject7]

Summary

• Adds 1MB size limit on IPFS responses in storyline, plot indexers and backfill cron
• Checks both Content-Length header (fast reject) and actual body length (defense in depth)
• Oversized responses throw/return null — fallback content handling takes over
• Normal story content (~10K chars) is well under the 1MB limit

Fixes #964

Test plan

• Normal story publishing unaffected (content well under 1MB)
• IPFS responses > 1MB are rejected before full body read (Content-Length check)
• IPFS responses > 1MB without Content-Length header are caught after read
• Backfill cron also rejects oversized IPFS content
• Fallback content still works when IPFS fetch fails due to size

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
plotlink	Ignored		Apr 23, 2026 11:37pm

[project7-interns]

Verdict: REQUEST CHANGES

Summary

The new Content-Length guard is good, but the fallback body-size check is measuring JavaScript string length rather than response bytes. That means a UTF-8 response with multi-byte characters can exceed the intended 1MB limit and still pass.

Findings

• [medium] The post-read size validation is not actually checking bytes. ipfsContent.length / text.length counts UTF-16 code units, not the number of bytes received over the network. A malicious IPFS response containing multi-byte UTF-8 characters can be larger than 1,000,000 bytes while still staying under the character-count threshold, so the "actual body size" defense-in-depth check is incomplete.
  • File: src/app/api/index/storyline/route.ts:126, src/app/api/index/plot/route.ts:98, src/app/api/cron/backfill/route.ts:42
  • Suggestion: Measure bytes instead, for example with new TextEncoder().encode(text).length or by enforcing the limit while streaming the response body.

Decision

Requesting changes because the current implementation can still allow oversized IPFS bodies through when the response uses multi-byte characters.

[project7-interns]

Verdict: APPROVE

Summary

The re-review update fixes the body-size validation by measuring UTF-8 byte length with TextEncoder rather than string length, so the 1MB cap now applies correctly to multi-byte IPFS responses as well. The header check plus byte-count fallback are in place for storyline, plot, and backfill paths.

Findings

• No blocking findings.

Decision

Approving because the previous correctness issue in the IPFS size-limit enforcement is resolved and the PR now satisfies the reviewed acceptance criteria. Checks visible to me were still pending at review time.