[#12] Cron Backfill

[realproject7]

Summary

• P1-7a: src/app/api/cron/backfill/route.ts — GET endpoint that:
  • Scans last ~200 blocks (~5 min on Base) for StoryFactory events
  • Processes StorylineCreated (storyline + genesis plot with IPFS fetch + hash verify + ERC-8004 agent detection)
  • Processes PlotChained (IPFS fetch + hash verify)
  • Processes Donation events
  • Upserts all records with (tx_hash, log_index) deduplication (safe to re-run)
  • Caches block timestamps to minimize RPC calls
  • Protected by CRON_SECRET env var (open in dev)
  • Gracefully skips when StoryFactory is not yet deployed (zero address)
• P1-7b: vercel.json — 5-minute cron schedule for /api/cron/backfill

Fixes #12

Test plan

• tsc --noEmit passes
• vitest run — 22/22 passing
• Verify cron runs on Vercel deployment
• Integration test: create storyline + plots, verify backfill picks them up

🤖 Generated with Claude Code

[project7-interns]

Verdict: REQUEST CHANGES

Summary

The cron route covers the three event types and adds the Vercel schedule, but it misses one of the required merge-check items for issue #12 and the overnight queue: start-block tracking. Right now it always rescans only the latest ~200 blocks, which is not the requested backfill mechanism.

Findings

• [high] There is no persisted start-block tracking; the route derives fromBlock from currentBlock - 200 on every run. That means the job cannot progress through older gaps and does not satisfy the ticket requirement for start-block tracking. If the app is down longer than the scan window, missed events older than ~200 blocks will never be backfilled.
  • File: src/app/api/cron/backfill/route.ts:69
  • Suggestion: persist the last processed block (for example in Supabase or another durable store), read it at the start of each run, scan from that checkpoint, and advance the checkpoint after a successful pass.

Decision

Request changes because the current implementation does not meet the core backfill requirement to track progress across cron runs, so it can miss events permanently once they fall outside the fixed scan window.

[project7-interns]

T2b Review — REQUEST CHANGES

Solid implementation overall. Covers all three event types, uses idempotent upserts, handles zero-address gracefully, and continues processing on per-event errors. The IPFS fetch + hash verify pattern is consistent with the existing indexer. However, there are two issues that need addressing before approval.

---

BLOCKING

B1. No persisted start-block cursor — events outside the 200-block window are permanently lost

The endpoint always scans currentBlock - 200 to currentBlock. If the cron job is delayed, skipped, or Vercel has a brief outage (>~7 minutes), the gap between runs exceeds 200 blocks and events in that gap are never scanned. This defeats the purpose of a "backfill" safety net — the very scenario it exists for (missed events) is also the scenario where it fails.

Issue #12 calls this a "safety net for when the inline indexer misses a plot." A safety net that has the same failure mode as the thing it's protecting against is not a safety net.

Fix: Store last_scanned_block in a Supabase table (e.g., cron_cursors). On each run, scan from last_scanned_block + 1 to currentBlock, then update the cursor. Fall back to currentBlock - SCAN_BLOCKS only on first run (no cursor row exists). This ensures zero gaps regardless of cron timing.

B2. Counters increment even when upsert is a no-op (misleading response)

storylinesInserted++ / plotsInserted++ / donationsInserted++ increment on every successfully processed event, regardless of whether the upsert actually inserted a new row or was a no-op (row already existed). The response JSON says "upserted" but the numbers don't reflect actual inserts.

For observability, check the Supabase upsert response: if the row already existed, don't count it as "upserted." Alternatively, rename the field to "processed" to be accurate.

---

NON-BLOCKING

N1. Genesis plot silently dropped on IPFS failure — no retry, no log

In processStorylineCreated, if IPFS fetch fails or hash doesn't match, the genesis plot is silently skipped but the storyline is still inserted. There's no indication this happened — the errors counter isn't incremented, and the response doesn't distinguish "storyline inserted but genesis plot failed." A future cron run will re-process the StorylineCreated event, but the upsert on storylines will be a no-op, and the genesis plot fetch is only attempted inside processStorylineCreated, so the plot is permanently orphaned unless there's a separate mechanism to retry.

Consider: either increment errors when the genesis plot fails, or track it as a separate counter (e.g., genesis_plot_failures).

N2. eslint-disable for any types on DecodedEvent and SupabaseClient

DecodedEvent and SupabaseClient are typed as any with eslint-disable comments. The Supabase client type is available from createServerClient() return type, and DecodedEvent can be narrowed with viem's DecodeEventLogReturnType. Minor, but worth a follow-up.

N3. PlotChained silently skips on IPFS failure — same orphan risk as N1

processPlotChained returns early if content is null or hash mismatches. Same concern: the errors counter isn't incremented, and subsequent runs will also skip (upsert on the event won't re-trigger the IPFS fetch). The plot is silently lost.

---

Verdict: REQUEST CHANGES — B1 (persisted cursor) is required. The current fixed-window approach has the same gap vulnerability as the inline indexer it's meant to back up. B2 is a smaller fix but affects operational trust in the cron's reporting.

[project7-interns]

Verdict: REQUEST CHANGES

Summary

The added cursor table is the right direction, but the current range calculation still allows permanent gaps when the cron falls behind. That means the core backfill guarantee is still not met.

Findings

• [high] The route reads last_block, but then clamps fromBlock forward to currentBlock - 200 whenever the cursor lags by more than the scan window. In that case, every block between last_block + 1 and currentBlock - 200 - 1 is skipped forever, and the cursor is still advanced to currentBlock. This is still permanent data loss under backlog, just with a persisted cursor attached.
  • File: src/app/api/cron/backfill/route.ts:80
  • Suggestion: when a cursor exists, always continue from last_block + 1; bound work by capping toBlock for this run instead of jumping fromBlock forward. Then advance the cursor only to the last block actually processed.

Decision

Request changes because the current implementation still cannot guarantee full backfill across outages or delayed cron runs, which is the core requirement of issue #12.

[project7-interns]

T2b Review: APPROVE

All issues resolved across three rounds:

1. Persistent cursor — backfill_cursor table with singleton row, gap-free scanning confirmed: fromBlock = last_block + 1, toBlock capped at fromBlock + 200, cursor advances to toBlock only.
2. Counter labels — renamed to processed, accurate.
3. No block skipping — backlog processed incrementally, no forward clamping.

Event scanning (StorylineCreated, PlotChained, Donation), CRON_SECRET auth, graceful zero-address handling, and Vercel cron config all look good. Phase 1 complete.

[project7-interns]

Verdict: APPROVE

Summary

The cursor logic now satisfies the backfill requirement: the job resumes from last_block + 1, limits work by capping toBlock per run, and advances the cursor only to the highest block actually scanned. The route also covers the required event types and includes the cron schedule, and the updated CI check passed.

Findings

• None.

Decision

Approve because the prior backlog-gap blocker is resolved and the implementation now matches the issue #12 and overnight queue merge criteria.