Skip to content

🦌 Add speculative PR metadata generation to hide Claude latency#258

Open
zdavison wants to merge 3 commits intomainfrom
deer/speculative-pr-metadata-generation
Open

🦌 Add speculative PR metadata generation to hide Claude latency#258
zdavison wants to merge 3 commits intomainfrom
deer/speculative-pr-metadata-generation

Conversation

@zdavison
Copy link
Copy Markdown
Owner

Summary

Adds an experimental speculative_close feature that starts generating PR metadata (branch name, title, body) in the background while the user is reading the post-session menu. On the happy path where the user picks "p" (create PR), the metadata is already ready or nearly ready, hiding most of the Claude inference latency.

If the user picks any other option, the speculative work is discarded at the cost of one extra Claude API call.

Changes

  • packages/deerbox/src/config.ts — adds experimental.speculativeClose config option with TOML and repo-local parsing support
  • packages/deerbox/src/cli.ts — passes speculativeClose from config into the post-session context
  • packages/deerbox/src/post-session.ts — kicks off speculative fetch and metadata generation before prompting the user; passes result to createPR; also fixes destroy() to use .catch(() => {}) instead of awaiting
  • packages/shared/src/git/finalize.ts — exports generatePRMetadata and PRMetadata; adds optional fetchDone param to reuse a prefetch; parallelises the three git reads after fetch using Promise.all; createPullRequest accepts and uses speculativeMetadata
  • packages/deerbox/src/git/finalize.ts — re-exports generatePRMetadata and PRMetadata from shared
  • packages/shared/src/index.ts — exports generatePRMetadata and PRMetadata from the shared package

Created by deer — review carefully.

/**
* Start generating PR metadata speculatively while the user reads the
* post-session menu. Hides most of the Claude inference latency on the
* happy path (user picks "p"). Incurs an extra Claude API call if the
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why does this incur an extra claude api call for other options?
list each option and what happens

also, speculativeClose should also work for m because commit message is generated with claude, and its used for both m and p

/**
* Pre-computed PR metadata from a speculative generation started before the
* user was prompted. If provided, skips `findPRTemplate` + `generatePRMetadata`.
* Note: generated before stageAndCommit, so may miss last-minute uncommitted changes.
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this going to cause problems? the flow before was stageAndCommit -> generate right? now it'S generating before stageAndCommit?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant