Skip to content

feat(iterate-pr): Add scripts for structured CI and feedback data#34

Merged
dcramer merged 3 commits intomainfrom
feat/iterate-pr-improvements
Feb 3, 2026
Merged

feat(iterate-pr): Add scripts for structured CI and feedback data#34
dcramer merged 3 commits intomainfrom
feat/iterate-pr-improvements

Conversation

@dcramer
Copy link
Member

@dcramer dcramer commented Feb 3, 2026

Add helper scripts that pre-process GitHub API responses to reduce token usage and improve the iterate-pr workflow.

Inspired by OpenAI's agent skills which use scripts to extract only relevant information rather than dumping raw API output.

Scripts:

fetch_pr_checks.py - Fetches CI check status and extracts failure snippets:

  • Pattern matching for common failure markers (error, traceback, FAILED, etc.)
  • Returns structured JSON with check name, status, and extracted snippets
  • Avoids sending full CI logs to the model

fetch_pr_feedback.py - Fetches and categorizes PR review feedback:

  • Categories: blocking, suggestion, bot, resolved
  • Uses GraphQL to get thread resolution status
  • Filters out PR author's own comments and resolved threads

SKILL.md changes:

  • Scripts are now the primary approach (not optional)
  • Reduced from 185 to 131 lines by removing redundant gh CLI documentation
  • Fallback commands moved to a single section at the end
  • Prompts user to select which suggestions to address instead of treating all feedback equally

dcramer and others added 2 commits February 2, 2026 18:01
Add helper scripts that pre-process GitHub API data to reduce token
usage and improve feedback categorization:

- fetch_pr_checks.py: Fetches CI checks and extracts relevant failure
  snippets using pattern matching instead of full logs
- fetch_pr_feedback.py: Categorizes PR feedback as blocking, suggestion,
  bot, or resolved

Update SKILL.md to:
- Reference the optional helper scripts with fallback to gh commands
- Add Step 6 for handling subjective feedback with user prompts
- Present suggestion-type feedback as numbered list for user selection

Co-Authored-By: Claude <noreply@anthropic.com>
Restructure SKILL.md to follow OpenAI skill patterns:
- Scripts are now the primary approach, not optional
- Reduced from 185 to 131 lines by removing redundant gh CLI docs
- Moved fallback commands to a single section at the end
- Clearer workflow with numbered steps

Co-Authored-By: Claude <noreply@anthropic.com>
@dcramer dcramer marked this pull request as ready for review February 3, 2026 02:11
Update fetch_pr_feedback.py to detect and use Sentry's LOGAF scale:
- h: / [h] / high: → high priority (must fix)
- m: / [m] / medium: → medium priority (should fix)
- l: / [l] / low: → low priority (optional)

Categories renamed from blocking/suggestion to high/medium/low to
align with Sentry's code review guidelines.

Refs: https://develop.sentry.dev/engineering-practices/code-review/#logaf-scale

Co-Authored-By: Claude <noreply@anthropic.com>
@dcramer dcramer merged commit 3d3c068 into main Feb 3, 2026
3 checks passed
@dcramer dcramer deleted the feat/iterate-pr-improvements branch February 3, 2026 02:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants