fix: focus-aware feedback prompts and multi-category support #16
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Move CATEGORY_DEFINITIONS to prompts/feedback_categories.py as single source of truth - Remove unused FeedbackFocus.ALL constant and dead from_raw() method - Add FeedbackFocus.ALLOWED derived from CATEGORY_DEFINITIONS keys - Update imports in feedback.py and prompts/feedback.py Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When --focus is provided, the JSON schema's findings.category field now only lists the selected categories plus "other" (for critical overrides) instead of all categories. This reduces LLM confusion. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add explicit negative instruction: "ONLY use category values listed in the schema. DO NOT use any other category values." - Add _normalize_finding_categories() to remap any out-of-scope categories to "other" after LLM returns - Add tests for the normalization function This belt-and-suspenders approach ensures LLMs cannot output categories outside the focus scope, even if they hallucinate from training data. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Cast category string to Any to satisfy mypy's Literal type requirement for FeedbackFinding.category. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add local mypy hook that runs via uv to use project dependencies. Fix pre-existing torch import type errors in mt.py with ignore comments. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add torch to mypy ignore_missing_imports override and use cast instead of type: ignore to avoid unused-ignore errors in CI. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Remove torch-specific mypy workarounds since torch is now required for development. Update documentation to clarify that --all-extras is mandatory for mypy to pass. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Rename SelectedOutlineRange to SectionsUnderReview for clarity - Omit line entirely when all sections are selected (redundant) - Add explicit instruction for brief alignment checking Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
--focusflag and needed to be guided to only evaluate requested categories while still surfacing critical issues from other categories.Description
FeedbackFocus.parse_listto parse comma-separated focus values and accept multiple normalized categories, and addedALLas a constant for completeness.list[str] | Nonefocus type throughprepare_context/FeedbackContextand the prompt-building flow so prompts receive the parsed list of categories directly.prompts/feedback.pyto include anIn-scope categoriesline, per-category definitions limited to selected categories, and aCritical overrideinstruction to always include high-severity issues from any category.feedback_cli.pyto accept comma-separated values and pass the parsed list toprepare_context, and added a dry-run unit testtest_feedback_focus_multiple_categories_limits_promptto validate prompt rendering.Testing
uv run pytestand all unit tests passed (154 passed).uv run ruff checkand linting passed (All checks passed!).uv run mypywhich reported unrelated type errors due to missingtorchstubs insrc/scribae/translate/mt.pyand therefore returned errors; this is an existing environmental typing issue not introduced by these changes.Codex Task