fix: bypass LLM generation for claims recording#54
Merged
Conversation
…source in screening The screening skill was storing search snippets as abstract_text instead of the official abstract from fetch_patent. This was caused by: 1. No explicit instruction to use fetch_patent.abstract_text (not snippet) 2. No legal_status column in screened_patents table, causing judgment to conflate relevance assessment with legal status 3. Tests only verified fetch_patent was invoked, not that results were used Changes: - Add legal_status column to screened_patents table - Change judgment CHECK constraint to only allow relevant/irrelevant - Update screening SKILL.md to explicitly distinguish abstract_text from snippet and batch fetch patents (up to 10 in parallel) - Add CRITICAL warnings against using snippet as abstract - Update record-screening.md with legal_status parameter - Add test checks: all_patents_screened, legal_status_recorded, patent_fetch_invoked for both screening and evaluating - Update all test fixtures to include legal_status in INSERT statements Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Change from relationship pattern MATCH (p:Patent)-[:claims]->(c:claims) to direct node match MATCH (c:claims) to avoid c.text returning null - Remove ORDER BY toInteger(c.number) which also causes c.text null bug - Add batch parallel fetch (up to 10 patents) for performance - Add CRITICAL warnings documenting the Cypher parser bugs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…nctions Claims text was being corrupted by LLM regeneration during recording. Now uses readfile() + json_each() + json_extract() to INSERT directly from fetch_patent output_file, eliminating LLM text intermediary. - Claims: output_file → sqlite3 JSON functions → DB (mechanical) - Elements: still via investigation-recording skill (LLM interpretation) - claim_type: initial INSERT defaults claim 1 to independent, then LLM reads from DB and UPDATEs correct independent claim numbers Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use sqlite3 readfile() + json_extract() to extract abstract_text and legal_status directly from fetch_patent output_file, eliminating LLM text regeneration that could corrupt patent abstracts. - abstract_text and legal_status: output_file → readfile() → json_extract() → DB (mechanical) - judgment and reason: LLM analysis (correct - these are interpretation tasks) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
output_file already contains all needed data (abstract_text, legal_status). No need to query the graph DB with execute_cypher. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…t-and-legal-status # Conflicts: # plugin/skills/evaluating/SKILL.md # plugin/skills/screening/SKILL.md
sonesuke
force-pushed
the
fix/screening-abstract-and-legal-status
branch
from
eea7c53 to
982028f
Compare
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Avoids version mismatch between CI cached versions and local npx. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
readfile()+json_each()+json_extract()to INSERT directly fromfetch_patentoutput_fileProblem
LLM was regenerating claim text when constructing SQL INSERT statements, causing:
Solution
output_file→readfile()→json_extract('$.claims')→json_each()→INSERTClaims text flows through the file system, not through LLM generation.
Test plan
evaluating/functionaltest🤖 Generated with Claude Code