Skip to content

fix: separate legal_status from judgment and ensure correct abstract source #52

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sonesuke merged 1 commit into from
Apr 5, 2026
Merged

fix: separate legal_status from judgment and ensure correct abstract source #52

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 21 additions & 19 deletions plugin/skills/investigation-preparing/references/schema.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,19 +30,21 @@ Stores patent master data imported from CSV files.

Stores latest screening results only (no history tracking).

| Column | Type | Description |
| ------------- | ------------- | ------------------------------------------------ |
| patent_id | TEXT PK | Patent number (FK to target_patents.patent_id) |
| judgment | TEXT NOT NULL | Judgment: `relevant`, `irrelevant`, or `expired` |
| reason | TEXT NOT NULL | Screening rationale |
| abstract_text | TEXT NOT NULL | Abstract content (fetched during screening) |
| screened_at | TEXT | Screening timestamp |
| updated_at | TEXT | Last update timestamp |
| Column | Type | Description |
| ------------- | ------------- | -------------------------------------------------------------------------- |
| patent_id | TEXT PK | Patent number (FK to target_patents.patent_id) |
| judgment | TEXT NOT NULL | Relevance: `relevant` or `irrelevant` |
| legal_status | TEXT | Legal status from `fetch_patent` (e.g., `Pending`, `Expired`, `Withdrawn`) |
| reason | TEXT NOT NULL | Screening rationale |
| abstract_text | TEXT NOT NULL | Abstract from `fetch_patent.abstract_text` |
| screened_at | TEXT | Screening timestamp |
| updated_at | TEXT | Last update timestamp |

**Constraints**:

- `patent_id` is a FOREIGN KEY referencing `target_patents(patent_id)` with `ON DELETE CASCADE`
- `judgment` only allows: `relevant`, `irrelevant`, `expired`
- `judgment` only allows: `relevant`, `irrelevant`
- `legal_status` reflects the patent's legal status from `fetch_patent`
- `reason` and `abstract_text` must NOT be NULL

### claims
Expand Down Expand Up @@ -176,13 +178,13 @@ Stores element-level mappings between patent elements and prior art references.

Aggregates screening statistics.

| Column | Type | Description |
| -------------- | ------- | ------------------------------------------- |
| total_targets | INTEGER | Count of all patents in target_patents |
| total_screened | INTEGER | Count of all patents in screened_patents |
| relevant | INTEGER | Count of patents with judgment='relevant' |
| irrelevant | INTEGER | Count of patents with judgment='irrelevant' |
| expired | INTEGER | Count of patents with judgment='expired' |
| Column | Type | Description |
| -------------- | ------- | ----------------------------------------------------------- |
| total_targets | INTEGER | Count of all patents in target_patents |
| total_screened | INTEGER | Count of all patents in screened_patents |
| relevant | INTEGER | Count of patents with judgment='relevant' |
| irrelevant | INTEGER | Count of patents with judgment='irrelevant' |
| expired | INTEGER | Count of patents with legal_status='Expired' or 'Withdrawn' |

## Triggers

Expand Down Expand Up @@ -225,9 +227,9 @@ target_patents (1) -----> (1) screened_patents (1) -----> (*) claims (1) ----->
| | | | |
|-- patent_id (PK) |-- patent_id (PK, FK) |-- patent_id (FK) |-- patent_id (PK, FK) |-- patent_id (PK, FK)
|-- title |-- judgment |-- claim_number (FK) |-- claim_number (PK, FK) |-- claim_number (PK, FK)
|-- country |-- reason |-- claim_type |-- element_label (PK) |-- element_label (PK, FK)
|-- assignee |-- abstract_text |-- claim_text |-- element_description |-- similarity_level
|-- extra_fields |-- screened_at |-- created_at |-- created_at |-- analysis_notes
|-- country |-- legal_status |-- claim_type |-- element_label (PK) |-- element_label (PK, FK)
|-- assignee |-- reason |-- claim_text |-- element_description |-- similarity_level
|-- extra_fields |-- abstract_text |-- created_at |-- created_at |-- analysis_notes
|-- publication_date |-- updated_at |-- updated_at |-- updated_at |-- analyzed_at
|-- filing_date | | | |-- updated_at
|-- grant_date | | |
Expand Down
5 changes: 3 additions & 2 deletions plugin/skills/investigation-preparing/references/sql/initialize-database.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,8 @@ CREATE TABLE IF NOT EXISTS target_patents (
-- Create screened_patents table (with CHECK constraint for judgment)
CREATE TABLE IF NOT EXISTS screened_patents (
patent_id TEXT PRIMARY KEY NOT NULL,
judgment TEXT NOT NULL CHECK(judgment IN ('relevant', 'irrelevant', 'expired')),
judgment TEXT NOT NULL CHECK(judgment IN ('relevant', 'irrelevant')),
legal_status TEXT,
reason TEXT NOT NULL,
abstract_text TEXT NOT NULL,
screened_at TEXT DEFAULT (datetime('now')),
Expand All @@ -52,7 +53,7 @@ SELECT
(SELECT COUNT(*) FROM screened_patents) as total_screened,
(SELECT COUNT(*) FROM screened_patents WHERE judgment = 'relevant') as relevant,
(SELECT COUNT(*) FROM screened_patents WHERE judgment = 'irrelevant') as irrelevant,
(SELECT COUNT(*) FROM screened_patents WHERE judgment = 'expired') as expired;
(SELECT COUNT(*) FROM screened_patents WHERE legal_status IN ('Expired', 'Withdrawn')) as expired;

-- Create timestamp triggers
CREATE TRIGGER IF NOT EXISTS update_target_patents_timestamp
Expand Down
2 changes: 1 addition & 1 deletion plugin/skills/investigation-recording/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ internal reference files for this skill's internal use only.

**Example requests**:

- "Record screening result: id=US1234567A1, judgment=relevant, reason=..."
- "Record screening result: id=US1234567A1, judgment=relevant, legal_status=Pending, reason=..."
- "Record claims for patent US1234567A1: claim_1=..., claim_2=..."
- "Record elements for patent US1234567A1: element_a=..., element_b=..."
- "Record similarities for patent US1234567A1: element_a=Significant, element_b=Moderate..."
Expand Down
55 changes: 32 additions & 23 deletions plugin/skills/investigation-recording/references/instructions/record-screening.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,11 @@ Save or update a screening judgment in the `screened_patents` table.
### Main Query (UPSERT)

```sql
INSERT OR REPLACE INTO screened_patents (patent_id, judgment, reason, abstract_text, updated_at)
INSERT OR REPLACE INTO screened_patents (patent_id, judgment, legal_status, reason, abstract_text, updated_at)
VALUES (
'<patent_id>',
'<judgment>',
'<legal_status>',
'<reason>',
'<abstract_text>',
datetime('now')
Expand All @@ -23,6 +24,9 @@ VALUES (

- `INSERT OR REPLACE` provides UPSERT semantics
- `patent_id` is a FOREIGN KEY referencing `target_patents(patent_id)`
- `judgment` must be `relevant` or `irrelevant`
- `legal_status` is the value from `fetch_patent` (e.g., `Pending`, `Expired`, `Withdrawn`)
- `abstract_text` must be from `fetch_patent.abstract_text` (NOT from `search_patents.snippet`)
- `reason` and `abstract_text` are required (NOT NULL)
- `updated_at` automatically set to current timestamp

Expand All @@ -32,30 +36,32 @@ VALUES (

```bash
# Record screening result
sqlite3 patents.db "INSERT OR REPLACE INTO screened_patents (patent_id, judgment, reason, abstract_text, updated_at)
VALUES ('US1234567A', 'relevant', 'Core technology for LLM systems', 'Abstract content fetched during screening', datetime('now'));"
sqlite3 patents.db "INSERT OR REPLACE INTO screened_patents (patent_id, judgment, legal_status, reason, abstract_text, updated_at)
VALUES ('US1234567A', 'relevant', 'Pending', 'Core technology for LLM systems', 'Abstract content fetched during screening', datetime('now'));"
```

### Using Variables

```bash
PATENT_ID="US1234567A"
JUDGMENT="relevant"
LEGAL_STATUS="Pending"
REASON="Core technology for LLM systems"
ABSTRACT_TEXT="Abstract content here"

sqlite3 patents.db "INSERT OR REPLACE INTO screened_patents (patent_id, judgment, reason, abstract_text, updated_at)
VALUES ('$PATENT_ID', '$JUDGMENT', '$REASON', '$ABSTRACT_TEXT', datetime('now'));"
sqlite3 patents.db "INSERT OR REPLACE INTO screened_patents (patent_id, judgment, legal_status, reason, abstract_text, updated_at)
VALUES ('$PATENT_ID', '$JUDGMENT', '$LEGAL_STATUS', '$REASON', '$ABSTRACT_TEXT', datetime('now'));"
```

### Multi-Line SQL

```bash
sqlite3 patents.db <<EOF
INSERT OR REPLACE INTO screened_patents (patent_id, judgment, reason, abstract_text, updated_at)
INSERT OR REPLACE INTO screened_patents (patent_id, judgment, legal_status, reason, abstract_text, updated_at)
VALUES (
'US1234567A',
'relevant',
'Pending',
'Core technology for LLM systems',
'Abstract content here',
datetime('now')
Expand All @@ -65,12 +71,13 @@ EOF

## Parameters

| Parameter | Type | Required | Default | Description |
| ------------- | ------ | -------- | ------- | ------------------------------------------------------------- |
| patent_id | string | Yes | - | Patent ID (must exist in target_patents) |
| judgment | string | Yes | - | One of: `relevant`, `irrelevant`, `expired` |
| reason | string | Yes | - | Screening rationale (must NOT be NULL) |
| abstract_text | string | Yes | - | Abstract content (must NOT be NULL, fetched during screening) |
| Parameter | Type | Required | Default | Description |
| ------------- | ------ | -------- | ------- | ---------------------------------------------------------------- |
| patent_id | string | Yes | - | Patent ID (must exist in target_patents) |
| judgment | string | Yes | - | One of: `relevant`, `irrelevant` |
| legal_status | string | No | NULL | Legal status from `fetch_patent` (e.g., `Pending`, `Expired`) |
| reason | string | Yes | - | Screening rationale (must NOT be NULL) |
| abstract_text | string | Yes | - | Abstract from `fetch_patent.abstract_text` (must NOT be snippet) |

## Output

Expand Down Expand Up @@ -122,7 +129,7 @@ fi
### Invalid Judgment

```bash
# Solution: Use only: relevant, irrelevant, expired
# Solution: Use only: relevant, irrelevant
JUDGMENT="relevant" # Valid
```

Expand All @@ -132,8 +139,8 @@ JUDGMENT="relevant" # Valid
# Escape single quotes by doubling
REASON="It''s a core technology"

sqlite3 patents.db "INSERT OR REPLACE INTO screened_patents (patent_id, judgment, reason, updated_at)
VALUES ('US1234567A', 'relevant', 'It''s a core technology', datetime('now'));"
sqlite3 patents.db "INSERT OR REPLACE INTO screened_patents (patent_id, judgment, legal_status, reason, updated_at)
VALUES ('US1234567A', 'relevant', 'Pending', 'It''s a core technology', datetime('now'));"
```

## Data Integrity
Expand Down Expand Up @@ -196,16 +203,17 @@ OFFSET=0
PATENT_ID=$(sqlite3 patents.db "SELECT patent_id FROM target_patents ORDER BY patent_id LIMIT 1 OFFSET $OFFSET;")

# Fetch and analyze (using MCP tool)
# fetch-patent "$PATENT_ID"
# fetch-patent "$PATENT_ID" → get abstract_text and legal_status

# Record result
sqlite3 patents.db <<EOF
INSERT OR REPLACE INTO screened_patents (patent_id, judgment, reason, abstract_text, updated_at)
INSERT OR REPLACE INTO screened_patents (patent_id, judgment, legal_status, reason, abstract_text, updated_at)
VALUES (
'$PATENT_ID',
'relevant',
'Pending',
'Core technology for multi-turn LLM systems',
'Abstract content fetched during screening',
'Abstract content from fetch_patent.abstract_text',
datetime('now')
);
EOF
Expand All @@ -214,10 +222,10 @@ EOF
### Bulk Screening from File

```bash
# Assume results.csv has: patent_id,judgment,reason
while IFS=',' read -r PATENT_ID JUDGMENT REASON; do
sqlite3 patents.db "INSERT OR REPLACE INTO screened_patents (patent_id, judgment, reason, updated_at)
VALUES ('$PATENT_ID', '$JUDGMENT', '$REASON', datetime('now'));"
# Assume results.csv has: patent_id,judgment,legal_status,reason
while IFS=',' read -r PATENT_ID JUDGMENT LEGAL_STATUS REASON; do
sqlite3 patents.db "INSERT OR REPLACE INTO screened_patents (patent_id, judgment, legal_status, reason, updated_at)
VALUES ('$PATENT_ID', '$JUDGMENT', '$LEGAL_STATUS', '$REASON', datetime('now'));"
done < results.csv
```

Expand All @@ -226,10 +234,11 @@ done < results.csv
```bash
# Change judgment from irrelevant to relevant
sqlite3 patents.db <<EOF
INSERT OR REPLACE INTO screened_patents (patent_id, judgment, reason, updated_at)
INSERT OR REPLACE INTO screened_patents (patent_id, judgment, legal_status, reason, updated_at)
VALUES (
'US1234567A',
'relevant',
'Pending',
'Re-evaluated: Actually core technology after review',
datetime('now')
);
Expand Down
32 changes: 19 additions & 13 deletions plugin/skills/screening/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,29 +59,35 @@ Filter collected patents by legal status and relevance to prepare for evaluation
1. **Get Patents to Screen**:
- Invoke `Skill: investigation-fetching` with request "Get list of unscreened patent IDs"

2. **For each patent**, execute Steps 2a–2d:

**2a. Read Specification**:
2. **Read Specification** (once):
- Read `specification.md` to understand Theme, Domain, and Target Product

**2b. Fetch Patent Data**:
- Invoke `Skill: google-patent-cli:patent-fetch` with patent ID
- Extract: title, abstract, legal status
3. **Batch Fetch Patent Data** (up to 10 patents in parallel):
- Split unscreened patents into batches of 10
- For each batch, invoke `Skill: google-patent-cli:patent-fetch` for all patents **in parallel**
- From each result, extract:
- `abstract_text` property — the official patent abstract (with 【課題】【解決手段】 format for JP patents)
- `legal_status` property — the patent's current legal status (e.g., `Pending`, `Expired`, `Withdrawn`)
- `title` property
- **CRITICAL**: Do NOT use `snippet` — `snippet` is a search result summary, NOT the official abstract. Always use `abstract_text`.

**2c. Evaluate and Judge**:
4. **Evaluate and Record** (for each patent):

Judgment criteria:
- **Expired or Withdrawn** → `expired`
Judgment criteria (relevance only):
- **Irrelevant**: Completely different industry from Theme/Domain
- **Relevant**: Matches Theme/Domain, Direct Competitors, Core Tech
- **Exception**: Even if domain differs, KEEP if technology could serve as infrastructure or common platform

Judgment values: `relevant`, `irrelevant`, `expired` (lowercase)
Legal status handling:
- Record `legal_status` from `fetch_patent` as-is in the database
- Note expired/withdrawn patents in the reason field, but judgment remains based on relevance

Judgment values: `relevant`, `irrelevant` (lowercase)

**2d. Record Result**:
- Invoke `Skill: investigation-recording` with request "Record screening result for patent <patent-id>: <judgment_data>"
For each patent, invoke `Skill: investigation-recording` with request "Record screening result for patent <patent-id>: judgment=<judgment>, legal_status=<legal_status>, reason=<reason>, abstract_text=<abstract_text from fetch_patent>"
- **CRITICAL**: The `abstract_text` passed to recording MUST be the `abstract_text` from `fetch_patent`, NOT the `snippet` from `search_patents`.

3. **Verify Results**: Confirm all patents have corresponding `screened_patents` entries
5. **Verify Results**: Confirm all patents have corresponding `screened_patents` entries

## State Management

Expand Down
2 changes: 1 addition & 1 deletion tests/claim-analyzing/functional-absent-feature.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Before asking me any questions about missing features, please use the question-r
command = """
sqlite3 patents.db < /workspaces/patent-kit/plugin/skills/investigation-preparing/references/sql/initialize-database.sql
sqlite3 patents.db "INSERT INTO target_patents (patent_id, title, country, publication_date) VALUES ('US12231380B1', 'Trigger-based transfer of conversations from a chatbot to a human agent', 'US', '2023-10-11');"
sqlite3 patents.db "INSERT INTO screened_patents (patent_id, judgment, reason, abstract_text) VALUES ('US12231380B1', 'relevant', 'Related to chatbot-to-human transfer mechanism', 'A system for triggering transfer of conversations from a chatbot to a human agent based on conversation context.');"
sqlite3 patents.db "INSERT INTO screened_patents (patent_id, judgment, legal_status, reason, abstract_text) VALUES ('US12231380B1', 'relevant', 'Pending', 'Related to chatbot-to-human transfer mechanism', 'A system for triggering transfer of conversations from a chatbot to a human agent based on conversation context.');"
sqlite3 patents.db "INSERT INTO claims (patent_id, claim_number, claim_type, claim_text) VALUES ('US12231380B1', 1, 'independent', '1. A computer-implemented method for managing conversations in a chatbot system, comprising: detecting a trigger condition in a conversation context; and transferring the conversation from the chatbot to a human agent based on the trigger condition.');"
sqlite3 patents.db "INSERT INTO elements (patent_id, claim_number, element_label, element_description) VALUES ('US12231380B1', 1, 'A', 'Detecting a trigger condition in a conversation context');"
sqlite3 patents.db "INSERT INTO elements (patent_id, claim_number, element_label, element_description) VALUES ('US12231380B1', 1, 'B', 'Transferring the conversation from the chatbot to a human agent based on the trigger condition');"
Expand Down
2 changes: 1 addition & 1 deletion tests/claim-analyzing/functional.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Before asking me any questions about missing features, please use the question-r
command = """
sqlite3 patents.db < /workspaces/patent-kit/plugin/skills/investigation-preparing/references/sql/initialize-database.sql
sqlite3 patents.db "INSERT INTO target_patents (patent_id, title, country, publication_date) VALUES ('US12231380B1', 'Trigger-based transfer of conversations from a chatbot to a human agent', 'US', '2023-10-11');"
sqlite3 patents.db "INSERT INTO screened_patents (patent_id, judgment, reason, abstract_text) VALUES ('US12231380B1', 'relevant', 'Related to chatbot-to-human transfer mechanism', 'A system for triggering transfer of conversations from a chatbot to a human agent based on conversation context.');"
sqlite3 patents.db "INSERT INTO screened_patents (patent_id, judgment, legal_status, reason, abstract_text) VALUES ('US12231380B1', 'relevant', 'Pending', 'Related to chatbot-to-human transfer mechanism', 'A system for triggering transfer of conversations from a chatbot to a human agent based on conversation context.');"
sqlite3 patents.db "INSERT INTO claims (patent_id, claim_number, claim_type, claim_text) VALUES ('US12231380B1', 1, 'independent', '1. A computer-implemented method for managing conversations in a chatbot system, comprising: detecting a trigger condition in a conversation context; and transferring the conversation from the chatbot to a human agent based on the trigger condition.');"
sqlite3 patents.db "INSERT INTO elements (patent_id, claim_number, element_label, element_description) VALUES ('US12231380B1', 1, 'A', 'Detecting a trigger condition in a conversation context');"
sqlite3 patents.db "INSERT INTO elements (patent_id, claim_number, element_label, element_description) VALUES ('US12231380B1', 1, 'B', 'Transferring the conversation from the chatbot to a human agent based on the trigger condition');"
Expand Down
Loading
Loading