Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
6babde3
chore: remove unused prior-art-template from investigation-reporting
web-flow Apr 4, 2026
793b839
test: add investigation-reporting tests and remove unused prior-art t…
web-flow Apr 4, 2026
3057d4e
docs: update AGENTS.md to reflect current skill-bench usage
web-flow Apr 4, 2026
d8a2876
test: copy PROGRESS.md to log output for human review
web-flow Apr 4, 2026
0a25679
test: verify legal-checking skill is loaded in progress report test
web-flow Apr 4, 2026
f0f2121
test: add specific patent report functional test
web-flow Apr 4, 2026
c35d31e
refactor: inline legal-checking instructions and add context: fork
web-flow Apr 4, 2026
55e8435
refactor: inline targeting instructions into SKILL.md
web-flow Apr 4, 2026
208758f
test: fix targeting functional test checks
web-flow Apr 4, 2026
3e12617
test: fix investigation-reporting checks to use log-contains
web-flow Apr 4, 2026
24b24c8
refactor: output specific patent report to file with legal check
web-flow Apr 4, 2026
40f24d6
test: verify legal-checking is invoked in overall progress test
web-flow Apr 4, 2026
c48c0c7
refactor: use SQL-based data retrieval for investigation reporting
web-flow Apr 4, 2026
3308229
refactor: redesign specific-patent-report with phase-aware output
web-flow Apr 4, 2026
49155d0
test: add pending-phases test for specific patent report
web-flow Apr 4, 2026
5ad3273
test: fix overall-progress checks to match actual output content
web-flow Apr 4, 2026
76b380e
refactor: move screening stats to fetching and redesign progress repo…
web-flow Apr 4, 2026
de0c16a
fix: enforce template compliance in overall progress report
web-flow Apr 4, 2026
4db33a4
refactor: remove Verdict term from template and add prior art test
web-flow Apr 4, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,12 +33,12 @@ An autonomous daemon that runs inside the devcontainer and checks for failing Gi
- **Workflow**: Finds failing PRs → Runs `claude` with `--worktree` → Analyzes the failure (typically using `npm run lint`) → Commits the fix and replies to the PR.
- **Requirements**: Requires GitHub CLI (`gh`) authenticated inside the devcontainer.

### Skill-Bench (`agents/skill-bench/runner.sh`)
### Skill-Bench (`skill-bench`)

An autonomous test runner that executes E2E tests for the `patent-kit` skills using **TOML-based test cases**.
A TOML-based E2E test runner for `patent-kit` skills, installed via `mise`.

- **Architecture**: All execution happens inside the devcontainer. Test cases are defined in TOML format under `cases/<skill>/<test>.toml`.
- **Workflow**: Reads test cases from `cases/*/*.toml` → Sets up isolated workspaces → Runs `claude -p` with test prompts → Evaluates results using check scripts → Generates summary in `logs/`.
- **Usage**: `bash agents/skill-bench/runner.sh [pattern]` (default: `cases/*/*.toml`).
- **Test Case Format**: TOML files with `test_prompt`, `timeout`, `[[setup]]`, and `[[checks]]` sections.
- **Requirements**: Requires `yq` and `jq` commands inside the devcontainer.
- **Architecture**: All execution happens inside the devcontainer. Test cases are defined in TOML format under `tests/<skill>/<test>.toml`.
- **Workflow**: Reads test cases → Sets up isolated workspaces → Runs `claude -p` with test prompts → Evaluates results using built-in check types → Generates summary in `logs/`.
- **Usage**: `mise run test` (runs all tests under `tests/`).
- **Test Case Format**: TOML files with `test_prompt`, `timeout`, `[[setup]]`, and `[[checks]]` sections. See `skill-bench help` for available check types.
- **Check Types**: `skill-loaded`, `skill-invoked`, `mcp-loaded`, `mcp-tool-invoked`, `mcp-success`, `tool-use`, `tool-param`, `workspace-file`, `workspace-dir`, `file-contains`, `log-contains`, `message-contains`, `db-query`.
17 changes: 17 additions & 0 deletions plugin/skills/investigation-fetching/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,17 @@ internal reference files for this skill's internal use only.
- "Get next relevant patent for evaluation"
- "Get list of all relevant patents"
- "Get list of relevant patents without evaluation"
- "Count relevant patents"
- "Count relevant patents without evaluation"
- "Get list of unscreened patent IDs"
- "Get next patent for claim analysis"
- "Get elements for patent <patent-id>"
- "Get list of patents with elements but no similarities"
- "Count patents without similarities"
- "Count screening progress"
- "Count claim analysis progress"
- "Count patents without prior arts"
- "Count prior art progress"
- "Search features"
- "Search feature: <feature_name>"
- "Execute SQL: SELECT COUNT(\*) FROM screened_patents WHERE judgment = 'relevant'"
Expand Down Expand Up @@ -69,10 +76,17 @@ When processing external requests, map them to internal instruction files:
| "Get next relevant patent for evaluation" | references/instructions/get-next-patent.md |
| "Get list of relevant patents without..." | references/instructions/get-relevant-patents.md |
| "Get all relevant patents" | references/instructions/get-relevant-patents.md |
| "Count relevant patents" | references/instructions/get-relevant-patents.md |
| "Count relevant patents without..." | references/instructions/get-relevant-patents.md |
| "Get list of unscreened patent IDs" | references/instructions/get-unscreened-patents.md |
| "Get next patent for claim analysis" | references/instructions/get-next-claim-analysis-patent.md |
| "Get elements for patent..." | references/instructions/get-elements.md |
| "Get list of patents with elements but..." | references/instructions/get-patents-without-similarities.md |
| "Count patents without similarities" | references/instructions/get-patents-without-similarities.md |
| "Count screening progress" | references/instructions/get-screening-statistics.md |
| "Count claim analysis progress" | references/instructions/get-claim-analysis-statistics.md |
| "Count patents without prior arts" | references/instructions/get-patents-without-prior-arts.md |
| "Count prior art progress" | references/instructions/get-prior-art-statistics.md |
| "Search features" | references/instructions/get-features.md |
| "Search feature: <feature_name>" | references/instructions/search-feature.md |

Expand Down Expand Up @@ -229,6 +243,9 @@ agents should NOT read these:
- `get-elements.md`: Get elements for a specific patent
- `get-patents-without-similarities.md`: Get list of patents with elements but no similarities
- `get-features.md`: Get all product features
- `get-screening-statistics.md`: Get screening progress counts
- `get-claim-analysis-statistics.md`: Get claim analysis progress counts
- `get-prior-art-statistics.md`: Get prior art research progress counts
- `search-feature.md`: Search for a specific feature by name
- \*\*references/schema.md`: Database schema documentation

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Get Claim Analysis Statistics

## Purpose

Retrieve aggregate claim analysis progress counts.

## Request Pattern

"Count claim analysis progress"

## SQL Query

```bash
sqlite3 -json patents.db "
SELECT
COUNT(DISTINCT patent_id) AS all_count,
SUM(CASE WHEN max_sim = 1 THEN 1 ELSE 0 END) AS limited_count,
SUM(CASE WHEN max_sim > 1 THEN 1 ELSE 0 END) AS not_limited_count
FROM (
SELECT
patent_id,
MAX(CASE similarity_level
WHEN 'Significant' THEN 3
WHEN 'Moderate' THEN 2
WHEN 'Limited' THEN 1
END) AS max_sim
FROM similarities
GROUP BY patent_id
);
"
```

## Expected Output

JSON array with one row:

- `all_count`: Total patents with similarity results
- `limited_count`: Patents where all similarities are Limited
- `not_limited_count`: Patents with at least one Significant or Moderate similarity
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
# Get Patents Without Prior Arts

Retrieves list of patents with Moderate/Significant similarities but no prior art elements recorded yet.
Retrieves list of patents with Moderate/Significant similarities but no prior art
elements recorded yet.

## SQL Query
## Variations

### List Patents Without Prior Arts

```bash
sqlite3 -json patents.db "
Expand All @@ -20,14 +23,38 @@ AND e.patent_id NOT IN (
"
```

### Count Patents Without Prior Arts

```bash
sqlite3 -json patents.db "
SELECT COUNT(DISTINCT e.patent_id) AS count
FROM elements e
WHERE e.patent_id IN (
SELECT s.patent_id
FROM similarities s
GROUP BY s.patent_id
HAVING SUM(CASE WHEN s.similarity_level = 'Limited' THEN 1 ELSE 0 END) = 0
)
AND e.patent_id NOT IN (
SELECT patent_id FROM prior_art_elements
);
"
```

## Output Format

JSON array of patent_ids:
JSON array of patent_ids (for list queries):

```json
[{ "patent_id": "US20240292070A1" }, { "patent_id": "US20240346271A1" }]
```

JSON array with count (for count queries):

```json
[{ "count": 2 }]
```

Empty array if no patents pending:

```json
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@

Retrieves list of patents that have elements but no similarities recorded yet.

## SQL Query
## Variations

### List Patents Without Similarities

```bash
sqlite3 -json patents.db "
Expand All @@ -15,14 +17,33 @@ WHERE s.patent_id IS NULL;
"
```

### Count Patents Without Similarities

```bash
sqlite3 -json patents.db "
SELECT COUNT(DISTINCT e.patent_id) AS count
FROM elements e
LEFT JOIN similarities s ON e.patent_id = s.patent_id
AND e.claim_number = s.claim_number
AND e.element_label = s.element_label
WHERE s.patent_id IS NULL;
"
```

## Output Format

JSON array of patent_ids:
JSON array of patent_ids (for list queries):

```json
[{ "patent_id": "US20240292070A1" }, { "patent_id": "US20240346271A1" }]
```

JSON array with count (for count queries):

```json
[{ "count": 3 }]
```

Empty array if no patents pending:

```json
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# Get Prior Art Statistics

## Purpose

Retrieve aggregate prior art research progress counts, scoped to Not Limited
patents (Significant/Moderate similarity only).

## Request Pattern

"Count prior art progress"

## SQL Query

```bash
sqlite3 -json patents.db "
SELECT
not_limited.all_count,
COALESCE(resolved.resolved_count, 0) AS resolved_count,
COALESCE(open_pat.open_count, 0) AS open_count,
not_limited.all_count - COALESCE(resolved.resolved_count, 0) - COALESCE(open_pat.open_count, 0) AS pending_count
FROM (
SELECT COUNT(*) AS all_count
FROM (
SELECT patent_id
FROM similarities
GROUP BY patent_id
HAVING MAX(CASE similarity_level
WHEN 'Significant' THEN 3
WHEN 'Moderate' THEN 2
WHEN 'Limited' THEN 1
END) > 1
)
) AS not_limited
LEFT JOIN (
SELECT COUNT(DISTINCT patent_id) AS resolved_count
FROM prior_art_elements
WHERE relevance_level = 'Significant'
AND patent_id IN (
SELECT patent_id
FROM similarities
GROUP BY patent_id
HAVING MAX(CASE similarity_level
WHEN 'Significant' THEN 3
WHEN 'Moderate' THEN 2
WHEN 'Limited' THEN 1
END) > 1
)
) AS resolved ON 1 = 1
LEFT JOIN (
SELECT COUNT(DISTINCT patent_id) AS open_count
FROM prior_art_elements
WHERE patent_id IN (
SELECT patent_id
FROM similarities
GROUP BY patent_id
HAVING MAX(CASE similarity_level
WHEN 'Significant' THEN 3
WHEN 'Moderate' THEN 2
WHEN 'Limited' THEN 1
END) > 1
)
AND patent_id NOT IN (
SELECT DISTINCT patent_id
FROM prior_art_elements
WHERE relevance_level = 'Significant'
)
) AS open_pat ON 1 = 1;
"
```

## Expected Output

JSON array with one row:

- `all_count`: Total Not Limited patents (Significant/Moderate similarity)
- `resolved_count`: Patents with prior art elements having Significant relevance
- `open_count`: Patents with prior art elements but none with Significant relevance
- `pending_count`: Not Limited patents with no prior art elements at all

## Verification

`all_count` = `resolved_count` + `open_count` + `pending_count`
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,35 @@ WHERE judgment = 'relevant'
"
```

### Count Relevant Patents

```bash
sqlite3 -json patents.db "
SELECT COUNT(*) AS count FROM screened_patents
WHERE judgment = 'relevant';
"
```

### Count Relevant Patents Without Evaluation

```bash
sqlite3 -json patents.db "
SELECT COUNT(*) AS count FROM screened_patents
WHERE judgment = 'relevant'
AND patent_id NOT IN (SELECT patent_id FROM claims);
"
```

## Output Format

JSON array of patent_ids:
JSON array of patent_ids (for list queries):

```json
[{ "patent_id": "US20240292070A1" }, { "patent_id": "US20240346271A1" }]
```

JSON array with count (for count queries):

```json
[{ "count": 5 }]
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Get Screening Statistics

## Purpose

Retrieve aggregate screening progress counts from the database.

## Request Pattern

"Count screening progress"

## SQL Query

```bash
sqlite3 -json patents.db "SELECT * FROM v_screening_progress"
```

## Expected Output

JSON array with one row:

- `total_targets`: Total patents in targeting
- `total_screened`: Total patents screened
- `relevant`: Relevant patent count
- `irrelevant`: Irrelevant patent count
- `expired`: Expired patent count
3 changes: 1 addition & 2 deletions plugin/skills/investigation-preparing/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,10 @@ description: |
Initializes the patent investigation database and imports CSV files.

Use this skill to set up the SQLite database (patents.db) before running
screening. Supports database initialization, CSV import, and progress queries.
screening. Supports database initialization and CSV import.

Example usage:
- "Initialize the patent database and import CSV files from csv/"
- "Get screening progress statistics"
context: fork
---

Expand Down
Loading
Loading