feat(rule-tracker): per-repo origin tagging + cross-repo pollution guard#35
Merged
rladmsgh34 merged 8 commits intomainfrom Apr 26, 2026
Merged
feat(rule-tracker): per-repo origin tagging + cross-repo pollution guard#35rladmsgh34 merged 8 commits intomainfrom
rladmsgh34 merged 8 commits intomainfrom
Conversation
The 30 existing rules in rules-history.json were the dataset for loop-efficacy measurement, but inspection revealed pollution: rule X induced from gwangcheon-shop carried snapshots from later vue/core runs, making "did this rule reduce regressions" structurally unanswerable. Schema additions - rules[].origin_repo: 'owner/repo' | 'self' | 'unknown' - rules[].origin_confidence: 'explicit' (recorded at induction) | 'inferred' (heuristic from rule text) | 'unknown' - snapshots[].repo: which target repo's analysis produced this snapshot Pollution guard - record_snapshot now skips attaching to a rule when the rule's origin_repo doesn't match the snapshot's repo. Legacy unknown-origin rules accept all snapshots (best-effort backward compat). - compute_effectiveness gains repo / include_self / include_inferred flags. Self-rules are excluded by default — measuring them would require the analyzer analyzing itself (recursive, unstable). Migration (idempotent) - scripts/migrate_rules_origin.py classifies the 30 legacy rules: 6 explicit self, 7 inferred vuejs/core, 17 unknown. Re-runs skip already-tagged rules. Workflow - Step 3 passes TARGET_REPO when calling record_snapshot and record_new_rules. New rules always recorded as explicit fact (we know which repo we're analyzing at that moment). - analyzer's own repo recorded as 'self' for the special case (matrix never targets analyzer itself, but local self-runs would). Living document framing - effectiveness_summary header marks output as accumulating; n=7 vue rules + min_days=7 means real numbers won't appear until the next cron cycle. CLI now supports --repo, --include-self, --strict. Tests - 7 new tests in tests/test_rule_tracker.py cover origin recording, pollution guard (the load-bearing one), repo filter, self exclusion, and strict mode. 94/94 pass.
Dry-run on PR branch (run 24957960684) verified the new origin tagging works for fresh writes (15 new rules tagged with explicit owner/repo, new snapshots carry repo field), but exposed leftover pollution: the 30 legacy rules retained 75 rule-attached snapshots and 1 global snapshot from before per-repo tracking landed. Those have no repo field, so the pollution guard can't tell which target produced them and downstream efficacy calc would still mix data. Migration now also purges any snapshot lacking a repo field. Same idempotent contract — already-clean histories stay no-op on re-run. Verified - Local migration purged 1 global + 75 rule-attached dirty snapshots - Second run: 0/0 purge (idempotent) - Live data file: zero repo-less snapshots remain - New test covers both purge correctness and idempotence
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
loop-efficacy 측정 무력화 원인 발견 —
rules-history.json이 레포 간 공유라 vue snapshot이 gwangcheon 유발 rule에 누적되며 pollution 발생. measurement infra만 깔고 숫자는 자라기 기다리는 living document 방식으로 baseline lock-in 회피.스키마 변경
rules[].origin_repo'owner/repo'|'self'|'unknown'rules[].origin_confidence'explicit'(induction 시점 fact) |'inferred'(heuristic) |'unknown'snapshots[].repoPollution 가드
record_snapshot: rule의 origin_repo와 다른 repo snapshot은 attach 안 함unknownorigin rule은 모든 snapshot 받음 (backward compat best-effort)compute_effectiveness:repo/include_self/include_inferred플래그self기본 제외 — analyzer가 자기 자신 분석하는 재귀 측정 신뢰도 문제마이그레이션 (idempotent)
scripts/migrate_rules_origin.py결과:selfvuejs/coreunknown→ default efficacy 계산은 vue 7 rule만 사용. 두 번째 실행은 30개 모두 skip (idempotent 검증 완료).
Living document 명시
effectiveness_summary헤더: "데이터 누적과 함께 정확도 상승"시퀀스 위치
이 PR은 measurement infra (#1). 다음:
Test plan
pytest— 94/94 통과 (87 prior + 7 new rule_tracker)--repo,--strict, default)Out of scope
🤖 Generated with Claude Code