feat: Path Portability Lint Gate for Distributed AI Artifacts in CI

[katriendg]

Summary

Distributed AI artifacts (agents, instructions, prompts, skills) contain hardcoded .github/ paths in prose content that break when loaded outside the source repository — VS Code extension installs, peer clones, and Copilot CLI plugins. We propose adding a lint validation gate to prevent future regressions and surface existing violations.

Background

Issue #741 documents the root problem: artifacts instruct the AI to read files using workspace-root-anchored paths like .github/instructions/design-thinking/dt-coaching-identity.instructions.md. When installed via extension, the actual file location is something like ~/.vscode-server-insiders/data/User/globalStorage/ise-hve-essentials.hve-core/.github/... — the AI resolves .github/ relative to the workspace root, not the artifact's disk location, so the file is not found.

The skills contributing guide already documents path portability rules and has structural validation via validate:skills. Agents, instructions, and prompts have no equivalent documentation or validation.

Proposal

New validation script

A PowerShell script at scripts/linting/Validate-ArtifactPathPortability.ps1 that:

• Scans distributable artifact files (.agent.md, .instructions.md, .prompt.md, SKILL.md) under .github/{agents,instructions,prompts,skills}/{collection-id}/...
• Detects .github/ path references in prose content only (outside fenced code blocks)
• Follows existing script conventions: imports LintingHelpers.psm1 + CIHelpers.psm1, supports -ChangedFilesOnly, -WarningsAsErrors, -BaseBranch
• Outputs JSON results to logs/artifact-path-portability-results.json
• Emits CI annotations with file, line number, and context snippet

Integration

• New npm script: lint:path-portability
• Chained into lint:all between lint:version-consistency and validate:skills

What is NOT scanned

Excluded	Reason
copilot-instructions.md	Root-level repo-specific artifact, never distributed
Root-level artifacts (e.g., .github/instructions/workflows.instructions.md)	Repo-specific, no subdirectory = not distributed
Content inside fenced code blocks	Code examples may demonstrate paths without intending resolution
YAML frontmatter	applyTo and other fields legitimately use .github/ glob patterns
docs/, scripts/, collections/, plugins/	Not AI artifacts, different distribution model
.github/deprecated/**	Already excluded from distribution

Risks and Edge Cases

1. False positives on informational/structural references

Some artifacts reference .github/ paths as documentation about repo structure, not as file paths the AI should resolve. Example from commit-message.instructions.md:

- `(agents)` - Custom agent definitions in `.github/agents/`

This describes a scope mapping. The AI doesn't navigate to it.

2. Teaching and meta content

prompt-builder.instructions.md teaches authors how artifacts are structured — it must reference .github/ paths to explain the directory layout. These are instruction-about-instructions.

3. Subagent glob patterns

Many agents use .github/agents/**/researcher-subagent.agent.md as a resolution pattern. These are real violations (they break in extension context), but fixing them requires changing how subagents are referenced — not just removing the path.

4. #file: directives with workspace-root paths

#file:.github/... is also workspace-root-anchored and should be flagged. The fix is to use relative paths like #file:../../instructions/....

Proposed Suppression Mechanisms

To handle legitimate .github/ references, authors can suppress detection:

Line-level suppression

<!-- path-portability-ignore-next-line -->
* `(agents)` - Custom agent definitions in `.github/agents/`

Block-level suppression

<!-- path-portability-ignore-start -->

| Directory | Purpose |
|---|---|
| `.github/agents/**` | Agent definitions |
| `.github/prompts/**` | Prompt templates |

<!-- path-portability-ignore-end -->

Severity and Rollout

Phase	Behavior	Purpose
Phase 1 (this PR)	Warnings only (exit 0)	Baseline violations, no CI breakage
Phase 2 (after #741 fixes land)	-WarningsAsErrors (exit 1)	Blocking gate, prevent regressions

Starting as warnings-only avoids blocking CI while providing visibility into the ~50 existing violations across the codebase.

Current Violations (Preview)

Based on analysis of the affected files listed in #741:

File	Approx. Violations	Nature
rpi-agent.agent.md	~15	Subagent glob paths, instruction references
doc-ops.agent.md	~12	Subagent paths, scope tables, markdown links
prompt-builder.instructions.md	~8	Teaching/meta content (mostly suppressible)
commit-message.instructions.md	~6	Scope descriptions (mostly suppressible)
dt-coach.agent.md	4	Ambient instruction references

Documentation Updates

The contributor docs gap should be addressed alongside or shortly after the lint gate:

• docs/contributing/ai-artifacts-common.md: Add a shared "Path Portability" section that all artifact type guides reference
• docs/contributing/custom-agents.md: Add path portability guidance (currently absent)
• docs/contributing/instructions.md: Add path portability guidance (currently absent)
• docs/contributing/skills.md: Already well-documented, no changes needed

Questions for Discussion

1. Suppression granularity — Are line-level, block-level, and file-level suppressions all needed, or would a subset suffice?
2. Phase 2 timing — Should -WarningsAsErrors be flipped in the same PR that fixes fix: Distributed AI artifacts reference hardcoded workspace-root paths that break in extension and peer-clone installs #741 violations, or as a separate follow-up?
3. Scope expansion — Should the scanner also flag absolute paths like /workspaces/hve-core/... or C:\Users\...? These are rare but equally non-portable.
4. copilot-instructions.md Project Structure section — This file is repo-specific and excluded from the scan, but its .github/ references are still consumed by the AI in-workspace. Should it be separately addressed?

References

• fix: Distributed AI artifacts reference hardcoded workspace-root paths that break in extension and peer-clone installs #741 — Fix: Distributed AI artifacts reference hardcoded workspace-root paths
• fix: AI generated .copilot-tracking files produce broken relative paths flooding VS Code Problems tab #698 — Generated .copilot-tracking files produce broken relative paths
• fix: Broken #file: directives and markdown links after collection directory reorg #742 — Broken #file: directives and markdown links after collection directory reorg
• docs/contributing/skills.md — Path Portability section

[katriendg]

@WilliamBerryiii @agreaves-ms I've started this thinking, not entirely finished as we need to ensure our fix works well with relative paths once we distribute through Extension and Plugins (see related issues above), but once we do validate, I believe having a CI linting addition could be beneficial for new AI artifacts contributions.
Wdyt?