From 2963bd4d298445b7c88fc2940bb17fd6a8a321d0 Mon Sep 17 00:00:00 2001 From: eshulman2 Date: Wed, 29 Apr 2026 14:50:56 +0300 Subject: [PATCH] proposals: add skill packages and repo config via Jira project metadata - 009: skill packages fetched automatically from forge.skills project property, eliminating manual installation and per-host config - 010: GITHUB_KNOWN_REPOS and GITHUB_DEFAULT_REPO moved to forge.repos and forge.default_repo project properties with .env fallback Co-Authored-By: Claude Sonnet 4.6 (1M context) --- proposals/009-skill-installer.md | 206 ++++++++++++++++++++++++ proposals/010-project-metadata-repos.md | 149 +++++++++++++++++ 2 files changed, 355 insertions(+) create mode 100644 proposals/009-skill-installer.md create mode 100644 proposals/010-project-metadata-repos.md diff --git a/proposals/009-skill-installer.md b/proposals/009-skill-installer.md new file mode 100644 index 0000000..5d66551 --- /dev/null +++ b/proposals/009-skill-installer.md @@ -0,0 +1,206 @@ +# Proposal: Skill Packages via Jira Project Metadata + +**Author:** eshulman2 +**Date:** 2026-04-29 +**Status:** Draft + +## Summary + +Store skill package sources as Jira project properties so the worker automatically fetches and installs the right skills for each project without any CLI intervention or `.env` changes. A CLI command remains available as a secondary tool for local development and one-off installs. + +## Motivation + +### Problem Statement + +Skills are the primary customization mechanism in Forge. Every team using Forge needs custom skills tuned to their stack. The current path is to manually author skill files in `skills//`. Several pain points arise: + +1. **No external sourcing.** There is no way to pull skills from a shared git repo — only copy-paste. +2. **No per-project config.** Skill configuration, like repo config, is global (`.env`). Two projects cannot have different skill sources without separate deployments. +3. **Bootstrap friction.** A new project has no skills until someone manually authors them. +4. **Operational overhead.** Installing or updating skills requires shelling into the host and restarting nothing (the worker already re-resolves skills per task), but there is no standard mechanism to do it. + +### Current Workarounds + +Teams copy skill files manually between projects or maintain a single shared `skills/default/` that grows to cover all stacks. + +## Proposal + +### Overview + +Use Jira project properties to store a list of skill package sources for each project. When the worker processes a ticket, it reads the project's `forge.skills` property, fetches any packages not yet present (or at a stale commit), copies their skill directories into `skills//`, and proceeds. The resolver and container runner are unchanged. + +Jira project properties are key-value metadata on the project itself, managed via the Jira REST API. They are already scoped per project, editable by project admins, and readable with the existing Jira credentials. + +### Detailed Design + +#### Jira Project Property + +Property key: `forge.skills` + +Value (JSON): +```json +[ + { + "source": "https://github.com/acme/forge-skills-python", + "ref": "v1.2.0" + }, + { + "source": "https://github.com/acme/forge-skills-shared", + "ref": "a3f8c21" + } +] +``` + +`ref` is optional. If omitted, the worker fetches `HEAD` of the default branch. `ref` can be a tag, branch, or commit SHA. + +#### Skill Package Format + +A skill package is a git repo whose root contains skill subdirectories, each with a `SKILL.md`: + +``` +forge-skills-python/ + generate-prd/ + SKILL.md + prd-template.md + implement-task/ + SKILL.md +``` + +An optional `forge-skills.yaml` manifest at the root can declare a `min_forge_version` guard. + +#### Worker Fetch Flow + +At the start of `setup_workspace` (before any skill resolution), the worker: + +1. Reads the `forge.skills` project property via Jira API, keyed on the project extracted from the ticket key. +2. Compares each entry against `skills/skills.lock`. If the source and resolved ref match the lock, skip. Otherwise fetch. +3. For each stale or missing package: `git clone --depth 1 --branch ` into a temp dir, copy skill subdirectories into `skills//`, update the lock entry. +4. Proceeds to skill resolution as normal. + +Step 1 result is cached in memory (keyed by project key) for the lifetime of the worker process to avoid a Jira API call on every task. + +#### Lock File + +`skills/skills.lock` tracks what was fetched and when, for auditability: + +```yaml +packages: + - source: https://github.com/acme/forge-skills-python + ref: v1.2.0 + resolved_commit: a3f8c21 + target: myproj + skills: + - generate-prd + - implement-task + fetched_at: 2026-04-29T14:00:00Z +``` + +The lock is updated on the host by the worker. It is not used to gate fetches (the project property is the source of truth); it exists for auditing and debugging. + +#### Jira Client Addition + +A new method on `JiraClient`: + +```python +async def get_project_property(self, project_key: str, property_key: str) -> Any | None: + """Fetch a project-level property value, or None if not set.""" +``` + +Using the existing Jira REST endpoint: `GET /rest/api/3/project/{projectKey}/properties/{propertyKey}`. + +#### CLI as Secondary Tool + +`forge skills install --project ` remains available for: +- Local development and testing of new skill packages before publishing +- One-off installs from local paths (not suitable for Jira property storage) +- Environments where the project property is not set + +It performs the same fetch-and-copy logic but writes directly to `skills//` and updates the lock file, without touching Jira. + +``` +forge skills install (--project | --default) [--ref ] +forge skills list +forge skills update [--project ] +``` + +### User Experience + +**Setting up skills for a new project (Jira UI or API):** +``` +Project: MYPROJ +Property key: forge.skills +Value: +[ + {"source": "https://github.com/acme/forge-skills-python", "ref": "v1.2"} +] +``` + +**Worker log on next task for MYPROJ-101:** +``` +INFO Fetching skill packages for project MYPROJ +INFO forge-skills-python: fetching v1.2 from github.com/acme/forge-skills-python +INFO forge-skills-python: installed generate-prd, implement-task (2 skills) +INFO Skills resolved for MYPROJ-101: [skills/default/, skills/myproj/] +``` + +**Subsequent tasks (cache hit):** +``` +INFO Skills resolved for MYPROJ-102: [skills/default/, skills/myproj/] +``` + +**Local install for testing:** +```bash +$ forge skills install ./my-draft-skill --project myproj +Installed 1 skill into skills/myproj/: generate-prd (overwrote existing) +``` + +## Alternatives Considered + +| Alternative | Pros | Cons | Why Not | +|-------------|------|------|---------| +| CLI-only installer | Simple, no Jira dependency | Manual step on every update, per-host | Operational friction defeats the purpose | +| `gh skill` (GitHub CLI) | Existing tool, maintained | Targets `.claude/skills/`, not Forge dirs; adds `gh` dependency | Wrong target, wrong abstraction | +| `.env` skill sources | Simple | Global, not per-project; requires restart awareness | Same problem as `GITHUB_KNOWN_REPOS` | +| Git submodules | Native git tracking | Complex UX, merge conflicts | Installer abstraction is simpler | + +## Implementation Plan + +### Phases + +1. **Phase 1: Jira project property client** — 1 day + - Add `get_project_property` to `JiraClient`. + - Add in-memory cache (dict keyed by project key, populated lazily). + +2. **Phase 2: Fetch and install logic** — 2 days + - `src/forge/skills/installer.py`: fetch, copy, lock file read/write. + - Wire into `setup_workspace` node before skill resolution. + +3. **Phase 3: CLI sub-command** — 1 day + - Add `forge skills install / list / update` under `cli.py`. + - Reuses installer module from Phase 2. + +### Dependencies + +- [ ] `git` on host (already assumed by workspace manager) +- [ ] `skills/skills.lock` added to `.gitignore` or committed — recommend committed for auditability + +### Risks + +| Risk | Likelihood | Impact | Mitigation | +|------|------------|--------|------------| +| Malicious skill package replaces built-in skills | Low | Medium | Skills affect container prompts only, not orchestrator logic; project admins control the property | +| Jira API unavailable at task start | Low | Low | Fall back to whatever is already on disk; log a warning | +| Skill update mid-sprint changes agent behavior unexpectedly | Medium | Medium | Pin to a tag or commit SHA in the property value | +| Local modifications to installed skills overwritten on update | Medium | Medium | Log a warning when overwriting a file with local git changes | + +## Open Questions + +- [ ] Should the in-memory cache have a TTL (e.g. 1 hour) so long-running workers pick up property changes without restart? +- [ ] Should `forge skills update` also push the resolved commit back to the Jira property (replacing a branch ref with a commit SHA for reproducibility)? +- [ ] Should overwriting locally modified skill files on update be a hard error or a warning? + +## References + +- Current resolver: `src/forge/skills/resolver.py` +- Proposal 004 (dynamic skill loading, implemented): `proposals/004-dynamic-skill-loading.md` +- Proposal 010 (project metadata for repo config): `proposals/010-project-metadata-repos.md` diff --git a/proposals/010-project-metadata-repos.md b/proposals/010-project-metadata-repos.md new file mode 100644 index 0000000..5bfd497 --- /dev/null +++ b/proposals/010-project-metadata-repos.md @@ -0,0 +1,149 @@ +# Proposal: Repository Configuration via Jira Project Metadata + +**Author:** eshulman2 +**Date:** 2026-04-29 +**Status:** Draft + +## Summary + +Move per-project GitHub repository configuration (`GITHUB_KNOWN_REPOS`, `GITHUB_DEFAULT_REPO`) from global `.env` settings to Jira project properties, making repo assignment project-specific and manageable without touching the host environment. + +## Motivation + +### Problem Statement + +Repository assignment in Forge is global: `GITHUB_KNOWN_REPOS` is a comma-separated list shared across all projects. This causes two problems: + +1. **No per-project scoping.** All projects see the same set of repos. A team running `MYPROJ` tickets has no way to restrict or extend repo choices independently of a team running `OTHERPROJ` tickets on the same Forge instance. +2. **Operational friction.** Adding a new repo for one project requires editing `.env` and is visible to all projects. On a shared Forge deployment this is noise at best, a security concern at worst. + +The same issue applies to `GITHUB_DEFAULT_REPO` — there is one global default, not one per project. + +### Current Workarounds + +Teams list all repos from all projects in `GITHUB_KNOWN_REPOS` and rely on the Epic decomposition agent to assign repos correctly from context. This works but is fragile and leaks cross-project repo information. + +## Proposal + +### Overview + +Store per-project repo configuration as Jira project properties, read at task time alongside the existing label-based repo hints. Global `.env` values remain as a fallback for projects that have not set their own metadata. + +### Detailed Design + +#### Jira Project Properties + +Two property keys on the Jira project: + +**`forge.repos`** — the set of repos available to this project for Epic/task assignment: +```json +["acme/backend", "acme/frontend", "acme/infra"] +``` + +**`forge.default_repo`** — the repo to assign when no explicit assignment is made: +```json +"acme/backend" +``` + +#### Updated Repo Resolution in Epic Decomposition + +Current logic in `epic_decomposition.py` builds `available_repos` from: +1. Feature ticket labels (`repo:owner/repo-name`) +2. `settings.known_repos` (global) + +Updated logic: +1. Feature ticket labels (`repo:owner/repo-name`) +2. `forge.repos` Jira project property (project-specific) +3. `settings.known_repos` (global fallback, used only if property is not set) + +`forge.default_repo` project property overrides `settings.github_default_repo` for tickets in that project. + +#### Jira Client Addition + +Reuses `get_project_property` introduced in proposal 009. No additional Jira client changes needed. + +#### Caching + +Same in-memory cache strategy as proposal 009: property values are fetched once per project key per worker lifetime, with an optional TTL. + +#### Backwards Compatibility + +`GITHUB_KNOWN_REPOS` and `GITHUB_DEFAULT_REPO` in `.env` continue to work as before. They are used when the corresponding project property is absent. Existing deployments require no changes. + +### User Experience + +**Before (global `.env`):** +```env +GITHUB_KNOWN_REPOS=acme/backend,acme/frontend,acme/infra,other/repo +GITHUB_DEFAULT_REPO=acme/backend +``` + +**After (per project in Jira):** + +Project `MYPROJ`: +``` +forge.repos → ["acme/backend", "acme/frontend"] +forge.default_repo → "acme/backend" +``` + +Project `OTHERPROJ`: +``` +forge.repos → ["other/repo"] +forge.default_repo → "other/repo" +``` + +`.env` fallback (for projects without metadata): +```env +GITHUB_DEFAULT_REPO=acme/backend +``` +`GITHUB_KNOWN_REPOS` can be left empty or removed once all active projects have their metadata set. + +**Worker log:** +``` +INFO Project MYPROJ: repos from Jira property: [acme/backend, acme/frontend] +INFO Project MYPROJ: default repo: acme/backend +``` + +## Alternatives Considered + +| Alternative | Pros | Cons | Why Not | +|-------------|------|------|---------| +| Keep global `.env` | No change | Not per-project, friction to update | The problem is real at scale | +| Repo config in `CLAUDE.md` or a config file per project | Version-controlled | Requires file in every repo, not Jira-native | Forge config should live with the ticket system | +| Jira labels on the project (not properties) | Visible in UI | Labels are for issues, not projects | Wrong API surface | + +## Implementation Plan + +### Phases + +1. **Phase 1: Project property read** — 0.5 days + - Reuse `get_project_property` from proposal 009 (or implement it first if 009 is not yet merged). + - Add `get_project_repos(project_key)` and `get_project_default_repo(project_key)` helpers that wrap the property fetch with the fallback logic. + +2. **Phase 2: Wire into epic decomposition** — 0.5 days + - Replace direct `settings.known_repos` read with `get_project_repos`. + - Replace `settings.github_default_repo` read with `get_project_default_repo`. + - Add tests covering property-set and fallback-to-env cases. + +### Dependencies + +- [ ] `get_project_property` on `JiraClient` (shared with proposal 009 — whichever lands first provides it) + +### Risks + +| Risk | Likelihood | Impact | Mitigation | +|------|------------|--------|------------| +| Project property not set for an existing project | High (migration period) | Low | Falls back to `GITHUB_KNOWN_REPOS` transparently | +| Jira API unavailable | Low | Low | Falls back to global settings; logs a warning | +| Project admin sets an invalid repo name | Low | Medium | Validate `owner/repo` format on read; log and skip malformed entries | + +## Open Questions + +- [ ] Should we add `forge.fork_owner` as a project property too, to allow per-project fork targets? +- [ ] Should there be a `forge config show ` CLI command that dumps all resolved project metadata (repos, default repo, skills) for debugging? + +## References + +- Current usage: `src/forge/workflow/nodes/epic_decomposition.py` (lines 63–75) +- Current config: `src/forge/config.py` (`github_known_repos`, `known_repos` property) +- Proposal 009 (skill packages via project metadata): `proposals/009-skill-installer.md`