feat(marketplace): add curated upstream sources (closes #1136)

CHANGELOG.md

@@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+
+- **Marketplace upstreams: curated pass-through with allow-list governance.** Selectively re-expose plugins from an external marketplace under your own, with build-time commit pinning in `apm.lock.yaml` and Anthropic-conformant emission (no `metadata.apm.*` keys injected). New `upstreams:` block in `apm.yml -> marketplace:`; new `apm marketplace upstream add/list/remove` CLI; `packages[]` entries pick between `source:` (direct) and `upstream:` + `plugin:` (pass-through). APM does **not** re-host content; consumer installs always fetch from the upstream git host. See the [Marketplace upstreams guide](https://microsoft.github.io/apm/guides/marketplace-upstreams/). (#1136)
+
 ## [0.12.2] - 2026-05-05
 
 ### Added

docs/astro.config.mjs

@@ -77,6 +77,7 @@ export default defineConfig({
 						{ label: 'Org-Wide Packages', slug: 'guides/org-packages' },
 						{ label: 'Marketplaces', slug: 'guides/marketplaces' },
 						{ label: 'Marketplace Authoring', slug: 'guides/marketplace-authoring' },
+						{ label: 'Marketplace upstreams', slug: 'guides/marketplace-upstreams' },
 						{ label: 'CI Policy Enforcement', slug: 'guides/ci-policy-setup' },
 						{ label: 'Agent Workflows (Experimental)', slug: 'guides/agent-workflows' },
 					],

docs/src/content/docs/enterprise/security.md

@@ -37,7 +37,9 @@ APM has no runtime footprint. Once `apm install` or `apm compile` completes, the
 
 ## Dependency provenance
 
-APM resolves dependencies directly from git repositories. There is no intermediary registry, proxy, or mirror.
+APM resolves dependencies directly from git repositories. APM does not run an artifact registry or proxy server, and does not re-host third-party content.
+
+Curators MAY publish a marketplace that exposes plugins from external upstream marketplaces via the [marketplace upstreams](../../guides/marketplace-upstreams/) feature. Upstreams are a curated allow-list with build-time commit pinning -- not a binary mirror. Consumer installs always fetch plugin content from the original git host; APM never proxies or re-hosts that content. See [Marketplace upstreams: trust model](../../guides/marketplace-upstreams/#trust-model) for the full contract.
 
 ### Exact commit pinning
 

docs/src/content/docs/guides/marketplace-authoring.md

@@ -481,6 +481,7 @@ Not in the first release. `apm marketplace publish` uses the `gh` CLI and assume
 ## Related reading
 
 - [Marketplaces guide](../marketplaces/) -- consumer-side: registering and installing from a marketplace.
+- [Marketplace upstreams](../marketplace-upstreams/) -- expose plugins from external marketplaces with allow-list governance and immutable commit pinning.
 - [CLI command reference](../../reference/cli-commands/) -- authoritative options for `apm pack` and every `apm marketplace` subcommand.
 - [Manifest schema](../../reference/manifest-schema/) -- the `apm.yml` shape including the `marketplace:` block.
 - [Plugins guide](../plugins/) -- what a plugin is and how consumers install one.

docs/src/content/docs/guides/marketplace-upstreams.md

@@ -0,0 +1,151 @@
+---
+title: "Marketplace upstreams"
+description: Selectively expose plugins from external marketplaces with allow-list governance and immutable commit pinning.
+sidebar:
+  order: 7
+---
+
+Upstreams are APM's equivalent of Artifactory remote repositories -- they let your internal marketplace selectively expose plugins from external sources, with allow-list governance and immutable commit pinning, without running an artifact server.
+
+This guide is for **marketplace curators** who want to re-expose plugins from a third-party marketplace (for example, a public Claude Code marketplace) inside their own marketplace, with control over which plugins are exposed and at what version. If you are authoring a marketplace from scratch, start with the [Authoring a marketplace](../marketplace-authoring/) guide first.
+
+## Quick start
+
+Register an external marketplace under a local alias, then expose one of its plugins:
+
+```bash
+# 1. Register the upstream marketplace, pinned to an immutable commit.
+#    Use a real 40-char SHA (preferred over tags -- tags can be re-pointed).
+#    Get the current SHA with:  git ls-remote https://github.com/owner/repo HEAD
+apm marketplace upstream add owner/repo \
+  --alias myupstream \
+  --ref 0000000000000000000000000000000000000001
+
+# 2. Confirm the upstream is registered in apm.yml
+apm marketplace upstream list
+
+# 3. Expose one plugin under your own display name
+apm marketplace package add \
+  --upstream myupstream \
+  --plugin original-plugin-name \
+  --name my-plugin
+
+# 4. Build your marketplace.json
+apm pack
+```
+
+The emitted `marketplace.json` is byte-for-byte Anthropic-conformant -- it does **not** carry any APM-specific keys. Provenance (manifest SHA, resolved plugin SHA, canonical owner) is recorded only in your `apm.lock.yaml` under the `upstreams:` section.
+
+## Schema
+
+In `apm.yml`:
+
+```yaml
+marketplace:
+  upstreams:
+    - alias: myupstream
+      repo: owner/repo
+      path: .claude-plugin/marketplace.json    # default
+      ref: <sha-or-tag>                         # required for reproducibility
+      branch: main                              # used only with allow_head
+      host: github.com                          # default
+      allow_head: false                         # default; opt-in to mutable refs
+  packages:
+    # Direct package
+    - name: my-skill
+      source: owner/repo
+      version: ">=1.0.0"
+
+    # Upstream-sourced package
+    - name: my-plugin                  # display name in your marketplace
+      upstream: myupstream             # references upstreams[].alias
+      plugin: original-plugin-name    # name in the upstream marketplace
+      version: ">=1.0.0"               # optional curator override
+```
+
+`upstream` and `source` are mutually exclusive on a single `packages[]` entry.
+
+## CLI reference
+
+| Command | Purpose |
+|---|---|
+| `apm marketplace upstream add <repo> --alias <alias> --ref <sha>` | Register an upstream marketplace pinned to a 40-char SHA (recommended) |
+| `apm marketplace upstream add <repo> --alias <alias> --ref v1.2.3` | Pin to an annotated tag (acceptable for stable upstreams; SHA still preferred) |
+| `apm marketplace upstream add <repo> --alias <alias> --branch main --allow-head` | Track a mutable branch -- requires explicit `--allow-head` opt-in (warned every build) |
+| `apm marketplace upstream list` | List registered upstreams |
+| `apm marketplace upstream remove <alias> [--yes]` | Remove an upstream (rejects if any package still references it) |
+| `apm marketplace package add --upstream <alias> --plugin <name> [--name ...]` | Expose an upstream plugin in your `packages[]` |
+
+### Tag vs SHA: when to use which
+
+- **Always prefer a 40-char SHA.** It is content-addressed: even if the upstream force-pushes the branch the tag points at, your build keeps resolving the original tree.
+- **Tags are acceptable** when the upstream maintainer has a strong stable-tag discipline (annotated, signed, never moved). APM still resolves the tag to its current SHA at build time and writes the resolved SHA to `apm.lock.yaml` -- so reproducibility holds for that lockfile, but a fresh `add` after the tag moves will resolve to a new SHA.
+- **Branches** (`--branch main --allow-head`) are explicitly opt-in. Every build emits a warning, and enterprise policy can reject HEAD-tracking entries entirely.
+
+## Reproducibility
+
+Every build pins:
+
+- The **upstream `marketplace.json` commit SHA** (so the manifest itself can't change under you).
+- Each **upstream plugin's resolved commit SHA** (so the plugin source code can't change under you).
+
+These pins are written to `apm.lock.yaml` under `upstreams:`. Subsequent rebuilds replay from the lock and produce byte-identical output.
+
+:::note[Planned]
+`apm marketplace upstream refresh` is not yet implemented. To advance the pins today, re-run `apm marketplace upstream add` with a new `--ref` value. A dedicated `refresh` command that shows an old-SHA-to-new-SHA diff before committing is planned for a future release.
+:::
+
+## Failure modes
+
+The builder fails closed on every upstream-resolution problem (exit code `2`) rather than silently skipping. You will see one of these named errors:
+
+```text
+[x] upstream alias 'myupstream' is not declared in marketplace.upstreams
+```
+
+The `--upstream` value on a `packages[]` entry must reference a declared `upstreams[].alias`. Add the upstream first, or fix the typo in the package entry.
+
+```text
+[x] upstream 'myupstream' canonical name has changed: declared 'old-owner/Repo' but GitHub returns 'new-owner/Repo' (possible repo rename or takeover)
+```
+
+The repo at the configured owner/repo path no longer matches what your lockfile recorded the last time the upstream was refreshed. Investigate before advancing the pin -- this is the same signal package-confusion attacks produce.
+
+```text
+[x] upstream 'myupstream' resolves to ref 'main' which is a moving branch; pass --allow-head to opt in or pin --ref to a SHA / tag
+```
+
+You attempted to register or build an upstream against a branch without explicit `--allow-head`. Either pin to an immutable SHA / tag (recommended) or opt in to HEAD-tracking with `--allow-head` and accept the per-build warning.
+
+## Trust model
+
+Upstreams are a **curated pass-through**, not a binary mirror.
+
+| Concern | v1 status |
+|---|---|
+| Allow-list governance (curator picks which plugins are exposed) | Yes |
+| Build-time commit pinning (manifest SHA + plugin SHA in lockfile) | Yes |
+| Reproducible curator builds (rebuild from lock = byte-identical output) | Yes |
+| Defence against upstream repo rename / takeover | Yes (canonical-owner check) |
+| Consumer-side artifact custody | No -- consumer clones from upstream git host at install |
+| Resilience to upstream takedown / force-push | No -- consumer install fails if upstream rewrites history |
+
+Air-gapped re-hosting is **out of scope for v1** and is tracked separately as a future `distribution: rehost` mode.
+
+## What is NOT supported in v1
+
+- **Re-hosting / artifact custody.** Consumer installs always fetch plugin content from the upstream git host. APM never proxies or stores that content.
+- **Transitive upstreams.** If marketplace B upstreams marketplace C, you cannot upstream B and inherit C transitively.
+- **Cross-host upstreams.** The upstream and its referenced plugins must live on the same git host family (for example, both on github.com).
+- **Search.** `apm marketplace upstream list` covers discovery in v1.
+
+:::note[Planned]
+`apm doctor` will gain upstream health checks in a future release: reachability testing, pinned-vs-HEAD drift reporting, and manifest-SHA round-trip verification against the lockfile.
+:::
+
+## Related
+
+- [Authoring a marketplace](../marketplace-authoring/) -- start here if your `apm.yml` has no `marketplace:` block yet.
+- [Marketplaces (consumer)](../marketplaces/) -- registering and consuming external marketplaces from a project.
+- [Security and trust](../../enterprise/security/) -- the full APM security model.
+

docs/src/content/docs/reference/cli-commands.md

@@ -1489,18 +1489,22 @@ apm marketplace package add SOURCE [OPTIONS]
 ```
 
 **Arguments:**
-- `SOURCE` - GitHub `owner/repo` reference
+- `SOURCE` - GitHub `owner/repo` reference (omit when `--upstream` is used)
 
 **Options:**
 - `--version TEXT` - Semver range constraint (e.g. `">=1.0.0"`)
 - `--ref TEXT` - Pin to a git ref (SHA, tag, or HEAD). Mutable refs are auto-resolved to SHA
 - `-d`, `--description TEXT` - Short description for the entry
 - `-s`, `--subdir TEXT` - Subdirectory inside source repo
+- `--upstream ALIAS` - Reference a registered upstream marketplace (mutually exclusive with `SOURCE`/`--subdir`). See [Marketplace Upstreams](../../guides/marketplace-upstreams/).
+- `--plugin NAME` - Plugin name in the upstream marketplace (used with `--upstream`; defaults to the entry's name)
+- `--name NAME` - Override the displayed package name in your marketplace
+- `--allow-head` - Permit upstream-sourced entries that resolve to a moving branch HEAD (warned)
 - `--include-prerelease` - Include pre-release versions
 - `--no-verify` - Skip remote repository verification
 - `--verbose` - Enable verbose output
 
-`--version` and `--ref` are mutually exclusive. When neither is provided, the current `HEAD` SHA is pinned automatically.
+`--version` and `--ref` are mutually exclusive. `SOURCE` and `--upstream` are mutually exclusive. When neither `--version` nor `--ref` is provided, the current `HEAD` SHA is pinned automatically.
 
 **Examples:**
 ```bash
@@ -1516,6 +1520,45 @@ apm marketplace package add acme/code-review
 # Add with description and skip verification (requires explicit --ref SHA)
 apm marketplace package add acme/code-review --ref abc123...40chars \
   --description "Code review skill" --no-verify
+
+# Add a package sourced from a registered upstream marketplace
+apm marketplace package add --upstream gitnexus --plugin gitnexus \
+  --name acme-gitnexus
+```
+
+#### `apm marketplace upstream` - Manage upstream marketplaces
+
+Curate plugins from external Claude Code marketplaces under your own
+allow-list. Subcommands edit `apm.yml -> marketplace.upstreams`. See
+the [Marketplace Upstreams guide](../../guides/marketplace-upstreams/)
+for the trust model and end-to-end example.
+
+```bash
+apm marketplace upstream add OWNER/REPO --alias ALIAS [OPTIONS]
+apm marketplace upstream list [--verbose]
+apm marketplace upstream remove ALIAS [--yes]
+```
+
+**`add` options:**
+- `--alias TEXT` (required) - Local alias used to reference the upstream from `packages[]`
+- `--ref TEXT` - Pin the upstream `marketplace.json` to a SHA or tag (mutable refs auto-resolve to SHA)
+- `--branch TEXT` - Track a mutable branch (requires `--allow-head`)
+- `--path TEXT` - Path to the upstream `marketplace.json` (default: `.claude-plugin/marketplace.json`)
+- `--host TEXT` - Git host (default: `github.com`; supports GHE/GHES)
+- `--allow-head` - Permit branch HEAD tracking (warned at build time)
+
+**Examples:**
+```bash
+# Register a public upstream pinned to a SHA
+apm marketplace upstream add abhigyanpatwari/GitNexus \
+  --alias gitnexus \
+  --ref abc1234...40chars
+
+# List registered upstreams
+apm marketplace upstream list
+
+# Remove an upstream (rejects if any package still references it)
+apm marketplace upstream remove gitnexus --yes
 ```
 
 #### `apm marketplace package set` - Update a package entry

packages/apm-guide/.apm/skills/apm-usage/commands.md

@@ -83,8 +83,12 @@
 | `apm marketplace doctor` | Diagnose git, network, auth, and marketplace config readiness | `-v` |
 | `apm marketplace publish` | Open PRs on consumer repos from `consumer-targets.yml` | `--targets PATH`, `--dry-run`, `--no-pr`, `--draft`, `--allow-downgrade`, `--allow-ref-change`, `--parallel N`, `-y` |
 | `apm marketplace package add <source>` | Add a plugin entry to `marketplace.plugins` (source accepts `owner/repo` or `./path`) | `--name`, `--version`, `--ref` (mutable refs auto-resolved to SHA), `-d`/`--description`, `-s`/`--subdir`, `--tag-pattern`, `--tags`, `--include-prerelease`, `--no-verify` |
+| `apm marketplace package add --upstream <alias> --plugin <name>` | Expose an upstream marketplace's plugin (mutually exclusive with positional `<source>`) | `--name`, `--version`, `--ref`, `--tag-pattern`, `--tags`, `--include-prerelease`, `--allow-head` |
 | `apm marketplace package set <name>` | Update fields on an existing plugin entry | `--version`, `--ref` (mutable refs auto-resolved to SHA), `--description`, `--subdir`, `--tag-pattern`, `--tags`, `--include-prerelease` |
 | `apm marketplace package remove <name>` | Remove a plugin entry from `marketplace.plugins` | `--yes` |
+| `apm marketplace upstream add <repo> --alias <alias>` | Register an upstream marketplace (allow-list governance, immutable commit pinning, no re-hosting) | `--ref` (immutable; mutable refs auto-resolved), `--branch` (requires `--allow-head`), `--path`, `--host`, `--allow-head`, `--no-verify` |
+| `apm marketplace upstream list` | List registered upstream marketplaces | `-v` |
+| `apm marketplace upstream remove <alias>` | Remove an upstream (rejects when any package still references the alias) | |
 
 To build the marketplace, run `apm pack` (it reads `apm.yml` and writes `.claude-plugin/marketplace.json` whenever the `marketplace:` block is present). `apm init --marketplace` is the equivalent shortcut at project-creation time -- it seeds a fresh `apm.yml` with the `marketplace:` block already in place.
 

packages/apm-guide/.apm/skills/apm-usage/package-authoring.md

@@ -284,8 +284,41 @@ marketplace:
       description: Plugin shipped alongside this repo
       source: ./plugins/local-tool       # local path (no remote fetch)
       version: 0.1.0
+
+  upstreams:                              # optional; expose external-marketplace plugins
+    - alias: gitnexus                     # local handle
+      repo: abhigyanpatwari/GitNexus      # upstream marketplace repo
+      ref: <40-char-sha>                  # required for reproducibility
+      # branch: main                      # alternative to ref; requires allow_head: true
+      # path: .claude-plugin/marketplace.json   # default
+      # host: github.com                  # default
+      # allow_head: false                 # default; opt-in to mutable refs
 ```
 
+Add `packages[]` entries that reference `upstreams[].alias` to expose
+specific plugins from an upstream:
+
+```yaml
+marketplace:
+  upstreams:
+    - alias: gitnexus
+      repo: abhigyanpatwari/GitNexus
+      ref: <40-char-sha>
+  packages:
+    - name: acme-gitnexus     # display name in your marketplace
+      upstream: gitnexus      # references upstreams[].alias
+      plugin: gitnexus        # name in the upstream marketplace
+      version: ">=1.0.0"      # optional curator override
+```
+
+`source` and `upstream` are mutually exclusive on a single entry.
+Upstreams are a curated allow-list with build-time commit pinning;
+APM does NOT re-host upstream content -- consumer installs always
+fetch plugin source from the upstream git host. Provenance (manifest
+SHA, resolved plugin SHA, canonical owner) is recorded only in
+`apm.lock.yaml`; the emitted `marketplace.json` stays
+Anthropic-conformant.
+
 Schema rules:
 - `owner.name` is required. `name`, `description`, `version` are
   optional inside the block (inherited from apm.yml top level).

scripts/test-integration.sh

@@ -552,6 +552,16 @@ run_e2e_tests() {
         exit 1
     fi
 
+    log_info "Running marketplace upstream build integration tests..."
+    echo "Command: pytest tests/integration/marketplace/test_upstream_build_integration.py -v --tb=short"
+
+    if pytest tests/integration/marketplace/test_upstream_build_integration.py -v --tb=short; then
+        log_success "Marketplace upstream build integration tests passed!"
+    else
+        log_error "Marketplace upstream build integration tests failed!"
+        exit 1
+    fi
+
     log_success "All integration test suites completed successfully!"
 
 

src/apm_cli/commands/marketplace/__init__.py

@@ -87,6 +87,7 @@ class MarketplaceGroup(click.Group):
         "doctor",
         "publish",
         "package",
+        "upstream",
         "migrate",
     ]
 
@@ -206,8 +207,10 @@ def marketplace(ctx):
 
 
 from .plugin import package  # noqa: E402
+from .upstream import upstream  # noqa: E402
 
 marketplace.add_command(package)
+marketplace.add_command(upstream)
 
 
 def _check_gitignore_for_marketplace_json(logger):

src/apm_cli/commands/marketplace/plugin/__init__.py

@@ -2,7 +2,6 @@
 
 from __future__ import annotations
 
-import re
 import sys
 from pathlib import Path
 
@@ -15,10 +14,9 @@
     MarketplaceYmlError,  # noqa: F401
     OfflineMissError,
 )
+from ....marketplace.ref_resolver import FULL_SHA_RE as _SHA_RE
 from ..._helpers import _is_interactive  # noqa: F401
 
-_SHA_RE = re.compile(r"^[0-9a-f]{40}$")
-
 
 def _yml_path() -> Path:
     """Return the active marketplace authoring config path in CWD."""