From 3b3a44e9bc2812eea41bf79da2436d174e04076b Mon Sep 17 00:00:00 2001 From: Daniele Martinoli Date: Mon, 1 Sep 2025 12:54:04 +0200 Subject: [PATCH] Add Kubernetes Registry proposal with catalog system design Signed-off-by: Daniele Martinoli --- docs/proposals/kubernetes-registry.md | 210 +++++++++++++++ .../kubernetes-registry/catalog-design.md | 200 +++++++++++++++ .../kubernetes-registry/crd-specifications.md | 114 +++++++++ .../implementation-plan.md | 122 +++++++++ .../kubernetes-registry/usage-examples.md | 241 ++++++++++++++++++ docs/proposals/mcp-server-template-system.md | 235 +++++++++++++++++ 6 files changed, 1122 insertions(+) create mode 100644 docs/proposals/kubernetes-registry.md create mode 100644 docs/proposals/kubernetes-registry/catalog-design.md create mode 100644 docs/proposals/kubernetes-registry/crd-specifications.md create mode 100644 docs/proposals/kubernetes-registry/implementation-plan.md create mode 100644 docs/proposals/kubernetes-registry/usage-examples.md create mode 100644 docs/proposals/mcp-server-template-system.md diff --git a/docs/proposals/kubernetes-registry.md b/docs/proposals/kubernetes-registry.md new file mode 100644 index 000000000..2dcba7244 --- /dev/null +++ b/docs/proposals/kubernetes-registry.md @@ -0,0 +1,210 @@ +# Kubernetes Registry Implementation for ToolHive Operator + +## Problem Statement + +The ToolHive operator currently supports managing individual MCP servers through `MCPServer` CRDs, but lacks a centralized registry mechanism within Kubernetes. This creates challenges in discoverability, catalog management, upstream compatibility, and operational complexity. + +## Goals + +- **Native Kubernetes Registry**: Implement registry functionality using Custom Resource Definitions +- **Upstream Format Support**: Leverage existing upstream conversion capabilities for ecosystem compatibility +- **Multi-Registry Support**: Support both local registry entries and external registry synchronization. +- **Registry Hierarchy**: Support the multi-registry hierarchy defined in the upstream model +- **Application Integration**: Provide REST API for programmatic access to registry data +- **GitOps Compatibility**: Enable declarative registry management through CRD-based operations + +## Architecture Overview + +The Kubernetes registry implementation extends the operator with the `MCPRegistry` CRD and supporting controllers that work with the existing MCPServer CRD to provide a complete registry-to-deployment workflow. + +## CRD Design Overview + +### MCPRegistry CRD + +The `MCPRegistry` CRD represents a registry source and synchronization configuration with these key components: + +- **Source Configuration**: Support for ConfigMap, URL, Git, and Registry API sources +- **Format Specification**: Handle both ToolHive and upstream registry formats +- **Sync Policy**: Automatic and manual synchronization with configurable intervals +- **Filtering**: Include/exclude servers based on names, tags, tiers, and transports + +### Job CRDs (Phase 3) + +Declarative operation CRDs for GitOps compatibility: +- `MCPRegistryImportJob`: Declarative import operations +- `MCPRegistryExportJob`: Declarative export operations +- `MCPRegistrySyncJob`: Declarative synchronization operations + +**Detailed specifications**: See [kubernetes-registry/crd-specifications.md](kubernetes-registry/crd-specifications.md) + +## Key Features and Capabilities + +### 1. Registry Management + +The ToolHive operator provides comprehensive registry management through specialized components: + +#### Registry Controller +- **Synchronization**: Automatic and manual synchronization with external registry sources +- **Format Conversion**: Bidirectional conversion between ToolHive and upstream registry formats +- **Filtering**: Include/exclude servers based on configurable criteria (names, tags, tiers, transports) +- **Status Tracking**: Monitor sync status, error conditions, and statistics +- **Server Labeling**: Automatically apply registry relationship labels to discovered servers + +#### Registry API Service +- **REST API**: HTTP endpoints for programmatic registry and server discovery +- **Authentication**: Integration with Kubernetes RBAC and service account tokens +- **Filtering**: Query servers by registry, category, transport type, and custom labels +- **Format Support**: Return data in both ToolHive and upstream registry formats + +### 2. Registry Sources + +The implementation supports multiple registry source types, all with both ToolHive and upstream formats. +All sources support configurable synchronization policies including automatic sync intervals, retry behavior, and update strategies. + +Registry sources can be organized in hierarchies as defined in the [MCP Registry Ecosystem Diagram](https://github.com/modelcontextprotocol/registry/blob/main/docs/ecosystem-diagram.excalidraw.svg), enabling upstream registries to aggregate from multiple sources. + +This aggregation approach, combined with maintaining the ToolHive registry schema, addresses provenance data handling by extracting it from upstream registry extensions during format conversion. + +#### ConfigMap Source +- Store registry data directly in Kubernetes ConfigMaps +- Ideal for small, manually managed registries +- Immediate updates when ConfigMap changes + +#### URL Source +- Fetch registry data from HTTP/HTTPS endpoints +- Support for authentication via Secret references +- Custom headers for API integration + +#### Git Source +- Clone registry data from Git repositories +- Branch and path specification +- Authentication via SSH keys or tokens +- Version tracking and change detection + +#### Registry Source +- Reference another registry's REST API endpoint as a data source +- Enables registry hierarchies and aggregation patterns across clusters +- Supports filtering and transformation of upstream registry data +- Works with any registry implementation that exposes the standard API +- Useful for creating curated subsets or company-specific views of upstream registries + +### 3. Server-Registry Relationships + +#### Automatic Labeling +When deployed servers are created from registries, the controller automatically applies standardized labels during resource creation: + +```yaml +labels: + toolhive.stacklok.io/registry-name: upstream-community + toolhive.stacklok.io/registry-namespace: toolhive-system + toolhive.stacklok.io/server-name: filesystem-server + toolhive.stacklok.io/tier: Official + toolhive.stacklok.io/category: filesystem +``` + +These labels enable filtering, grouping, and querying servers by their registry source. + +#### Pre-deployed Server Association +Existing MCPServer resources can be associated with registries by applying the standard labels, enabling unified management across manually deployed and registry-synchronized servers. + +## Quick Start Example + +```yaml +apiVersion: toolhive.stacklok.io/v1alpha1 +kind: MCPRegistry +metadata: + name: upstream-community + namespace: toolhive-system +spec: + displayName: "MCP Community Registry" + format: upstream + source: + type: url + url: + url: "https://registry.modelcontextprotocol.io/servers.json" + syncPolicy: + enabled: true + interval: "1h" +``` + +**Comprehensive examples**: See [kubernetes-registry/usage-examples.md](kubernetes-registry/usage-examples.md) + +## Implementation Overview + +The implementation follows a phased approach: + +1. **Phase 1**: Core Registry CRD and basic synchronization +2. **Phase 2**: External sources, REST API for applications +3. **Phase 3**: CRD-based operations, automatic labeling +4. **Phase 4**: Production features and filtering +5. **Phase 5**: Advanced integration (optional) + +**Detailed implementation plan**: See [kubernetes-registry/implementation-plan.md](kubernetes-registry/implementation-plan.md) + +## CLI Integration + +New registry management commands: +- `thv registry list/add/sync/remove` - Registry lifecycle management +- `thv registry import/export` - Data migration operations +- `thv search/show` - Enhanced server discovery across registries + +**Complete CLI reference**: See [kubernetes-registry/usage-examples.md](kubernetes-registry/usage-examples.md) + +## Security and Operations + +### Security Model +- **RBAC Integration**: Granular permissions for registry operations +- **Source Validation**: URL restrictions and content validation +- **Authentication**: Secure handling of external source credentials +- **Audit Logging**: Comprehensive operation tracking + +### Success Metrics +- **Adoption**: Registry resource creation and server association rates +- **Performance**: <30s sync time, >99% success rate, <100MB memory usage +- **Usability**: Reduced manual configuration complexity +- **Ecosystem**: Upstream registry coverage and format conversion accuracy + +**Complete details**: See [kubernetes-registry/implementation-plan.md](kubernetes-registry/implementation-plan.md) + +## Future Enhancements + +1. **Catalog System** (see [kubernetes-registry/catalog-design.md](kubernetes-registry/catalog-design.md)) + - MCPCatalog CRD for curated server collections + - Approval workflows and validation pipelines + - OCI artifact distribution for catalog sharing + - Role-based catalog access and governance + +2. **Advanced Registry Features** + - Registry federation and cross-cluster synchronization + - Webhook-based real-time registry updates + - Registry analytics and usage metrics + - Content verification and signature validation + +3. **Template System** (see [../mcp-server-template-system.md](../mcp-server-template-system.md)) + - Comprehensive template parameter system + - Template versioning and inheritance + - Integration with Helm and Kustomize + - Interactive template wizards and validation + +4. **Integration Expansions** + - GitOps workflow integration with ArgoCD/Flux + - CI/CD pipeline integration + - Service mesh integration for advanced networking + - Multi-cluster registry synchronization + +5. **Community Features** + - Community ratings and reviews for registry entries + - Automated server discovery from popular repositories + - Registry contribution workflows and governance + +## Conclusion + +The Kubernetes Registry implementation provides a cloud-native approach to MCP server management that: + +- **Leverages Kubernetes APIs** for native resource management and RBAC +- **Integrates with existing tooling** through standard kubectl and custom CLI commands +- **Supports ecosystem growth** through upstream format compatibility and conversion +- **Enables GitOps workflows** through declarative resource definitions +- **Scales operationally** with automated synchronization and registry-based deployment + +This implementation transforms ToolHive into a comprehensive Kubernetes-native platform for MCP server lifecycle management, maintaining backward compatibility while enabling ecosystem integration through upstream format support. \ No newline at end of file diff --git a/docs/proposals/kubernetes-registry/catalog-design.md b/docs/proposals/kubernetes-registry/catalog-design.md new file mode 100644 index 000000000..43b417365 --- /dev/null +++ b/docs/proposals/kubernetes-registry/catalog-design.md @@ -0,0 +1,200 @@ +# Catalog System Design for Kubernetes Registry + +## Overview + +A **Catalog** is essentially a **special registry with trusted servers** that have undergone validation and approval processes. This document explores implementing catalogs as enhanced MCPRegistry resources, though a dedicated MCPCatalog CRD may be more appropriate depending on the complexity of approval workflows and governance requirements. + +## Catalog as Special Registry + +Catalogs follow the same promotion workflow: +``` +Raw Registry → Staging Registry → Catalog (Trusted Registry) → Production Deployment +``` + +### Trust Levels +- **Raw Registry**: All servers from upstream sources (trust-level: raw) +- **Staging Registry**: Filtered and tested subset (trust-level: tested) +- **Catalog Registry**: Validated and approved servers (trust-level: trusted) + +### Unified Model Benefits +- **Consistent API**: Same MCPRegistry CRD for all trust levels +- **Natural Promotion**: Clear path from raw → tested → trusted +- **Simplified Tooling**: No separate catalog vs registry commands +- **Environment Mapping**: Dev uses raw/tested, Production uses trusted catalogs + +## MCPRegistry with Catalog Features + +### Enhanced MCPRegistry for Catalogs + +```yaml +apiVersion: toolhive.stacklok.io/v1alpha1 +kind: MCPRegistry +metadata: + name: production-catalog + namespace: toolhive-system +spec: + displayName: "Production Approved MCP Servers" + description: "Trusted servers for production deployment" + format: toolhive + + # Trust level determines validation requirements + trustLevel: trusted # raw, tested, trusted + + # Source from staging registry + source: + type: registry + registry: + url: "http://staging-registry-api:8080/api/v1/servers" + + # Enhanced validation for trusted registries + validationPolicy: + approvalRequired: true + allowedTiers: ["Official", "Verified"] + blockedPatterns: ["*-experimental", "*-alpha"] + securityScanRequired: true + + # Promotion criteria for catalog inclusion + promotionCriteria: + successfulDeployments: 10 + approvedBy: ["security-team", "platform-team"] + testValidationPassed: true + +status: + # Standard registry status plus catalog-specific fields + catalogState: + approvedServers: 12 + pendingPromotion: 3 + rejectedServers: 1 + lastValidation: "2024-01-15T10:30:00Z" +``` + +## Key Catalog Features + +### 1. **Trust-Based Validation** +- **Automatic Promotion**: Servers move from raw → tested → trusted based on criteria +- **Policy Enforcement**: Only approved servers reach trusted catalogs +- **Security Integration**: Vulnerability scanning and compliance checks +- **Approval Workflows**: Human approval gates for production promotion + +### 2. **Unified Deployment Interface** +```bash +# Same command, different trust levels +thv install postgres-server --registry upstream-raw # Development +thv install postgres-server --registry staging-tested # QA/Testing +thv install postgres-server --registry production-catalog # Production (trusted) +``` + +### 3. **OCI Distribution** +- **Registry Packaging**: Export trusted registries as OCI artifacts +- **Version Control**: Semantic versioning of catalog releases +- **Signature Verification**: Signed artifacts for supply chain security +- **Cross-Cluster Sharing**: Distribute catalogs between environments + +## Registry Promotion Workflow + +### Data Flow +``` +External Sources → Raw Registry → Staging Registry → Trusted Registry (Catalog) → Production +``` + +### Promotion Pipeline +- **Source Registration**: Raw registries sync from external sources +- **Testing Phase**: Staging registries receive filtered subset for validation +- **Approval Gate**: Manual/automated promotion to trusted catalogs +- **Production Deployment**: Only trusted catalog servers reach production + +### Validation Pipeline Requirements + +The trust-based promotion requires a **validation pipeline** that needs deeper design: + +#### Essential Validations +- **Security Scanning**: Vulnerability and compliance checks +- **Functional Testing**: MCP protocol compatibility and performance +- **Approval Workflows**: Human gates for production promotion +- **Audit Trails**: Complete promotion history and compliance tracking + +*Note: Comprehensive validation pipeline design is outside scope of this proposal.* + +## Review and Approval Process + +### Approval Workflow Requirements + +The transition from staging to production catalogs requires a **human review and approval process**: + +#### **Review Queue Management** +- **Pending Servers**: Track servers awaiting approval with metadata (test results, deployment history) +- **Reviewer Assignment**: Route servers to appropriate teams based on category/risk level +- **Review Status**: Track review progress (submitted, under-review, approved, rejected) +- **Approval History**: Maintain audit trail of who approved what and when + +#### **Integration Points** +- **External Systems**: JIRA, ServiceNow, GitHub PR workflows for approval tracking +- **Notification Systems**: Alert reviewers when servers are ready for approval +- **CLI Tools**: `thv review list/approve/reject` commands for reviewer workflow +- **Dashboard Integration**: Web UI for reviewing server details and approval status + +#### **Approval Criteria** +- **Automated Gates**: Security scans, performance benchmarks, successful deployment count +- **Manual Assessment**: Code quality review, documentation completeness, business impact +- **Multi-Team Approval**: Require sign-off from security, platform, and business teams +- **Escalation Rules**: Auto-approve after timeout or escalate to management + +### Server Approval Status Tracking + +Each server in the promotion pipeline needs detailed status tracking: + +```yaml +serverApprovalStatus: + - serverName: "postgres-server" + version: "1.3.0" + currentStatus: "under-review" # pending, under-review, approved, rejected + submittedAt: "2024-01-15T10:00:00Z" + assignedReviewers: ["security-team", "platform-team"] + approvalProgress: + securityTeam: "approved" + platformTeam: "pending" + automatedChecks: + securityScan: "passed" + performanceTest: "passed" + deploymentCount: 15 + reviewComments: + - author: "alice@security" + timestamp: "2024-01-16T09:00:00Z" + comment: "Security review completed - approved" + externalTickets: + - system: "JIRA" + ticketId: "SEC-12345" + status: "Approved" +``` + +This approval tracking ensures **governance transparency** and **audit compliance** while enabling automated promotion when all criteria are met. + +## Initial Implementation + +### Simple Approach: Pre-approved Registry +Instead of complex validation pipelines, start with **manual curation**: + +```yaml +apiVersion: toolhive.stacklok.io/v1alpha1 +kind: MCPRegistry +metadata: + name: production-catalog +spec: + displayName: "Production Approved Servers" + trustLevel: trusted + source: + type: url + url: + url: "https://catalog.company.com/approved-servers.json" + # JSON contains only manually approved servers +``` + +### Migration Path +1. **Phase 1**: Manual JSON curation (immediate) +2. **Phase 2**: Add trust levels to MCPRegistry CRD +3. **Phase 3**: Build automated validation pipeline +4. **Phase 4**: Full promotion workflow automation + +## Conclusion + +Catalogs are **trusted registries** that provide governance and curation for production environments. By treating catalogs as enhanced MCPRegistry resources with trust levels, the system maintains consistency while enabling sophisticated validation and promotion workflows. This unified approach simplifies tooling and provides a natural path from development (raw registries) to production (trusted catalogs). \ No newline at end of file diff --git a/docs/proposals/kubernetes-registry/crd-specifications.md b/docs/proposals/kubernetes-registry/crd-specifications.md new file mode 100644 index 000000000..debb8a3e9 --- /dev/null +++ b/docs/proposals/kubernetes-registry/crd-specifications.md @@ -0,0 +1,114 @@ +# CRD Specifications for Kubernetes Registry + +This document provides detailed field reference documentation for the Kubernetes Registry CRDs. + +## MCPRegistry CRD + +The `MCPRegistry` CRD represents a registry source and its synchronization configuration. + +### MCPRegistry Spec + +| Field | Description | Required | Default | Values | +|-------|-------------|----------|---------|---------| +| `displayName` | Human-readable name for the registry | No | - | Any string | +| `description` | Additional context about the registry's purpose | No | - | Any string | +| `format` | Format of the registry data | No | - | `toolhive`, `upstream` | +| `source` | Defines where to fetch registry data from | Yes | - | See Source Configuration | +| `syncPolicy` | How and when to synchronize with the source | No | - | See Sync Policy | +| `filter` | Which servers to include/exclude from the registry | No | - | See Filter Configuration | + +### Source Configuration + +| Field | Description | Required | Default | Values | +|-------|-------------|----------|---------|---------| +| `type` | Source type | Yes | - | `configmap`, `url`, `git`, `registry` | +| `configMap` | ConfigMap source configuration | No | - | See ConfigMap Source | +| `url` | URL source configuration | No | - | See URL Source | +| `git` | Git source configuration | No | - | See Git Source | +| `registry` | Registry source configuration | No | - | See Registry Source | + +### ConfigMap Source + +| Field | Description | Required | Default | Values | +|-------|-------------|----------|---------|---------| +| `name` | Name of the ConfigMap | Yes | - | Any valid ConfigMap name | +| `key` | Key containing registry data | No | `registry.json` | Any key in ConfigMap | + +### URL Source + +| Field | Description | Required | Default | Values | +|-------|-------------|----------|---------|---------| +| `url` | HTTP(S) URL to fetch registry data | Yes | - | Valid HTTP/HTTPS URL | +| `headers` | Optional HTTP headers | No | - | Key-value map | +| `authSecret` | Secret containing auth credentials | No | - | See Auth Secret Reference | + +### Git Source + +| Field | Description | Required | Default | Values | +|-------|-------------|----------|---------|---------| +| `repository` | Git repository URL | Yes | - | Valid Git repository URL | +| `branch` | Git branch to use | No | `main` | Any valid branch name | +| `path` | Path to registry file in repo | No | `registry.json` | Any file path | +| `authSecret` | Secret containing Git auth credentials | No | - | See Auth Secret Reference | + +### Registry Source + +| Field | Description | Required | Default | Values | +|-------|-------------|----------|---------|---------| +| `url` | Registry API endpoint URL | Yes | - | Valid HTTP/HTTPS URL to registry API | +| `headers` | Optional HTTP headers | No | - | Key-value map | +| `authSecret` | Secret containing auth credentials | No | - | See Auth Secret Reference | + +### Sync Policy + +| Field | Description | Required | Default | Values | +|-------|-------------|----------|---------|---------| +| `enabled` | Enable automatic synchronization | No | `true` | `true`, `false` | +| `interval` | How often to synchronize | No | `1h` | Duration (e.g., `30m`, `2h`) | +| `onUpdate` | Behavior when registry content changes | No | `update` | `create`, `update`, `recreate` | +| `retry` | Retry behavior for failed syncs | No | - | See Retry Policy | + +### Retry Policy + +| Field | Description | Required | Default | Values | +|-------|-------------|----------|---------|---------| +| `maxRetries` | Maximum number of retry attempts | No | 3 | 0-∞ | +| `backoffInterval` | Initial backoff interval | No | `30s` | Duration (e.g., `10s`, `1m`) | + +### Filter Configuration + +| Field | Description | Required | Default | Values | +|-------|-------------|----------|---------|---------| +| `include` | Patterns for servers to include | No | - | See Filter Criteria | +| `exclude` | Patterns for servers to exclude | No | - | See Filter Criteria | + +### Filter Criteria + +| Field | Description | Required | Default | Values | +|-------|-------------|----------|---------|---------| +| `names` | Server name patterns (supports wildcards) | No | - | String array | +| `tags` | Tag patterns to match | No | - | String array | +| `tiers` | Tier levels to match | No | - | String array (e.g., `Official`, `Community`) | +| `transports` | Transport types to match | No | - | String array (e.g., `stdio`, `sse`) | + +## Job CRDs + +### MCPRegistryImportJob + +Declarative import operations for registry data. + +**Status Field**: Includes job lifecycle status (`idle`, `scheduled`, `running`, `completed`, `failed`) and operation results. For large result sets (e.g., per-server import status), detailed output can be stored in a dedicated ConfigMap referenced from the status. + +### MCPRegistryExportJob + +Declarative export operations for registry data. + +**Status Field**: Includes job lifecycle status and export results. Large exported data or detailed operation logs can be stored in a ConfigMap for efficient access and storage management. + +### MCPRegistrySyncJob + +Declarative synchronization operations for registry sources. + +**Status Field**: Includes job lifecycle status and sync results. Per-registry or per-server sync details can be stored in ConfigMaps when result data exceeds reasonable status field limits. + +*Note: Detailed specifications for job CRDs will be defined in Phase 3 implementation.* \ No newline at end of file diff --git a/docs/proposals/kubernetes-registry/implementation-plan.md b/docs/proposals/kubernetes-registry/implementation-plan.md new file mode 100644 index 000000000..c5d5ae4a5 --- /dev/null +++ b/docs/proposals/kubernetes-registry/implementation-plan.md @@ -0,0 +1,122 @@ +# Implementation Plan for Kubernetes Registry + +This document outlines the detailed implementation phases and technical considerations. + +## Implementation Phases + +### Phase 1: Core Registry Implementation +- Define `MCPRegistry` CRD with comprehensive field validation +- Implement basic Registry Controller with ConfigMap source support +- Add format conversion for upstream registry compatibility +- Implement CLI commands: `thv registry list`, `thv registry sync`, `thv registry add` +- Template system evaluation per `docs/proposals/mcp-server-template-system.md` + +### Phase 2: External Sources and Advanced Features +- Add URL and Git source support to Registry Controller +- Integrate existing upstream conversion logic from `pkg/registry/upstream_conversion.go` +- Add authentication support for external sources +- Implement periodic synchronization with configurable intervals +- Add REST API service for application registry access + +### Phase 3: CRD-Based Operations and Job Management +- Add dedicated CRDs: `MCPRegistryImportJob`, `MCPRegistryExportJob`, `MCPRegistrySyncJob` +- Implement job controllers with retry logic and status tracking +- Add server-registry label relationships and automatic labeling +- GitOps-compatible declarative operations + +### Phase 4: Production Features and Filtering +- Add registry filtering support +- Implement comprehensive monitoring and observability +- Add security scanning and validation +- Performance optimization and caching + +### Phase 5: Advanced Integration (Optional) +- Template system implementation (if customer demand justifies complexity) +- Multi-cluster registry synchronization +- Advanced filtering and analytics + +## Technical Architecture + +### Controller Design +- **Registry Controller**: Manages MCPRegistry lifecycle and synchronization +- **Job Controllers**: Handle declarative import/export/sync operations +- **API Service**: REST endpoints for application integration + +### Data Flow +1. Registry Controller syncs external sources +2. Format conversion transforms upstream data to ToolHive format +3. MCPServer resources created/updated with automatic labeling +4. REST API serves processed registry data to applications +5. Job CRDs provide declarative operations for GitOps workflows + +### Security Considerations +- RBAC integration for CLI and API access +- Secret management for external source authentication +- Content validation and signature verification +- Network policies for registry sync operations + +## Testing Strategy + +### Unit Tests +- CRD validation and defaulting logic +- Format conversion accuracy and round-trip testing +- Registry filtering logic +- REST API endpoint validation + +### Integration Tests +- End-to-end registry synchronization workflows +- External source authentication and error handling +- Controller reconciliation and status updates +- Label-based filtering and querying + +### E2E Tests +- Complete registry-to-deployment workflows +- Multi-registry scenarios with label-based organization +- CLI integration and user workflows +- REST API integration with sample applications + +## Migration Strategy + +### Backward Compatibility +- Existing MCPServer resources remain unchanged +- CLI registry commands continue to work with file-based registries +- Gradual migration path from file-based to Kubernetes-native registries + +### Migration Tools +```bash +# Convert existing registry files to Kubernetes resources +thv registry migrate registry.json --output k8s-registry.yaml + +# Bulk import existing servers with registry association +thv registry import --file registry.json --registry local-servers + +# Export Kubernetes registry back to file format +thv registry export upstream-community --format toolhive --output exported.json +``` + +### Hybrid Operation +- Support for both file-based and Kubernetes registries +- Automatic detection and prioritization +- Cross-format compatibility and conversion + +## Success Metrics + +### Adoption Metrics +- Number of MCPRegistry resources created and actively synced +- Number of unique upstream registries integrated +- Number of MCPServer resources created with registry annotations + +### Performance Metrics +- Registry synchronization time and success rate (target: <30s sync time, >99% success rate) +- Resource consumption of registry controllers (target: <100MB memory, <0.1 CPU cores) +- Time from registry update to server availability + +### Usability Metrics +- Reduction in manual server configuration complexity +- Time to discover and deploy servers from external registries +- CLI command usage patterns for registry operations + +### Ecosystem Integration +- Coverage of upstream MCP servers through registry synchronization +- Successful format conversion rate between upstream and ToolHive formats +- Community adoption of Kubernetes-native registry workflows \ No newline at end of file diff --git a/docs/proposals/kubernetes-registry/usage-examples.md b/docs/proposals/kubernetes-registry/usage-examples.md new file mode 100644 index 000000000..1f9ab3280 --- /dev/null +++ b/docs/proposals/kubernetes-registry/usage-examples.md @@ -0,0 +1,241 @@ +# Usage Examples for Kubernetes Registry + +This document provides comprehensive examples of using the Kubernetes Registry system. + +## Registry Configuration Examples + +### Creating a Registry from URL Source + +```yaml +apiVersion: toolhive.stacklok.io/v1alpha1 +kind: MCPRegistry +metadata: + name: upstream-community + namespace: toolhive-system +spec: + displayName: "MCP Community Registry" + description: "Official community registry for MCP servers" + format: upstream + source: + type: url + url: + url: "https://registry.modelcontextprotocol.io/servers.json" + syncPolicy: + enabled: true + interval: "1h" + onUpdate: update + filter: + include: + tiers: ["Official", "Community"] + exclude: + names: ["*-experimental-*"] +``` + +### Creating a Registry from Git Source + +```yaml +apiVersion: toolhive.stacklok.io/v1alpha1 +kind: MCPRegistry +metadata: + name: company-internal + namespace: production +spec: + displayName: "Company Internal MCP Registry" + description: "Internal registry for company-specific MCP servers" + format: toolhive + source: + type: git + git: + repository: "https://github.com/company/mcp-registry.git" + branch: "main" + path: "registry/toolhive-format.json" + authSecret: + name: git-credentials + usernameKey: username + passwordKey: token + syncPolicy: + interval: "30m" + onUpdate: update +``` + +### Creating a Registry from Another Registry API + +```yaml +apiVersion: toolhive.stacklok.io/v1alpha1 +kind: MCPRegistry +metadata: + name: curated-upstream + namespace: production +spec: + displayName: "Curated Upstream Registry" + description: "Filtered view of upstream community registry" + format: upstream + source: + type: registry + registry: + url: "http://registry-api.upstream-cluster.svc.cluster.local:8080/api/v1/servers" + authSecret: + name: upstream-api-credentials + usernameKey: username + passwordKey: token + filter: + include: + tiers: ["Official"] + categories: ["filesystem", "database"] + syncPolicy: + interval: "1h" + onUpdate: update +``` + +### Creating an MCPServer from Registry Data + +```bash +# Direct creation using registry information +kubectl create -f - <