Skip to content

Commit 4ef4a16

Browse files
JAORMXclaude
andauthored
vMCP: Implement CLI serve command and session management (#2406)
* Implement CLI serve command and session management for Virtual MCP Server This completes the MCP protocol server implementation by adding CLI integration, session management, and health endpoints. The Virtual MCP Server can now aggregate multiple backend MCP servers from a ToolHive group and serve them through a unified MCP endpoint. Changes: - Implement 'vmcp serve' command with full component wiring - Integrate MCP session management using ToolHive's session.Manager - Add /health and /ping endpoints with minimal security-conscious responses - Create session adapter exposing session.Manager via SDK interface - Add comprehensive test coverage for session lifecycle and health endpoints - Provide example configuration file with documentation Session Management Architecture: Sessions are ENTIRELY managed by ToolHive's session.Manager (the same component used in pkg/transport/proxy/streamable for MCP sessions). The mark3labs SDK does NOT manage sessions - it only calls our sessionIDAdapter interface during MCP protocol flows: - Generate(): Called on MCP initialize to create session IDs - Validate(): Called on every request to check session validity - Terminate(): Called on HTTP DELETE to end sessions All session storage, TTL-based cleanup, and lifecycle management is handled by ToolHive infrastructure. The adapter is purely glue code following DDD principles (infrastructure in server bounded context). Security: - Health endpoint intentionally minimal to prevent information disclosure - No version, session counts, or capability metrics exposed - Cryptographically secure session IDs per MCP specification - Session termination properly handled with 404 semantics Testing: - Added 381 lines of test coverage for new functionality - Session adapter: 9 test cases covering full lifecycle - Health endpoints: 3 test cases including security validation The Virtual MCP Server is now functionally complete for basic operation, pending authentication implementation in separate issues. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Address PR review feedback and fix test failures Improve error handling and add timeouts: - Return empty session ID when storage fails, allowing graceful degradation - Add 30-second timeout for capability aggregation to prevent hangs Fix parallel test execution: - Support port 0 for dynamic port allocation in tests - Add listener tracking to return actual bound port - Protect listener access with RWMutex to prevent data races All tests pass with race detection enabled. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 7792dba commit 4ef4a16

File tree

7 files changed

+1066
-39
lines changed

7 files changed

+1066
-39
lines changed

cmd/vmcp/README.md

Lines changed: 61 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -6,16 +6,25 @@ The Virtual MCP Server (vmcp) is a standalone binary that aggregates multiple MC
66

77
## Features
88

9-
- **Group-Based Backend Management**: References an existing ToolHive group for automatic workload discovery
10-
- **Tool Aggregation**: Combines tools from multiple MCP servers with conflict resolution (prefix, priority, or manual)
11-
- **Resource & Prompt Aggregation**: Unified access to resources and prompts from all backends
12-
- **Two Authentication Boundaries**:
13-
- **Incoming Auth** (Client → Virtual MCP): OIDC, local, or anonymous authentication
14-
- **Outgoing Auth** (Virtual MCP → Backend APIs): RFC 8693 token exchange for backend API access
15-
- **Per-Backend Token Exchange**: Different authentication strategies per backend (pass_through, token_exchange, service_account)
16-
- **Authorization**: Cedar policy-based access control
17-
- **Operational Features**: Circuit breakers, health checks, timeout management, failure handling
18-
- **Future**: Composite tools with elicitation support for multi-step workflows
9+
### Implemented (Phase 1)
10+
-**Group-Based Backend Management**: Automatic workload discovery from ToolHive groups
11+
-**Tool Aggregation**: Combines tools from multiple MCP servers with conflict resolution (prefix, priority, manual)
12+
-**Resource & Prompt Aggregation**: Unified access to resources and prompts from all backends
13+
-**Request Routing**: Intelligent routing of tool/resource/prompt requests to correct backends
14+
-**Session Management**: MCP protocol session tracking with TTL-based cleanup
15+
-**Health Endpoints**: `/health` and `/ping` for service monitoring
16+
-**Configuration Validation**: `vmcp validate` command for config verification
17+
18+
### In Progress
19+
- 🚧 **Incoming Authentication** (Issue #165): OIDC, local, anonymous authentication
20+
- 🚧 **Outgoing Authentication** (Issue #160): RFC 8693 token exchange for backend API access
21+
- 🚧 **Token Caching**: Memory and Redis cache providers
22+
- 🚧 **Health Monitoring** (Issue #166): Circuit breakers, backend health checks
23+
24+
### Future (Phase 2+)
25+
- 📋 **Authorization**: Cedar policy-based access control
26+
- 📋 **Composite Tools**: Multi-step workflows with elicitation support
27+
- 📋 **Advanced Routing**: Load balancing, failover strategies
1928

2029
## Installation
2130

@@ -39,6 +48,48 @@ task build-vmcp-image
3948
docker pull ghcr.io/stacklok/toolhive/vmcp:latest
4049
```
4150

51+
## Quick Start
52+
53+
```bash
54+
# 1. Create a ToolHive group
55+
thv group create my-team
56+
57+
# 2. Run some MCP servers in the group
58+
thv run github --name github-mcp --group my-team
59+
thv run fetch --name fetch-mcp --group my-team
60+
61+
# 3. Create a vmcp configuration file (see example-config.yaml)
62+
cat > vmcp-config.yaml <<EOF
63+
name: "my-vmcp"
64+
group: "my-team"
65+
incoming_auth:
66+
type: anonymous
67+
outgoing_auth:
68+
source: inline
69+
default:
70+
type: pass_through
71+
aggregation:
72+
conflict_resolution: prefix
73+
conflict_resolution_config:
74+
prefix_format: "{workload}_"
75+
EOF
76+
77+
# 4. Validate the configuration
78+
vmcp validate --config vmcp-config.yaml
79+
80+
# 5. Start the Virtual MCP Server
81+
vmcp serve --config vmcp-config.yaml
82+
83+
# 6. Test the health endpoint
84+
curl http://127.0.0.1:4483/health
85+
# {"status":"ok"}
86+
87+
# 7. Connect your MCP client to http://127.0.0.1:4483/mcp
88+
# The client will see aggregated tools from all backends:
89+
# - github-mcp_create_issue, github-mcp_list_repos, ...
90+
# - fetch-mcp_fetch, ...
91+
```
92+
4293
## Usage
4394

4495
### CLI Commands

cmd/vmcp/app/commands.go

Lines changed: 122 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -2,13 +2,21 @@
22
package app
33

44
import (
5+
"context"
56
"fmt"
7+
"time"
68

79
"github.com/spf13/cobra"
810
"github.com/spf13/viper"
911

12+
"github.com/stacklok/toolhive/pkg/groups"
1013
"github.com/stacklok/toolhive/pkg/logger"
14+
"github.com/stacklok/toolhive/pkg/vmcp/aggregator"
15+
vmcpclient "github.com/stacklok/toolhive/pkg/vmcp/client"
1116
"github.com/stacklok/toolhive/pkg/vmcp/config"
17+
vmcprouter "github.com/stacklok/toolhive/pkg/vmcp/router"
18+
vmcpserver "github.com/stacklok/toolhive/pkg/vmcp/server"
19+
"github.com/stacklok/toolhive/pkg/workloads"
1220
)
1321

1422
var rootCmd = &cobra.Command{
@@ -74,18 +82,7 @@ func newServeCmd() *cobra.Command {
7482
The server will read the configuration file specified by --config flag and start
7583
listening for MCP client connections. It will aggregate tools, resources, and prompts
7684
from all configured backend MCP servers.`,
77-
RunE: func(_ *cobra.Command, _ []string) error {
78-
configPath := viper.GetString("config")
79-
if configPath == "" {
80-
return fmt.Errorf("no configuration file specified, use --config flag")
81-
}
82-
83-
logger.Infof("Loading configuration from: %s", configPath)
84-
// TODO: Load configuration and start server
85-
// This will be implemented in a future PR when pkg/vmcp is added
86-
87-
return fmt.Errorf("serve command not yet implemented")
88-
},
85+
RunE: runServe,
8986
}
9087
}
9188

@@ -171,3 +168,116 @@ func getVersion() string {
171168
// This will be replaced with actual version info using ldflags
172169
return "dev"
173170
}
171+
172+
// runServe implements the serve command logic
173+
func runServe(cmd *cobra.Command, _ []string) error {
174+
ctx := cmd.Context()
175+
configPath := viper.GetString("config")
176+
177+
if configPath == "" {
178+
return fmt.Errorf("no configuration file specified, use --config flag")
179+
}
180+
181+
logger.Infof("Loading configuration from: %s", configPath)
182+
183+
// Load configuration from YAML
184+
loader := config.NewYAMLLoader(configPath)
185+
cfg, err := loader.Load()
186+
if err != nil {
187+
logger.Errorf("Failed to load configuration: %v", err)
188+
return fmt.Errorf("configuration loading failed: %w", err)
189+
}
190+
191+
// Validate configuration
192+
validator := config.NewValidator()
193+
if err := validator.Validate(cfg); err != nil {
194+
logger.Errorf("Configuration validation failed: %v", err)
195+
return fmt.Errorf("validation failed: %w", err)
196+
}
197+
198+
logger.Infof("Configuration loaded and validated successfully")
199+
logger.Infof(" Name: %s", cfg.Name)
200+
logger.Infof(" Group: %s", cfg.GroupRef)
201+
logger.Infof(" Conflict Resolution: %s", cfg.Aggregation.ConflictResolution)
202+
203+
// Initialize managers for backend discovery
204+
logger.Info("Initializing workload and group managers")
205+
workloadsManager, err := workloads.NewManager(ctx)
206+
if err != nil {
207+
return fmt.Errorf("failed to create workloads manager: %w", err)
208+
}
209+
210+
groupsManager, err := groups.NewManager()
211+
if err != nil {
212+
return fmt.Errorf("failed to create groups manager: %w", err)
213+
}
214+
215+
// Create backend discoverer
216+
discoverer := aggregator.NewCLIBackendDiscoverer(workloadsManager, groupsManager)
217+
218+
// Discover backends from the configured group
219+
logger.Infof("Discovering backends in group: %s", cfg.GroupRef)
220+
backends, err := discoverer.Discover(ctx, cfg.GroupRef)
221+
if err != nil {
222+
return fmt.Errorf("failed to discover backends: %w", err)
223+
}
224+
225+
if len(backends) == 0 {
226+
return fmt.Errorf("no backends found in group %s", cfg.GroupRef)
227+
}
228+
229+
logger.Infof("Discovered %d backends", len(backends))
230+
231+
// Create backend client
232+
backendClient := vmcpclient.NewHTTPBackendClient()
233+
234+
// Create conflict resolver based on configuration
235+
// Use the factory method that handles all strategies
236+
conflictResolver, err := aggregator.NewConflictResolver(cfg.Aggregation)
237+
if err != nil {
238+
return fmt.Errorf("failed to create conflict resolver: %w", err)
239+
}
240+
241+
// Create aggregator
242+
agg := aggregator.NewDefaultAggregator(backendClient, conflictResolver, cfg.Aggregation.Tools)
243+
244+
// Aggregate capabilities from all backends with timeout
245+
logger.Info("Aggregating capabilities from backends")
246+
aggCtx, cancel := context.WithTimeout(ctx, 30*time.Second)
247+
defer cancel()
248+
249+
capabilities, err := agg.AggregateCapabilities(aggCtx, backends)
250+
if err != nil {
251+
return fmt.Errorf("failed to aggregate capabilities: %w", err)
252+
}
253+
254+
logger.Infof("Aggregated %d tools, %d resources, %d prompts from %d backends",
255+
capabilities.Metadata.ToolCount,
256+
capabilities.Metadata.ResourceCount,
257+
capabilities.Metadata.PromptCount,
258+
capabilities.Metadata.BackendCount)
259+
260+
// Create router
261+
rtr := vmcprouter.NewDefaultRouter()
262+
263+
// Create server configuration
264+
serverCfg := &vmcpserver.Config{
265+
Name: cfg.Name,
266+
Version: getVersion(),
267+
Host: "127.0.0.1", // TODO: Make configurable
268+
Port: 4483, // TODO: Make configurable
269+
}
270+
271+
// Create server
272+
srv := vmcpserver.New(serverCfg, rtr, backendClient)
273+
274+
// Register capabilities
275+
logger.Info("Registering capabilities with server")
276+
if err := srv.RegisterCapabilities(ctx, capabilities); err != nil {
277+
return fmt.Errorf("failed to register capabilities: %w", err)
278+
}
279+
280+
// Start server (blocks until shutdown signal)
281+
logger.Infof("Starting Virtual MCP Server at %s", srv.Address())
282+
return srv.Start(ctx)
283+
}

cmd/vmcp/example-config.yaml

Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,118 @@
1+
# Virtual MCP Server Example Configuration
2+
#
3+
# This is a minimal example configuration for the Virtual MCP Server.
4+
# The Virtual MCP Server aggregates multiple MCP server workloads from a
5+
# ToolHive group into a single unified MCP endpoint.
6+
#
7+
# Usage:
8+
# vmcp serve --config example-config.yaml
9+
#
10+
# Prerequisites:
11+
# 1. Create a ToolHive group: thv group create engineering-team
12+
# 2. Run backend MCP servers: thv run github --group engineering-team
13+
# 3. Start Virtual MCP: vmcp serve --config this-file.yaml
14+
15+
# Virtual MCP Server name
16+
name: "engineering-vmcp"
17+
18+
# Reference to ToolHive group containing backend MCP servers
19+
# This group should contain one or more running MCP server workloads
20+
group: "engineering-team"
21+
22+
# ===== INCOMING AUTHENTICATION (Client → Virtual MCP) =====
23+
# Currently not implemented - this configuration is a placeholder for
24+
# future implementation (Issue #165)
25+
incoming_auth:
26+
type: anonymous # Options: oidc | anonymous | local
27+
# OIDC configuration (when type=oidc, not yet implemented):
28+
# oidc:
29+
# issuer: "https://keycloak.example.com/realms/myrealm"
30+
# client_id: "vmcp-client"
31+
# client_secret_env: "VMCP_CLIENT_SECRET"
32+
# audience: "vmcp"
33+
# scopes: ["openid", "profile", "email"]
34+
35+
# ===== OUTGOING AUTHENTICATION (Virtual MCP → Backends) =====
36+
# Currently not implemented - this configuration is a placeholder for
37+
# future implementation (Issue #160)
38+
outgoing_auth:
39+
source: inline # Options: inline | discovered
40+
41+
# Default behavior for backends without explicit config
42+
default:
43+
type: pass_through # Options: pass_through | token_exchange | service_account
44+
45+
# Per-backend authentication (not yet implemented)
46+
# backends:
47+
# github:
48+
# type: token_exchange
49+
# token_exchange:
50+
# token_url: "https://keycloak.example.com/realms/myrealm/protocol/openid-connect/token"
51+
# client_id: "vmcp-github-exchange"
52+
# client_secret_env: "GITHUB_EXCHANGE_SECRET"
53+
# audience: "github-api"
54+
# scopes: ["repo", "read:org"]
55+
56+
# ===== TOOL AGGREGATION =====
57+
aggregation:
58+
# Conflict resolution strategy when multiple backends have tools with the same name
59+
# Options: prefix | priority | manual
60+
conflict_resolution: prefix
61+
62+
# Configuration for the chosen strategy
63+
conflict_resolution_config:
64+
# For prefix strategy: format for prefixing tool names
65+
# Options: {workload}_ | {workload}. | custom-prefix-
66+
prefix_format: "{workload}_"
67+
68+
# For priority strategy: explicit backend ordering (first wins)
69+
# priority_order: ["github", "jira", "slack"]
70+
71+
# Per-workload tool filtering and overrides (optional)
72+
# tools:
73+
# - workload: "github"
74+
# # Include only specific tools (omit to include all)
75+
# filter: ["create_pr", "merge_pr", "list_issues"]
76+
# # Rename tools to avoid conflicts
77+
# overrides:
78+
# create_pr:
79+
# name: "gh_create_pr"
80+
# description: "Create a GitHub pull request"
81+
82+
# ===== TOKEN CACHING =====
83+
# Token cache configuration (not yet implemented)
84+
# This will be used when outgoing authentication is implemented
85+
# token_cache:
86+
# provider: memory # Options: memory | redis
87+
# config:
88+
# max_entries: 1000
89+
# ttl_offset: "5m" # Refresh tokens 5 minutes before expiry
90+
91+
# ===== OPERATIONAL SETTINGS =====
92+
# Operational configuration (partially implemented)
93+
# operational:
94+
# timeouts:
95+
# default: 30s
96+
# per_workload:
97+
# github: 45s
98+
#
99+
# failure_handling:
100+
# health_check_interval: 30s
101+
# unhealthy_threshold: 3
102+
# partial_failure_mode: fail # Options: fail | best_effort
103+
# circuit_breaker:
104+
# enabled: true
105+
# failure_threshold: 5
106+
# timeout: 60s
107+
108+
# ===== COMPOSITE TOOLS (Phase 2) =====
109+
# Composite tools for multi-step workflows (not yet implemented)
110+
# composite_tools:
111+
# - name: "deploy_and_notify"
112+
# description: "Deploy PR and notify team"
113+
# parameters:
114+
# pr_number: {type: "integer"}
115+
# steps:
116+
# - id: "merge"
117+
# tool: "github.merge_pr"
118+
# arguments: {pr: "{{.params.pr_number}}"}

0 commit comments

Comments
 (0)