diff --git a/docs/docs/architecture/adr/023-one-time-authentication-servers.md b/docs/docs/architecture/adr/023-one-time-authentication-servers.md new file mode 100644 index 000000000..29c5e379d --- /dev/null +++ b/docs/docs/architecture/adr/023-one-time-authentication-servers.md @@ -0,0 +1,207 @@ +# ADR-023: One-Time Authentication for Gateway Registration + +**Status:** Accepted +**Date:** 2025-11-10 +**Deciders:** Keval Mahajan +**Technical Story:** [#1387](https://github.com/IBM/mcp-context-forge/issues/1387) + +## Context and Problem Statement + +Context Forge Gateway allows users to register MCP servers with authentication credentials to proxy and expose their tools through virtual servers. Prior to this implementation, the gateway required storing authentication credentials (bearer tokens, basic auth) in the database to maintain persistent connections and perform health checks on registered MCP servers. + +However, a significant use case emerged: organizations and users who want to leverage the gateway's tool proxying capabilities but cannot or will not store authentication credentials in the gateway database due to security policies, compliance requirements, or organizational constraints. These users still needed a mechanism to register MCP servers with authentication, discover available tools, and access them through the gateway using runtime-provided credentials via passthrough headers. + +The existing authentication flow required credentials to be persisted, making it incompatible with scenarios where: +- Security policies prohibit credential storage in third-party systems +- Organizations prefer to use runtime authentication with short-lived tokens +- Users want to leverage passthrough header mechanisms for authentication +- Vault plugins or external secret management solutions are not available or suitable + +**How can users register authenticated MCP servers and use their tools through the gateway without persisting credentials in the database?** + +## Decision Drivers + +- Security requirement: Avoid storing authentication credentials in the gateway database +- Compliance requirement: Meet organizational policies that prohibit credential persistence +- Usability requirement: Maintain automatic tool discovery and registration benefits +- Flexibility requirement: Support runtime authentication with passthrough headers +- Compatibility requirement: Work with existing transport protocols (SSE, STREAMABLEHTTP) + +## Considered Options + +1. **One-Time Authentication with Passthrough Headers** (Chosen) +2. Two-Step Registration API (Add gateway, then connect separately) +3. Mandatory Vault Plugin Integration +4. Credential-Free Registration with Manual Tool Configuration + +## Decision Outcome + +Chosen option: **"One-Time Authentication with Passthrough Headers"**, because it provides the optimal balance between security, usability, and flexibility. It enables automatic tool discovery while ensuring credentials are never persisted, and it works seamlessly with the existing passthrough header infrastructure. + +### Implementation Details + +The solution introduces a `one_time_auth` boolean flag in the gateway registration API (`POST /gateways`). When enabled, the authentication flow works as follows: + +1. **Initial Registration**: User provides authentication credentials (`auth_type` and `auth_token`) along with `one_time_auth: true` in the registration request +2. **One-Time Connection**: Gateway uses the provided credentials exactly once to establish a connection with the MCP server +3. **Tool Discovery**: During this single authenticated connection, the gateway performs complete tool discovery, retrieving all available tools, their schemas, and metadata +4. **Credential Discard**: After tool discovery completes successfully, the authentication credentials are immediately discarded and not persisted to the database +5. **Passthrough Configuration**: Users must configure `passthrough_headers` (specifically `X-Upstream-Authorization` for authorization headers) to enable runtime authentication +6. **Runtime Authentication**: Subsequent requests to the MCP server tools must include authentication credentials via passthrough headers provided by the client + +```mermaid +graph LR + A[User] --> B["Add MCP Server"] + B --> C{"Enable One-Time Authentication?"} + C -->|Yes| D["Do Not Store Credentials in DB"] + C -->|No| E["Store Credentials in DB"] + D --> F["Configure Passthrough Headers
(X-Upstream-Authorization)"] + F --> G["Create Virtual Server"] + G --> H["Link to MCP Server"] + H --> I["Add Authentication Headers"] + I --> J["Test Connection"] + J --> K["Successful Authentication"] + E --> L["Create Virtual Server with Stored Credentials"] + + classDef step fill:#1e1e1e,stroke:#ffffff,stroke-width:2px,color:#f5f5f5; + class A,B,C,D,E,F,G,H,I,J,K,L step; +``` + +### Technical Changes + +Key technical changes include: + +- Added `one_time_auth` boolean field to the gateway registration schema +- Modified the gateway registration flow to conditionally store or discard credentials based on the flag +- Implemented automatic disabling of health checks for one-time auth gateways (since no stored credentials are available for health check execution) +- Updated the frontend UI to include a one-time authentication checkbox in the gateway configuration forms +- Modified the authentication handling logic to support both stored credentials and passthrough header workflows +- Ensured compatibility with both SSE and STREAMABLEHTTP transport protocols + +Authentication Flow: + +```mermaid +sequenceDiagram + participant User + participant Gateway + participant Database + participant MCP Server + + Note over User,MCP Server: Traditional Flow (Credentials Stored) + User->>Gateway: POST /gateways (with credentials) + Gateway->>MCP Server: Connect with credentials + MCP Server-->>Gateway: Tool list + Gateway->>Database: Store credentials + Gateway-->>User: Gateway registered + + Note over User,MCP Server: Later requests + User->>Gateway: Request tool + Gateway->>Database: Retrieve credentials + Gateway->>MCP Server: Call tool (with stored credentials) + MCP Server-->>Gateway: Tool response + Gateway-->>User: Response + + Note over User,MCP Server: One-Time Auth Flow (Credentials NOT Stored) + User->>Gateway: POST /gateways (one_time_auth=true, credentials) + Gateway->>MCP Server: Connect with credentials (ONE TIME) + MCP Server-->>Gateway: Tool list + Gateway->>Gateway: Discard credentials + Gateway-->>User: Gateway registered (no credentials saved) + + Note over User,MCP Server: Later requests + User->>Gateway: Request tool (with X-Upstream-Authorization header) + Gateway->>MCP Server: Call tool (forward passthrough auth) + MCP Server-->>Gateway: Tool response + Gateway-->>User: Response +``` + + +### Positive Consequences + +- **Enhanced Security Posture**: Credentials are never persisted in the gateway database, reducing the attack surface and exposure risk +- **Compliance Enablement**: Organizations with strict security policies can now use the gateway without violating credential storage restrictions +- **Runtime Authentication Flexibility**: Users can rotate credentials, use short-lived tokens, or implement dynamic authentication strategies without updating gateway configuration +- **Preserved Tool Discovery**: Users still benefit from automatic tool discovery and schema registration without manual configuration +- **Backward Compatibility**: The feature is opt-in and does not affect existing gateway registration workflows + +### Negative Consequences + +- **Health Check Limitation**: Health checks are automatically disabled for one-time auth gateways since no stored credentials are available to authenticate health check requests +- **Client Configuration Complexity**: MCP clients must be configured to provide passthrough headers with authentication credentials for every request, increasing configuration complexity +- **Credential Management Burden**: Users are responsible for managing and distributing authentication credentials to all clients, rather than centralizing them in the gateway +- **Reconnection Requirements**: If tools need to be re-discovered or the gateway needs to reconnect, users must re-register the gateway with credentials since none are stored + +## Pros and Cons of the Options + +### Option 1: One-Time Authentication with Passthrough Headers (Chosen) + +**Description**: Introduce a `one_time_auth` flag that uses credentials once for tool discovery, then discards them. Runtime authentication is handled via passthrough headers. + +- **Good**, because it eliminates credential persistence security risks +- **Good**, because it preserves automatic tool discovery benefits +- **Good**, because it provides flexibility for runtime authentication strategies +- **Good**, because it is backward compatible and opt-in +- **Bad**, because health checks cannot be performed without stored credentials +- **Bad**, because it increases client configuration complexity + +### Option 2: Two-Step Registration API + +**Description**: Split the `POST /gateways` API into two separate operations: (1) Add gateway metadata, (2) Establish connection and perform tool discovery. + +- **Good**, because it separates concerns between registration and authentication +- **Bad**, because it requires two API calls instead of one atomic operation +- **Bad**, because it increases implementation complexity +- **Bad**, because it creates potential for incomplete registrations +- **Neutral**, because it could enable other use cases in the future + +### Option 3: Mandatory Vault Plugin Integration + +**Description**: Require all users concerned about credential storage to use the Vault plugin for external secret management. + +- **Good**, because it provides centralized secret management +- **Good**, because it enables health checks with externally stored credentials +- **Bad**, because not all users have vault infrastructure available +- **Bad**, because it imposes infrastructure requirements on users +- **Bad**, because it doesn't support scenarios where runtime passthrough authentication is preferred + +### Option 4: Credential-Free Registration with Manual Tool Configuration + +**Description**: Allow users to register gateways without any initial authentication and manually configure tool schemas. + +- **Good**, because no credentials are ever provided to the gateway +- **Bad**, because it eliminates automatic tool discovery benefits +- **Bad**, because it requires extensive manual configuration of tool schemas +- **Bad**, because it creates maintenance burden when tools change + +## Validation + +The feature was tested and validated in the following scenarios: + +- MCP servers with SSE transport using Bearer authentication +- MCP servers with SSE transport using Basic authentication +- MCP servers with STREAMABLEHTTP transport using Bearer authentication +- MCP servers with STREAMABLEHTTP transport using Basic authentication +- Tool invocation from Admin UI +- Tool invocation from MCP Inspector via virtual server + +All test scenarios successfully validated that: +- Credentials were not stored in the database +- Tool discovery completed successfully during one-time authentication +- Tool calls functioned correctly when authentication was provided via passthrough headers +- Both transport protocols worked as expected + +## Links + +- GitHub Pull Request: [#1407](https://github.com/IBM/mcp-context-forge/pull/1407) +- GitHub Issue: [#1387](https://github.com/IBM/mcp-context-forge/issues/1387) +- Usage Documentation: `https://ibm.github.io/mcp-context-forge/overview/passthrough/#usage-with-one-time-auth` +- MCP Authorization Specification: OAuth 2.1 with Protected Resource Metadata (RFC 9728) + +## Notes + +- Documentation for this feature has been added to `docs/docs` including usage examples with Claude Desktop configuration +- The feature supports both bearer and basic authentication schemes +- The `X-Upstream-Authorization` passthrough header is mapped to the `Authorization` header when forwarding requests to upstream MCP servers +- Implementation maintains compatibility with existing gateway features including virtual servers, tool proxying, and RBAC +- Lint checks (`make lint`) and test suite (`make test`) pass successfully +- CHANGELOG has been updated with user-facing changes diff --git a/docs/docs/overview/passthrough.md b/docs/docs/overview/passthrough.md index a3b8c6476..6585cdc15 100644 --- a/docs/docs/overview/passthrough.md +++ b/docs/docs/overview/passthrough.md @@ -286,6 +286,195 @@ DEFAULT_PASSTHROUGH_HEADERS=["X-Tenant-Id", "X-Trace-Id", "X-Request-Id"] } ``` + +## Usage with One-Time Auth + +The One-Time Authentication feature enables integration with an authenticated MCP server without persisting server credentials in the database. Authentication is performed using a one-time token during initial registration, combined with passthrough headers to securely forward authentication context for subsequent requests. + +### Workflow Overview + +The one-time auth workflow consists of three main steps: + +1. **Register the Gateway** - Authenticate once to discover tools, then discard credentials +2. **Create Virtual Servers** - Expose selected tools through virtual server endpoints +3. **Connect MCP Clients** - Use passthrough headers to authenticate requests + +```mermaid +graph LR + A[User] --> B["Add MCP Server"] + B --> C{"Enable One-Time Authentication?"} + C -->|Yes| D["Do Not Store Credentials in DB"] + C -->|No| E["Store Credentials in DB"] + D --> F["Configure Passthrough Headers (X-Upstream-Authorization)"] + F --> G["Create Virtual Server"] + G --> H["Link to MCP Server"] + H --> I["Add Authentication Headers"] + I --> J["Test Connection"] + J --> K["Successful Authentication"] + E --> L["Create Virtual Server with Stored Credentials"] + + %% Define styling for steps (Dark mode colors) + classDef step fill:#1e1e1e,stroke:#ffffff,stroke-width:2px,color:#f5f5f5; + class A,B,C,D,E,F,G,H,I,J,K,L step; +``` + +### Step 1: Register Gateway with One-Time Authentication + +Register an MCP server with authentication details and enable one-time auth. The gateway will: +- Use the provided credentials **once** to connect to the MCP server +- Discover and retrieve all available tools and metadata +- **Discard the authentication details** without storing them in the database + +**API Request:** + +```bash +POST /gateways +Content-Type: application/json +Authorization: Bearer + +{ + "name": "my-server", + "url": "http://localhost:8005/mcp", + "description": "Authenticated MCP server with one-time auth", + "transport": "STREAMABLEHTTP", + "passthrough_headers": [ + "X-Upstream-Authorization" + ], + "auth_type": "bearer", + "auth_token": "super-secret-123", + "one_time_auth": true +} +``` + +**Key Fields:** + +| Field | Required | Description | +|-------|----------|-------------| +| `name` | Yes | Unique identifier for the gateway | +| `url` | Yes | MCP server endpoint URL | +| `transport` | Yes | Transport protocol (`STREAMABLEHTTP` or `SSE`) | +| `passthrough_headers` | Yes | Use include `X-Upstream-Authorization` for `Authorization` headers, for other headers, use as it is | +| `auth_type` | Yes | Authentication scheme (`bearer` or `basic`, etc) | +| `auth_token` | Yes | Token used **only once** for initial connection | +| `one_time_auth` | Yes | Set to `true` to enable one-time authentication | + +**What Happens:** + +1. Gateway connects to the MCP server using `auth_token` +2. Tool discovery is performed, retrieving all available tools +3. Tool metadata and schemas are stored in the database +4. **Credentials are immediately discarded** - not saved anywhere +5. Gateway registration completes successfully +6. Health checks are automatically disabled (no stored credentials available) + +### Step 2: Create Virtual Servers + +After gateway registration, create virtual servers to expose selected tools from the authenticated MCP server. Virtual servers act as proxies that use passthrough headers for authentication. + +**API Request:** + +```bash +POST /servers +Content-Type: application/json +Authorization: Bearer + +{ + "server": { + "name": "my-virtual-server", + "description": "Virtual server exposing authenticated tools", + "associated_tools": [ + "1a4712afecbf44408960da065c5183e3" + ], + "visibility": "public" + } +} +``` + +**Response:** + +The API returns the virtual server details including its unique ID (e.g., `aa9980f23d2c45d5a07727993565e9c8`), which is used to construct the MCP endpoint URL. + +### Step 3: Connect MCP Clients with Passthrough Headers + +Connect any MCP-compatible client to the virtual server, providing authentication via passthrough headers. The client must supply: + +1. **Gateway Authentication** - Token to authenticate with Context Forge Gateway (`Authorization` header) +2. **MCP Server Authentication** - Token forwarded to the upstream MCP server (`X-Upstream-Authorization` header) + +#### Example: Claude Desktop Configuration + +Add the following configuration to Claude Desktop's MCP settings: + +```json +{ + "mcpServers": { + "new-virtual-server": { + "command": "npx", + "args": [ + "-y", + "mcp-remote", + "http://localhost:4444/servers/aa9980f23d2c45d5a07727993565e9c8/mcp", + "--header", + "Authorization:${GATEWAY_AUTH_TOKEN}", + "--header", + "X-Upstream-Authorization:${MCP_SERVER_AUTH_TOKEN}" + ], + "env": { + "GATEWAY_AUTH_TOKEN": "Bearer ", + "MCP_SERVER_AUTH_TOKEN": "Bearer " + } + } + } +} +``` + +**Configuration Breakdown:** + +| Component | Purpose | +|-----------|---------| +| `mcp-remote` | NPM package for remote MCP connections | +| `http://localhost:4444/servers/...` | Virtual server endpoint URL | +| `Authorization` header | Authenticates with Context Forge Gateway | +| `X-Upstream-Authorization` header | Forwarded to the upstream MCP server | +| `GATEWAY_AUTH_TOKEN` | JWT or Bearer token for gateway access | +| `MCP_SERVER_AUTH_TOKEN` | Bearer token for the authenticated MCP server | + +**Environment Variables:** + +Replace the placeholder tokens with actual values: + +```bash +# Gateway authentication token (obtained from Context Forge) +GATEWAY_AUTH_TOKEN="Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..." + +# MCP server authentication token (your server's auth token) +MCP_SERVER_AUTH_TOKEN="Bearer super-secret-123" +``` + +### Authentication Flow Diagram + +```mermaid +sequenceDiagram + participant Client as MCP Client + participant Gateway as Context Forge Gateway + participant MCP as MCP Server + + Note over Gateway,MCP: Step 1: One-Time Registration + Gateway->>MCP: Connect with auth_token (one-time) + MCP-->>Gateway: Return tools & metadata + Note over Gateway: Store tools, discard credentials + + Note over Client,MCP: Step 2: Runtime Requests + Client->>Gateway: Request with Authorization header + Note over Gateway: Validate gateway auth + Client->>Gateway: Include X-Upstream-Authorization header + Gateway->>MCP: Forward request with Authorization header + Note over Gateway: Map X-Upstream-Authorization → Authorization + MCP-->>Gateway: Return response + Gateway-->>Client: Forward response +``` + + ## Troubleshooting ### Common Issues diff --git a/mcpgateway/admin.py b/mcpgateway/admin.py index 2cb621d06..603985159 100644 --- a/mcpgateway/admin.py +++ b/mcpgateway/admin.py @@ -6783,6 +6783,7 @@ async def admin_add_gateway(request: Request, db: Session = Depends(get_db), use auth_header_value=str(form.get("auth_header_value", "")), auth_headers=auth_headers if auth_headers else None, oauth_config=oauth_config, + one_time_auth=form.get("one_time_auth", False), passthrough_headers=passthrough_headers, visibility=visibility, ca_certificate=ca_certificate, @@ -7091,6 +7092,7 @@ async def admin_edit_gateway( auth_header_value=str(form.get("auth_header_value", "")), auth_value=str(form.get("auth_value", "")), auth_headers=auth_headers if auth_headers else None, + one_time_auth=form.get("one_time_auth", False), passthrough_headers=passthrough_headers, oauth_config=oauth_config, visibility=visibility, diff --git a/mcpgateway/schemas.py b/mcpgateway/schemas.py index 0e4164c42..01d0e6e91 100644 --- a/mcpgateway/schemas.py +++ b/mcpgateway/schemas.py @@ -612,7 +612,7 @@ def assemble_auth(cls, values: Dict[str, Any]) -> Dict[str, Any]: ) auth_type = values.get("auth_type") - if auth_type: + if auth_type and auth_type.lower() != "one_time_auth": if auth_type.lower() == "basic": creds = base64.b64encode(f"{values.get('auth_username', '')}:{values.get('auth_password', '')}".encode("utf-8")).decode() encoded_auth = encode_auth({"Authorization": f"Basic {creds}"}) @@ -1032,7 +1032,7 @@ def assemble_auth(cls, values: Dict[str, Any]) -> Dict[str, Any]: ) auth_type = values.get("auth_type") - if auth_type: + if auth_type and auth_type.lower() != "one_time_auth": if auth_type.lower() == "basic": creds = base64.b64encode(f"{values.get('auth_username', '')}:{values.get('auth_password', '')}".encode("utf-8")).decode() encoded_auth = encode_auth({"Authorization": f"Basic {creds}"}) @@ -2413,6 +2413,10 @@ class GatewayCreate(BaseModel): # Adding `auth_value` as an alias for better access post-validation auth_value: Optional[str] = Field(None, validate_default=True) + + # One time auth - do not store the auth in gateway flag + one_time_auth: Optional[bool] = Field(default=False, description="The authentication should be used only once and not stored in the gateway") + tags: Optional[List[str]] = Field(default_factory=list, description="Tags for categorizing the gateway") # Team scoping fields for resource organization @@ -2646,6 +2650,9 @@ def _process_auth_fields(info: ValidationInfo) -> Optional[str]: return encode_auth({header_key: header_value}) + if auth_type == "one_time_auth": + return None # No auth_value needed for one-time auth + raise ValueError("Invalid 'auth_type'. Must be one of: basic, bearer, oauth, or headers.") @@ -2677,6 +2684,9 @@ class GatewayUpdate(BaseModelWithConfigDict): # OAuth 2.0 configuration oauth_config: Optional[Dict[str, Any]] = Field(None, description="OAuth 2.0 configuration including grant_type, client_id, encrypted client_secret, URLs, and scopes") + # One time auth - do not store the auth in gateway flag + one_time_auth: Optional[bool] = Field(default=False, description="The authentication should be used only once and not stored in the gateway") + tags: Optional[List[str]] = Field(None, description="Tags for categorizing the gateway") # Team scoping fields for resource organization @@ -2879,6 +2889,9 @@ def _process_auth_fields(info: ValidationInfo) -> Optional[str]: return encode_auth({header_key: header_value}) + if auth_type == "one_time_auth": + return None # No auth_value needed for one-time auth + raise ValueError("Invalid 'auth_type'. Must be one of: basic, bearer, oauth, or headers.") @@ -3037,6 +3050,10 @@ def _populate_auth(self) -> Self: # They use oauth_config instead return self + if auth_type == "one_time_auth": + # One-time auth gateways don't store auth_value + return self + # If no encoded value is present, nothing to populate if not auth_value_encoded: return self @@ -4141,6 +4158,10 @@ def _process_auth_fields(info: ValidationInfo) -> Optional[str]: return encode_auth({header_key: header_value}) + if auth_type == "one_time_auth": + # One-time auth does not require encoding here + return None + raise ValueError("Invalid 'auth_type'. Must be one of: basic, bearer, oauth, or headers.") @@ -4425,6 +4446,10 @@ def _process_auth_fields(info: ValidationInfo) -> Optional[str]: return encode_auth({header_key: header_value}) + if auth_type == "one_time_auth": + # One-time auth does not require encoding here + return None + raise ValueError("Invalid 'auth_type'. Must be one of: basic, bearer, oauth, or headers.") @@ -4575,6 +4600,9 @@ def _populate_auth(self) -> Self: # They use oauth_config instead return self + if auth_type == "one_time_auth": + return self + # If no encoded value is present, nothing to populate if not auth_value_encoded: return self diff --git a/mcpgateway/services/gateway_service.py b/mcpgateway/services/gateway_service.py index 5472e17ee..8d3704351 100644 --- a/mcpgateway/services/gateway_service.py +++ b/mcpgateway/services/gateway_service.py @@ -738,6 +738,12 @@ async def register_gateway( ca_certificate = getattr(gateway, "ca_certificate", None) capabilities, tools, resources, prompts = await self._initialize_gateway(normalized_url, authentication_headers, gateway.transport, auth_type, oauth_config, ca_certificate) + if gateway.one_time_auth: + # For one-time auth, clear auth_type and auth_value after initialization + auth_type = "one_time_auth" + auth_value = None + oauth_config = None + tools = [ DbTool( original_name=tool.name, @@ -1407,6 +1413,12 @@ async def update_gateway( new_resource_uris = [resource.uri for resource in resources] new_prompt_names = [prompt.name for prompt in prompts] + if gateway_update.one_time_auth: + # For one-time auth, clear auth_type and auth_value after initialization + gateway.auth_type = "one_time_auth" + gateway.auth_value = None + gateway.oauth_config = None + # Update tools using helper method tools_to_add = self._update_or_create_tools(db, tools, gateway, "update") @@ -2182,6 +2194,10 @@ async def check_health_of_gateways(self, db: Session, gateways: List[DbGateway], # Create trace span for health check batch with create_span("gateway.health_check_batch", {"gateway.count": len(gateways), "check.type": "health"}) as batch_span: for gateway in gateways: + + if gateway.auth_type == "one_time_auth": + continue # Skip health check for one-time auth gateways as these are authenticated with passthrough headers only + # Create span for individual gateway health check with create_span( "gateway.health_check", diff --git a/mcpgateway/templates/admin.html b/mcpgateway/templates/admin.html index 1a702c800..4e3fd64bc 100644 --- a/mcpgateway/templates/admin.html +++ b/mcpgateway/templates/admin.html @@ -4608,6 +4608,32 @@

+ +
+ + +
+
+
+ + +