Skip to content

feat(networkproxy): Add TLS MITM, automatic CA management, header injection, and IPv6 support#323

Open
Danny-Wei wants to merge 18 commits intomainfrom
tls-mitm-and-key-injection
Open

feat(networkproxy): Add TLS MITM, automatic CA management, header injection, and IPv6 support#323
Danny-Wei wants to merge 18 commits intomainfrom
tls-mitm-and-key-injection

Conversation

@Danny-Wei
Copy link
Copy Markdown
Member

@Danny-Wei Danny-Wei commented Apr 28, 2026

Summary

This PR implements Phase 2 of the NetworkProxy enforcer: TLS Man-in-the-Middle (MITM) termination with automatic per-policy CA management, HTTP header injection (API key injection), and anti-Domain-Fronting protection. It also migrates the NetworkProxy configuration store from ConfigMap to Secret, adds IPv6 support, introduces configurable sidecar resource quotas, and reorganizes the profile generation code.

Major Changes

1. TLS MITM with Auto-Generated Per-Policy CA

The MITMConfig CRD has been redesigned:

  • Removed caSecretRef: vArmor now automatically generates a self-signed CA (ECDSA P-256, 10-year validity) per policy. Users no longer need to manually create and manage CA Secrets.
  • Added headerMutations: Per-domain HTTP header injection rules for injecting API keys, authorization tokens, or custom headers into decrypted requests.
    • HeaderAction supports either a literal value or a secretRef pointing to a user-managed Kubernetes Secret.
    • The Controller resolves secretRef at reconcile time and inlines the value into the Envoy xDS configuration.

Certificate lifecycle:

  • CA is generated once on first create or when MITM is toggled from off to on.
  • When domains change, only the leaf certificate is re-signed (CA remains stable so the CA bundle exposed to application containers does not churn).
  • The leaf certificate is a single multi-SAN certificate where SANs are populated verbatim from mitm.domains (including wildcard and IP SANs).

Anti-Domain-Fronting protection:

  • When MITM is enabled, Envoy terminates TLS and inspects the HTTP :authority header.
  • If SNI ≠ Host, the request returns 404 before reaching the router or injecting headers, preventing Domain-Fronting attacks.
  • Users are recommended to add all whitelist domains to mitm.domains even if they do not need key injection, to gain full anti-fronting coverage.

2. CA Bundle Injection into Application Containers

To make application containers trust the per-policy MITM CA without user configuration:

  • The Controller embeds the Mozilla CA Bundle (go:embed) and appends the policy's MITM CA to produce ca-bundle.crt.
  • The unified Secret stores ca-bundle.crt alongside xDS configuration and certificate material.
  • The Webhook injects a projected Secret volume (mitm-ca-bundle.crtca-certificates.crt) into every target container at /etc/varmor/ca-bundle/.
  • Four standard environment variables are injected (with idempotency checks) to point common TLS runtimes at the bundle:
    • SSL_CERT_FILE (Go, Rust, .NET)
    • REQUESTS_CA_BUNDLE (Python requests)
    • NODE_EXTRA_CA_CERTS (Node.js — append semantics)
    • CURL_CA_BUNDLE (curl/libcurl)

Compatibility: Works for Go, Python, Node.js, curl out of the box. Java (JKS truststore) is not supported in Phase 2.

Known risk: The injected environment variables cause runtimes to ignore the image's built-in CA store. Custom CAs added to the container image will be lost. This will be addressed in Phase 3 via CASecretRef.

3. ConfigMap → Secret Migration

The NetworkProxy configuration store is migrated from ConfigMap to Kubernetes Secret:

  • All xDS configs (bootstrap.yaml, lds.yaml, cds.yaml), MITM certificates, and the CA bundle are stored in a single unified Secret per policy.
  • Three projected Secret volumes are mounted into pods:
    1. varmor-network-proxy-config → Envoy sidecar /etc/envoy/ (bootstrap + xDS)
    2. varmor-network-proxy-mitm-tls → Envoy sidecar /etc/envoy/tls/ (leaf cert/key + upstream CA bundle)
    3. varmor-network-proxy-mitm-ca-bundle → App containers /etc/varmor/ca-bundle/ (public CA bundle only)
  • RBAC updated: configmaps permission removed; secrets (get, create, update, delete) added.
  • No automatic cleanup of legacy ConfigMaps is performed; users must manually delete old varmor-* ConfigMaps after upgrading.

4. Translator / Renderer Extension for MITM

  • classifyEgress(): HTTPS traffic matching MITM domains is routed to MITM filter chains instead of Phase 1 TLS passthrough.
  • buildMITMChains(): Generates up to two filter chains — a DNS chain (server_names) and an IP/CIDR chain (prefix_ranges). They are split because Envoy applies AND semantics between server_names and prefix_ranges in the same match.
  • DownstreamTlsContext: All MITM chains share the same leaf certificate pair mounted at /etc/envoy/tls/leaf.crt and leaf.key.
  • UpstreamTlsContext: A dedicated mitm_upstream cluster uses auto_sni and auto_san_validation to preserve the original upstream SNI and certificate validation. The upstream trust store uses the concatenated Mozilla + vArmor CA bundle.
  • Header mutation: Injected headers use OVERWRITE_IF_EXISTS_OR_ADD semantics at the virtual host level.
  • L7 RBAC reuse: MITM HCM reuses the existing HTTP RBAC filters (shadow → deny → allow → router) from Phase 1.
  • Dead-rule cleanup: When MITM is enabled, egress/HTTP rules targeting MITM domains are filtered out of the tls_chain to avoid generating dead rules.

5. IPv6 Support for NetworkProxy Enforcer

Added detection of cluster single-stack vs dual-stack configuration. The controller now creates dedicated listeners to handle both IPv4 and IPv6 egress traffic correctly.

6. Code Organization Refactoring

  • Moved internal/networkproxy/*.gointernal/networkproxy/profile/ (translator, renderer, tests, MITM resolver).
  • Added internal/networkproxy/mitm/ package: CA generation (ca.go), leaf signing (leaf.go), Mozilla bundle management (bundle.go), and comprehensive unit tests.
  • Added internal/networkproxy/profile/networkproxy.go as the orchestration entry point (Kubernetes object lifecycle) and internal/networkproxy/profile/mitm_resolver.go for Controller-side Secret resolution.

7. Example Policies

Added 11 example YAMLs under test/examples/7-networkproxy-mitm/ covering:

  • DNS literal, wildcard, multi-domain, mixed IP/DNS
  • IP literal, CIDR
  • Header injection from literal value and from Secret reference
  • Integration with L4 rules, deny HTTP rules, and audit logging

8. Configurable Sidecar Resource Quotas

MITM mode requires the sidecar to perform dual TLS handshakes (client-side + upstream), consuming significantly more resources than a plain L4 proxy. The previously hard-coded defaults (50m/64Mi requests, 200m/128Mi limits) were insufficient for MITM workloads.

MITM-aware default resources:

Mode CPU requests Memory requests CPU limits Memory limits
Non-MITM 50m 64Mi 500m 256Mi
MITM 100m 128Mi 1000m 512Mi

Defaults are provided by DefaultProxyResources(mitmEnabled bool) in internal/policy/proxy_resources.go.

CRD per-policy override:

A new ProxyResourceOverride type is added to NetworkProxyConfig, supporting field-level merge — users only specify the fields they want to override, and the rest retain the defaults:

spec:
  policy:
    networkProxyConfig:
      resources:
        requests:
          cpu: "200m"
        limits:
          cpu: "2000m"
          memory: "1Gi"

ResolveProxyResources(override, mitmEnabled) performs the field-level merge and is called by both the Webhook (mutation.go) and the Policy Controller (update.go).

Validation:

validateProxyResources() in validate.go ensures that when both requests and limits specify the same resource type, limits must be greater than or equal to requests.

Immutable proxy fields:

ProxyUID, ProxyPort, and ProxyAdminPort are now enforced as immutable during policy updates. The validation path receives the old ProxyConfig to detect mutations, while the controller reconcile path passes nil to skip this check.

Phase 3 direction:

A cluster-level default override via the varmor-config ConfigMap is planned, forming a three-tier merge chain: per-policy override → global ConfigMap → built-in defaults.

9. Dependency Updates

  • Upgraded Envoy sidecar image to v1.38-latest.

Files Changed

Area Files
CRD apis/varmor/v1beta1/networkproxy.go, zz_generated.deepcopy.go, CRD YAMLs
MITM crypto internal/networkproxy/mitm/ca.go, leaf.go, bundle.go, certs/mozilla.pem
Profile / xDS internal/networkproxy/profile/translator*.go, renderer.go, networkproxy.go, mitm_resolver.go
Controller internal/policy/*_controller.go, update.go, proxy_config_propagator.go, validate.go, proxy_resources.go
Webhook internal/webhooks/mutation.go, server.go
RBAC config/k8s-resource/rbac/manager-clusterrole.yaml, Helm templates
Tests *_test.go across mitm/, profile/, policy/, webhooks/
Examples test/examples/7-networkproxy-mitm/*.yaml

Test Plan

  • go test ./internal/networkproxy/mitm/... — CA generation, leaf signing, bundle concatenation, renewal semantics
  • go test ./internal/networkproxy/profile/... — MITM chain generation, translator / renderer output, dead-rule filtering
  • go test ./internal/policy/... — Controller reconcile with resolved headers, Secret size guards, error handling
  • go test ./internal/webhooks/... — Pod mutation with MITM volumes and env vars
  • Deploy an example MITM policy and verify:
    • HTTPS traffic to target domains is decrypted and headers are injected
    • SNI ≠ Host requests return 404
    • Non-MITM domains remain passthrough
    • Application containers trust the MITM CA via injected env vars

Known Limitations

  1. IP-based MITM + plaintext HTTP L7 rules: When a /32 IP is configured in mitm.domains, plaintext HTTP traffic to that IP cannot reach the http_chain due to Envoy filter chain matching semantics (prefix_ranges causes less-specific chains to be eliminated). L7 rules for that IP will not execute. Use domain-based rules instead.
  2. Java applications: Java uses JKS/PKCS12 truststores and does not read PEM files via environment variables. Manual keytool -import is required.
  3. Custom CA loss: The injected SSL_CERT_FILE etc. override the image's default CA store. Custom CAs baked into the image are lost until Phase 3 CASecretRef.
  4. SecretRef rotation: Updating a referenced user Secret in-place does not automatically propagate to Envoy. Rotate by creating a new Secret and updating the policy's secretRef.name to trigger reconcile and hot reload.
  5. No ConfigMap auto-cleanup: Old varmor-* ConfigMaps from Phase 1 must be manually deleted after upgrade.

Future Work (Phase 3)

  • CASecretRef: Allow users to supply their own CA for MITM signing.
  • passthroughDomains: Explicitly declare domains that cannot be MITM'd (e.g., due to certificate pinning) and accept the Domain-Fronting risk.
  • Watch referenced Secrets and auto-reconcile on changes.
  • Java JKS keystore support.
  • Global sidecar resource defaults via varmor-config ConfigMap.

Comment thread internal/networkproxy/mitm/bundle.go Fixed
@Danny-Wei Danny-Wei changed the title Tls mitm and key injection feat(networkproxy): Add TLS MITM, automatic CA management, header injection, and IPv6 support Apr 28, 2026
Add an overflow check before make() to resolve CodeQL
go/allocation-size-overflow warning.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Comment thread internal/networkproxy/mitm/bundle.go Fixed
Comment thread internal/networkproxy/mitm/bundle.go Fixed
Comment thread internal/networkproxy/mitm/bundle.go Fixed
Remove the +1 from the make() capacity expression; the overflow check
for mozLen+caLen is sufficient. The final append may reallocate once
if a separator newline is inserted, which is negligible for this data
size and eliminates the CodeQL go/allocation-size-overflow alert.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@Danny-Wei Danny-Wei force-pushed the tls-mitm-and-key-injection branch from 63ea236 to 122bafa Compare April 28, 2026 07:25
Add MITM-aware default resource values and a CRD field for per-policy
override, so users can fine-tune proxy sidecar CPU/memory without
modifying global configuration.

Key changes:

CRD & Types:
- Add ProxyResourceOverride type with Requests/Limits fields
- Add NetworkProxyConfig.Resources field (json:"resources")
- Built-in defaults: non-MITM 50m/64Mi→500m/256Mi,
  MITM 100m/128Mi→1000m/512Mi

Resource computation (internal/policy/proxy_resources.go):
- DefaultProxyResources(mitmEnabled) returns built-in defaults
- ResolveProxyResources(override, mitmEnabled) merges user overrides
  with defaults (field-level merge, unset fields retain defaults)
- MarshalProxyResourcesJSON() for webhook sidecar injection

Webhook & Controller integration:
- mutation.go: replace hardcoded resource JSON with dynamic computation
- update.go: remove hardcoded Resources from proxyContainer template,
  call ResolveProxyResources() for Deployment/StatefulSet/DaemonSet
- validate.go: add validateProxyResources() ensuring limits >= requests

Immutable proxy field enforcement:
- ValidateUpdatePolicy() gains oldProxyConfig parameter to block
  modifications to ProxyUID/ProxyPort/ProxyAdminPort after creation
- Webhook passes real old ProxyConfig (nil→zero-value for "first set"
  edge case); controller passes nil to skip check
- Add ptrInt64Equal/ptrUint16Equal helpers for nil-safe comparison

Policycacher selective update:
- updateProxyConfigMutableFields() refreshes only MITM and Resources
  in cache, preserving immutable ProxyUID/Port/AdminPort from the
  add-time snapshot
Comment thread internal/webhooks/mutation.go Dismissed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants