diff --git a/README.md b/README.md index 833835c..8fa0dc3 100644 --- a/README.md +++ b/README.md @@ -26,9 +26,13 @@ What is already in place: - frozen node-centric gRPC and async contracts - contract CI with `buf breaking`, AsyncAPI checks, and boundary naming policy - container-backed integration tests +- full kernel journey end-to-end coverage: + - projection -> query -> compatibility -> command -> admin + - full TLS variant across gRPC, NATS, Valkey, and Neo4j in the test harness - agentic end-to-end proofs: - - pull-driven runtime flow + - pull-driven runtime flow against a narrow runtime contract shape - event-driven runtime trigger flow +- cluster-verifiable starship journey demo for a production-like graph case - runtime integration reference docs and runnable client example What is intentionally out of scope for this repo: @@ -122,8 +126,13 @@ bash scripts/ci/integration-nats-compatibility.sh bash scripts/ci/integration-grpc-compatibility.sh bash scripts/ci/integration-agentic-context.sh bash scripts/ci/integration-agentic-event-context.sh +bash scripts/ci/integration-kernel-full-journey.sh +bash scripts/ci/integration-kernel-full-journey-tls.sh ``` +The `agentic_*` suites prove a reusable runtime integration shape. The strongest +kernel-owned end-to-end paths are the `kernel-full-journey*` suites above. + For deployed kernels, the generic projection runtime is enabled separately from legacy compatibility NATS and persists its own state in Valkey through `REHYDRATION_RUNTIME_STATE_URI`. @@ -183,7 +192,7 @@ See: The kernel now owns a standalone OCI image intended for external download and evaluation. -Planned public location: +Public location: - `ghcr.io/underpass-ai/rehydration-kernel` @@ -199,13 +208,20 @@ for environment variables, tags, and usage. Helm chart: - source chart: [`charts/rehydration-kernel`](./charts/rehydration-kernel) -- planned OCI location: `oci://ghcr.io/underpass-ai/charts/rehydration-kernel` +- OCI location: `oci://ghcr.io/underpass-ai/charts/rehydration-kernel` The default chart values are intentionally secure: - no implicit `latest` image tag - no inline backend URIs by default - production-style installs should use `image.digest` or a pinned tag plus `secrets.existingSecret` +- optional `ingress.enabled` can expose the gRPC service through a controller-managed ingress +- optional `neo4jTls.*` can mount a custom Neo4j CA for secure `graphUri` values + +The sibling-runtime deployment profiles are: + +- [`charts/rehydration-kernel/values.underpass-runtime.yaml`](./charts/rehydration-kernel/values.underpass-runtime.yaml) for the current cluster wiring, including the NGINX gRPC ingress host `rehydration-kernel.underpassai.com` +- [`charts/rehydration-kernel/values.underpass-runtime.secure.example.yaml`](./charts/rehydration-kernel/values.underpass-runtime.secure.example.yaml) for the staged Neo4j TLS target once the shared graph service publishes a CA-backed TLS endpoint For local evaluation only, use [`values.dev.yaml`](./charts/rehydration-kernel/values.dev.yaml). diff --git a/charts/rehydration-kernel/templates/NOTES.txt b/charts/rehydration-kernel/templates/NOTES.txt index a334c14..ac00e06 100644 --- a/charts/rehydration-kernel/templates/NOTES.txt +++ b/charts/rehydration-kernel/templates/NOTES.txt @@ -2,21 +2,35 @@ kubectl get svc {{ include "rehydration-kernel.fullname" . }} -2. Provide an immutable image reference for non-development installs: +{{- if .Values.ingress.enabled }} +2. The chart also rendered an Ingress for external gRPC access: + + kubectl get ingress {{ include "rehydration-kernel.fullname" . }} + +{{- else }} +2. If you need controller-managed external access, enable `ingress.enabled` and + provide gRPC-compatible controller annotations such as an NGINX + `backend-protocol: GRPC` annotation. +{{- end }} + +3. Provide an immutable image reference for non-development installs: - set `image.digest` - or set `image.tag` to a non-`latest` value -3. Provide connection settings through a secret for non-development installs: +4. Provide connection settings through a secret for non-development installs: - set `secrets.existingSecret` - keys default to `graphUri`, `detailUri`, `snapshotUri`, `runtimeStateUri`, and `natsUrl` + - if your `graphUri` uses Neo4j private trust, mount `neo4jTls.existingSecret` + and include the final `tls_ca_path` in the secret-backed URI -4. Development-only installs may override this through: +5. Development-only installs may override this through: - `development.allowMutableImageTags=true` - `development.allowInlineConnections=true` - values under `connections` + - optional `neo4jTls.*`, `natsTls.*`, and `valkeyTls.*` when testing secure transports -5. The chart deploys the standalone kernel service. Product adapters and +6. The chart deploys the standalone kernel service. Product adapters and product-specific event bridges stay outside this chart. diff --git a/charts/rehydration-kernel/templates/_helpers.tpl b/charts/rehydration-kernel/templates/_helpers.tpl index c8fc94b..666ecca 100644 --- a/charts/rehydration-kernel/templates/_helpers.tpl +++ b/charts/rehydration-kernel/templates/_helpers.tpl @@ -52,6 +52,12 @@ app.kubernetes.io/instance: {{ .Release.Name }} {{- $natsTlsCaKey := default "" .Values.natsTls.keys.ca -}} {{- $natsTlsCertKey := default "" .Values.natsTls.keys.cert -}} {{- $natsTlsKeyKey := default "" .Values.natsTls.keys.key -}} +{{- $ingressEnabled := default false .Values.ingress.enabled -}} +{{- $ingressHosts := default (list) .Values.ingress.hosts -}} +{{- $neo4jTlsEnabled := default false .Values.neo4jTls.enabled -}} +{{- $neo4jTlsSecret := default "" .Values.neo4jTls.existingSecret -}} +{{- $neo4jTlsMountPath := default "" .Values.neo4jTls.mountPath -}} +{{- $neo4jTlsCaKey := default "" .Values.neo4jTls.keys.ca -}} {{- $valkeyTlsEnabled := default false .Values.valkeyTls.enabled -}} {{- $valkeyTlsSecret := default "" .Values.valkeyTls.existingSecret -}} {{- $valkeyTlsMountPath := default "" .Values.valkeyTls.mountPath -}} @@ -73,6 +79,18 @@ app.kubernetes.io/instance: {{ .Release.Name }} {{- if and (eq (default "" .Values.secrets.existingSecret) "") (not $allowInlineConnections) -}} {{- fail "set secrets.existingSecret for connection URIs or explicitly enable development.allowInlineConnections=true" -}} {{- end -}} +{{- if and $ingressEnabled (eq (len $ingressHosts) 0) -}} +{{- fail "ingress.hosts must contain at least one host when ingress.enabled=true" -}} +{{- end -}} +{{- if and $neo4jTlsEnabled (eq $neo4jTlsSecret "") (ne $neo4jTlsCaKey "") -}} +{{- fail "neo4jTls.existingSecret is required when neo4jTls.keys.ca is configured" -}} +{{- end -}} +{{- if and (ne $neo4jTlsSecret "") (eq $neo4jTlsMountPath "") -}} +{{- fail "neo4jTls.mountPath is required when neo4jTls.existingSecret is set" -}} +{{- end -}} +{{- if and $neo4jTlsEnabled (eq $neo4jTlsCaKey "") -}} +{{- fail "neo4jTls.keys.ca is required when neo4jTls.enabled=true" -}} +{{- end -}} {{- if ne $grpcTlsMode "disabled" -}} {{- if eq (default "" .Values.tls.existingSecret) "" -}} {{- fail "tls.existingSecret is required when tls.mode is server or mutual" -}} @@ -127,6 +145,11 @@ app.kubernetes.io/instance: {{ .Release.Name }} {{- if eq (default "" .Values.connections.runtimeStateUri) "" -}} {{- fail "connections.runtimeStateUri is required when development.allowInlineConnections=true" -}} {{- end -}} +{{- if $neo4jTlsEnabled -}} +{{- if not (or (hasPrefix "bolt+s://" .Values.connections.graphUri) (hasPrefix "bolt+ssc://" .Values.connections.graphUri) (hasPrefix "neo4j+s://" .Values.connections.graphUri) (hasPrefix "neo4j+ssc://" .Values.connections.graphUri)) -}} +{{- fail "neo4jTls.enabled requires connections.graphUri to use bolt+s://, bolt+ssc://, neo4j+s://, or neo4j+ssc:// when development.allowInlineConnections=true" -}} +{{- end -}} +{{- end -}} {{- if $valkeyTlsEnabled -}} {{- range $connection := list .Values.connections.detailUri .Values.connections.snapshotUri .Values.connections.runtimeStateUri -}} {{- if not (or (hasPrefix "redis://" $connection) (hasPrefix "valkey://" $connection) (hasPrefix "rediss://" $connection) (hasPrefix "valkeys://" $connection)) -}} @@ -169,6 +192,28 @@ app.kubernetes.io/instance: {{ .Release.Name }} {{- end -}} {{- end -}} +{{- define "rehydration-kernel.inlineGraphUri" -}} +{{- $uri := .uri -}} +{{- $tls := .tls -}} +{{- if not $tls.enabled -}} +{{- $uri -}} +{{- else -}} +{{- $params := list -}} +{{- if and (ne (default "" $tls.existingSecret) "") (ne (default "" $tls.keys.ca) "") -}} +{{- $params = append $params (printf "tls_ca_path=%s/%s" $tls.mountPath $tls.keys.ca) -}} +{{- end -}} +{{- if gt (len $params) 0 -}} +{{- if contains "?" $uri -}} +{{- printf "%s&%s" $uri (join "&" $params) -}} +{{- else -}} +{{- printf "%s?%s" $uri (join "&" $params) -}} +{{- end -}} +{{- else -}} +{{- $uri -}} +{{- end -}} +{{- end -}} +{{- end -}} + {{- define "rehydration-kernel.image" -}} {{- $repository := .Values.image.repository -}} {{- $tag := default "" .Values.image.tag -}} diff --git a/charts/rehydration-kernel/templates/deployment.yaml b/charts/rehydration-kernel/templates/deployment.yaml index 1080622..8753bc1 100644 --- a/charts/rehydration-kernel/templates/deployment.yaml +++ b/charts/rehydration-kernel/templates/deployment.yaml @@ -110,7 +110,7 @@ spec: key: {{ .Values.secrets.keys.natsUrl | quote }} {{- else if .Values.development.allowInlineConnections }} - name: REHYDRATION_GRAPH_URI - value: {{ .Values.connections.graphUri | quote }} + value: {{ include "rehydration-kernel.inlineGraphUri" (dict "uri" .Values.connections.graphUri "tls" .Values.neo4jTls) | quote }} - name: REHYDRATION_DETAIL_URI value: {{ include "rehydration-kernel.inlineValkeyUri" (dict "uri" .Values.connections.detailUri "tls" .Values.valkeyTls) | quote }} - name: REHYDRATION_SNAPSHOT_URI @@ -123,7 +123,7 @@ spec: {{- with .Values.extraEnv }} {{- toYaml . | nindent 12 }} {{- end }} - {{- if or (ne .Values.tls.mode "disabled") .Values.natsTls.existingSecret .Values.valkeyTls.existingSecret }} + {{- if or (ne .Values.tls.mode "disabled") .Values.natsTls.existingSecret .Values.valkeyTls.existingSecret .Values.neo4jTls.existingSecret }} volumeMounts: {{- if ne .Values.tls.mode "disabled" }} - name: grpc-tls @@ -140,10 +140,15 @@ spec: mountPath: {{ .Values.valkeyTls.mountPath | quote }} readOnly: true {{- end }} + {{- if .Values.neo4jTls.existingSecret }} + - name: neo4j-tls + mountPath: {{ .Values.neo4jTls.mountPath | quote }} + readOnly: true + {{- end }} {{- end }} resources: {{- toYaml .Values.resources | nindent 12 }} - {{- if or (ne .Values.tls.mode "disabled") .Values.natsTls.existingSecret .Values.valkeyTls.existingSecret }} + {{- if or (ne .Values.tls.mode "disabled") .Values.natsTls.existingSecret .Values.valkeyTls.existingSecret .Values.neo4jTls.existingSecret }} volumes: {{- if ne .Values.tls.mode "disabled" }} - name: grpc-tls @@ -160,6 +165,11 @@ spec: secret: secretName: {{ .Values.valkeyTls.existingSecret | quote }} {{- end }} + {{- if .Values.neo4jTls.existingSecret }} + - name: neo4j-tls + secret: + secretName: {{ .Values.neo4jTls.existingSecret | quote }} + {{- end }} {{- end }} {{- with .Values.nodeSelector }} nodeSelector: diff --git a/charts/rehydration-kernel/templates/ingress.yaml b/charts/rehydration-kernel/templates/ingress.yaml new file mode 100644 index 0000000..4f93ea1 --- /dev/null +++ b/charts/rehydration-kernel/templates/ingress.yaml @@ -0,0 +1,35 @@ +{{- if .Values.ingress.enabled }} +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: {{ include "rehydration-kernel.fullname" . }} + labels: + {{- include "rehydration-kernel.labels" . | nindent 4 }} + {{- with .Values.ingress.annotations }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} +spec: + {{- with .Values.ingress.className }} + ingressClassName: {{ . | quote }} + {{- end }} + rules: + {{- range .Values.ingress.hosts }} + - host: {{ .host | quote }} + http: + paths: + {{- range .paths }} + - path: {{ .path | quote }} + pathType: {{ .pathType | quote }} + backend: + service: + name: {{ include "rehydration-kernel.fullname" $ }} + port: + number: {{ $.Values.service.grpcPort }} + {{- end }} + {{- end }} + {{- with .Values.ingress.tls }} + tls: + {{- toYaml . | nindent 4 }} + {{- end }} +{{- end }} diff --git a/charts/rehydration-kernel/templates/service.yaml b/charts/rehydration-kernel/templates/service.yaml index 64a2235..d895121 100644 --- a/charts/rehydration-kernel/templates/service.yaml +++ b/charts/rehydration-kernel/templates/service.yaml @@ -4,6 +4,10 @@ metadata: name: {{ include "rehydration-kernel.fullname" . }} labels: {{- include "rehydration-kernel.labels" . | nindent 4 }} + {{- with .Values.service.annotations }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} spec: type: {{ .Values.service.type }} selector: diff --git a/charts/rehydration-kernel/values.underpass-runtime.secure.example.yaml b/charts/rehydration-kernel/values.underpass-runtime.secure.example.yaml new file mode 100644 index 0000000..fb426a5 --- /dev/null +++ b/charts/rehydration-kernel/values.underpass-runtime.secure.example.yaml @@ -0,0 +1,39 @@ +# Secure sibling-runtime profile once the shared Neo4j endpoint exposes TLS and +# a namespace-local CA secret has been provisioned for the kernel release. +image: + pullPolicy: Always + +ingress: + enabled: true + className: nginx + annotations: + nginx.ingress.kubernetes.io/backend-protocol: GRPC + hosts: + - host: rehydration-kernel.underpassai.com + paths: + - path: / + pathType: Prefix + tls: + - hosts: + - rehydration-kernel.underpassai.com + secretName: rehydration-kernel-ingress-tls + +neo4jTls: + enabled: true + existingSecret: rehydration-kernel-neo4j-tls + keys: + ca: ca.crt + +config: + enableNats: false + enableProjectionNats: true + +connections: + graphUri: neo4j+s://neo4j:underpassai@neo4j.swe-ai-fleet.svc.cluster.local:7687 + detailUri: redis://valkey.underpass-runtime.svc.cluster.local:6379 + snapshotUri: redis://valkey.underpass-runtime.svc.cluster.local:6379 + runtimeStateUri: redis://valkey.underpass-runtime.svc.cluster.local:6379 + natsUrl: nats://nats.underpass-runtime.svc.cluster.local:4222 + +development: + allowInlineConnections: true diff --git a/charts/rehydration-kernel/values.underpass-runtime.yaml b/charts/rehydration-kernel/values.underpass-runtime.yaml index fd66e26..f407537 100644 --- a/charts/rehydration-kernel/values.underpass-runtime.yaml +++ b/charts/rehydration-kernel/values.underpass-runtime.yaml @@ -1,6 +1,17 @@ image: pullPolicy: Always +ingress: + enabled: true + className: nginx + annotations: + nginx.ingress.kubernetes.io/backend-protocol: GRPC + hosts: + - host: rehydration-kernel.underpassai.com + paths: + - path: / + pathType: Prefix + config: enableNats: false enableProjectionNats: true diff --git a/charts/rehydration-kernel/values.yaml b/charts/rehydration-kernel/values.yaml index e16b232..310220d 100644 --- a/charts/rehydration-kernel/values.yaml +++ b/charts/rehydration-kernel/values.yaml @@ -35,6 +35,14 @@ securityContext: service: type: ClusterIP grpcPort: 50054 + annotations: {} + +ingress: + enabled: false + className: "" + annotations: {} + hosts: [] + tls: [] tls: mode: disabled @@ -64,6 +72,13 @@ valkeyTls: cert: "" key: "" +neo4jTls: + enabled: false + existingSecret: "" + mountPath: /var/run/rehydration-kernel/neo4j-tls + keys: + ca: "" + config: serviceName: rehydration-kernel grpcBind: 0.0.0.0:50054 diff --git a/docs/migration/README.md b/docs/migration/README.md index d4ec214..8a1a9e2 100644 --- a/docs/migration/README.md +++ b/docs/migration/README.md @@ -36,7 +36,7 @@ Phase 0 status: - runtime integration reference for external consumers: complete - runnable runtime reference client outside tests: complete - LLM response determinism strategy: planned and documented -- transport security v1: planned and documented +- transport security v1: implemented for gRPC, outbound NATS, outbound Valkey, and Neo4j CA wiring; Neo4j client identity remains open - repo closeout and handoff to integrating products: complete - shadow mode specification for `swe-ai-fleet`: complete as documentation - deferred kernel maintenance milestone: consolidate the integration harness diff --git a/docs/migration/context-service-rust-roadmap.md b/docs/migration/context-service-rust-roadmap.md index 31f885e..ea4e31e 100644 --- a/docs/migration/context-service-rust-roadmap.md +++ b/docs/migration/context-service-rust-roadmap.md @@ -125,7 +125,7 @@ Reference: ## Strategic Security Milestone -Status: `planned` +Status: `mostly_complete` Title: @@ -134,9 +134,12 @@ Title: Why it exists: - the kernel is already strong in contract, CI, packaging, and deployment, but - still assumes plaintext trusted-network transport -- production-grade standalone deployment needs first-class TLS and mTLS support - for gRPC + still needed first-class transport hardening for standalone deployment +- inbound gRPC TLS and mTLS are now delivered +- outbound NATS TLS and outbound Valkey TLS are now delivered +- Neo4j custom CA wiring in Helm is now delivered +- the remaining transport gap is any future Neo4j client identity plus admin + hardening if those become necessary Reference: diff --git a/docs/migration/kernel-repo-closeout.md b/docs/migration/kernel-repo-closeout.md index 6bf59a2..e978478 100644 --- a/docs/migration/kernel-repo-closeout.md +++ b/docs/migration/kernel-repo-closeout.md @@ -77,6 +77,10 @@ Its non-responsibilities are: - container-backed gRPC compatibility tests - container-backed generic NATS tests +- full kernel journey E2E across projection, query, compatibility, command, and + admin +- full TLS kernel journey E2E across gRPC, NATS, Valkey, and Neo4j in the test + harness - agentic end-to-end proof using runtime tool execution - event-driven agentic end-to-end proof triggered from `context.bundle.generated` diff --git a/docs/migration/transport-security-v1.md b/docs/migration/transport-security-v1.md index 7c6cc0a..6e8f643 100644 --- a/docs/migration/transport-security-v1.md +++ b/docs/migration/transport-security-v1.md @@ -1,38 +1,36 @@ # Transport Security v1 -Status: `in_progress` +Status: `implemented_with_follow_up_gap` ## Purpose Define the first transport-security milestone for `rehydration-kernel`. -This milestone starts with the most important gap: +This milestone started with the most important gap: - inbound gRPC transport security for the standalone kernel -The goal is to make the kernel deployable in less-trusted cluster networks +The delivered result is a kernel that is deployable in less-trusted cluster +networks without forcing a service mesh or sidecar proxy. ## Why This Matters -Today the kernel is strong in: +The kernel is now strong in: - contract discipline - packaging - CI quality gates - Helm deployment -But it still assumes a trusted internal network at transport level. +The original transport gap is closed for the kernel-owned surfaces. Current limitations: -- gRPC server runs in plaintext -- no client certificate validation -- no server certificate configuration -- Helm chart has no first-class TLS or mTLS wiring +- the chart does not yet expose Neo4j client identity as a first-class feature +- HTTP admin transport is still a placeholder, not a hardened public surface -That is acceptable for internal evaluation, but not for a serious production -posture. +Those are narrower follow-up items, not the original blocker. ## Current State @@ -42,15 +40,15 @@ Current server implementation: - [`crates/rehydration-transport-grpc/src/transport/grpc_server.rs`](../../crates/rehydration-transport-grpc/src/transport/grpc_server.rs) -It uses: +It now uses: - `tonic::transport::Server::builder()` -It does not use: +It now supports: -- `ServerTlsConfig` -- server certificate loading -- client CA verification +- plaintext mode +- server TLS +- mutual TLS with client CA verification ### Configuration @@ -58,13 +56,13 @@ Current app config: - [`crates/rehydration-config/src/app_config.rs`](../../crates/rehydration-config/src/app_config.rs) -It exposes: +It now exposes: - `grpc_bind` - `admin_bind` - backend URIs -It does not expose: +It now includes: - TLS mode - cert path @@ -78,11 +76,17 @@ Current chart wiring: - [`charts/rehydration-kernel/templates/deployment.yaml`](../../charts/rehydration-kernel/templates/deployment.yaml) - [`charts/rehydration-kernel/values.yaml`](../../charts/rehydration-kernel/values.yaml) -It does not support: +It now supports: -- TLS secret mounts -- mTLS CA mounts +- TLS secret mounts for inbound gRPC +- mTLS CA mounts for inbound gRPC - transport mode selection +- outbound NATS TLS and outbound Valkey TLS wiring +- custom Neo4j CA wiring for secure inline `graphUri` values + +It does not yet support: + +- first-class Neo4j client identity wiring Current progress: @@ -93,6 +97,11 @@ Current progress: - outbound Valkey TLS and `rediss://` / `valkeys://` support are implemented - the current chart slice is wiring certificate mounts and operator-facing values for outbound NATS and Valkey TLS +- custom Neo4j CA wiring is implemented in the chart for secure inline + `graphUri` values +- cluster smoke coverage is implemented for gRPC TLS and mTLS plus outbound TLS +- container-backed full-journey TLS coverage is implemented across gRPC, NATS, + Valkey, and Neo4j ## Decision @@ -125,12 +134,12 @@ cache connections: - NATS client TLS and mTLS - Valkey TLS via `rediss://` and `valkeys://` -Out of scope for v1: +Remaining follow-up beyond the delivered v1 slice: - RBAC or authorization based on certificate identity - SPIFFE or SPIRE integration - automatic certificate rotation -- Valkey TLS +- first-class Neo4j client identity wiring - additional HTTP admin transport security Those belong to later milestones. diff --git a/docs/operations/container-image.md b/docs/operations/container-image.md index e2f6ce5..8199803 100644 --- a/docs/operations/container-image.md +++ b/docs/operations/container-image.md @@ -10,7 +10,7 @@ Registry: - `ghcr.io/underpass-ai/rehydration-kernel` -Expected tags: +Published tags: - `latest` on the default branch - `main` @@ -29,9 +29,10 @@ owned by sibling repos. - gRPC: `50054` -## Default Environment In The Image +## Default Runtime Configuration In The Container -The image sets container-oriented defaults: +Between image environment variables and server-side config defaults, the +container starts with: - `REHYDRATION_SERVICE_NAME=rehydration-kernel` - `REHYDRATION_GRPC_BIND=0.0.0.0:50054` @@ -50,7 +51,8 @@ override it when they want compatibility NATS flows enabled. `ENABLE_PROJECTION_NATS=true` is the generic kernel default. The projection runtime persists deduplication markers and checkpoints in Valkey under dedicated -prefixes. +prefixes, so standalone runs without a reachable NATS endpoint must override it +to `false`. The admin bind setting is carried in config for forward compatibility, but the standalone image currently exposes only the gRPC port. @@ -72,10 +74,11 @@ docker run --rm \ -e REHYDRATION_DETAIL_URI=redis://host.docker.internal:6379 \ -e REHYDRATION_SNAPSHOT_URI=redis://host.docker.internal:6379 \ -e REHYDRATION_RUNTIME_STATE_URI=redis://host.docker.internal:6379 \ + -e ENABLE_PROJECTION_NATS=false \ ghcr.io/underpass-ai/rehydration-kernel:latest ``` -To enable both the generic projection runtime and compatibility NATS flows: +To enable the generic projection runtime, provide a reachable NATS endpoint: ```bash docker run --rm \ @@ -127,6 +130,16 @@ Security posture of the chart: - inline `connections.*` are reserved for development-only overrides such as [`values.dev.yaml`](../../charts/rehydration-kernel/values.dev.yaml) +Current chart boundary: + +- inbound gRPC TLS and mTLS are first-class +- outbound NATS TLS and outbound Valkey TLS are first-class +- optional Kubernetes Ingress rendering is first-class for controller-managed + gRPC exposure +- Neo4j secure schemes plus custom CA mounts are first-class for inline + `graphUri` values +- Neo4j client identity is not yet a first-class chart feature + Local validation: ```bash diff --git a/docs/operations/kubernetes-deploy.md b/docs/operations/kubernetes-deploy.md index 8cd6b29..8fad8fc 100644 --- a/docs/operations/kubernetes-deploy.md +++ b/docs/operations/kubernetes-deploy.md @@ -47,7 +47,16 @@ The default values file is: - [`charts/rehydration-kernel/values.underpass-runtime.yaml`](../../charts/rehydration-kernel/values.underpass-runtime.yaml) -That keeps the deploy path aligned with the sibling runtime environment. +That keeps the deploy path aligned with the sibling runtime environment and now +includes the controller-managed gRPC ingress host currently reserved for the +kernel. + +The secure follow-up profile is: + +- [`charts/rehydration-kernel/values.underpass-runtime.secure.example.yaml`](../../charts/rehydration-kernel/values.underpass-runtime.secure.example.yaml) + +That file captures the intended sibling-runtime target once the shared Neo4j +deployment exposes TLS and the kernel namespace has the matching CA secret. ## Local Equivalent @@ -113,6 +122,47 @@ tls: existingSecret: rehydration-kernel-grpc-tls ``` +## Ingress Exposure + +The chart can optionally render a Kubernetes `Ingress` in front of the gRPC +service. + +The core values are: + +- `ingress.enabled` +- `ingress.className` +- `ingress.annotations` +- `ingress.hosts` +- `ingress.tls` + +Example for an NGINX ingress controller serving gRPC: + +```yaml +ingress: + enabled: true + className: nginx + annotations: + nginx.ingress.kubernetes.io/backend-protocol: GRPC + hosts: + - host: rehydration-kernel.example.com + paths: + - path: / + pathType: Prefix + tls: + - hosts: + - rehydration-kernel.example.com + secretName: rehydration-kernel-ingress-tls +``` + +Controller-specific annotations stay with the operator because gRPC ingress +behavior is controller-specific. + +The sibling runtime profile now enables this directly with: + +- host: `rehydration-kernel.underpassai.com` +- class: `nginx` +- annotation: `nginx.ingress.kubernetes.io/backend-protocol: GRPC` + ## Outbound NATS TLS The chart now exposes NATS client TLS directly: @@ -205,6 +255,51 @@ the secret-backed values. In that case, store the final `rediss://` or `valkeys://` URIs in the secret, including `tls_ca_path`, `tls_cert_path`, and `tls_key_path` query parameters that match the mounted paths. +## Neo4j TLS Today + +The Neo4j adapter supports secure schemes such as: + +- `bolt+s://` +- `bolt+ssc://` +- `neo4j+s://` +- `neo4j+ssc://` + +The chart now supports first-class mounting of a custom Neo4j CA for inline +`connections.graphUri` values. + +The shared cluster still serves Neo4j over plaintext `neo4j://` today, so the +default sibling-runtime values stay on the plaintext URI. The staged secure +target lives in +[`charts/rehydration-kernel/values.underpass-runtime.secure.example.yaml`](../../charts/rehydration-kernel/values.underpass-runtime.secure.example.yaml) +and switches the graph connection to `neo4j+s://...` plus `neo4jTls.*`. + +Example values override: + +```yaml +neo4jTls: + enabled: true + existingSecret: rehydration-kernel-neo4j-tls + keys: + ca: ca.crt + +connections: + graphUri: bolt+s://neo4j.example.internal:7687 +``` + +That renders `REHYDRATION_GRAPH_URI` with the matching `tls_ca_path` query +parameter that points at the mounted CA file. + +Current boundary: + +- publicly trusted Neo4j certificates can be handled through the base image + trust store plus a secure `graphUri` +- private-trust Neo4j deployments can use `neo4jTls.*` with inline + `connections.graphUri` +- secret-backed `graphUri` values still need the final secure URI, including + `tls_ca_path`, to be stored in the secret because Helm cannot rewrite secret + contents +- Neo4j client identity is still outside the current chart surface + ## Notes - `dry_run=true` uses `helm --dry-run=server` diff --git a/scripts/ci/helm-lint.sh b/scripts/ci/helm-lint.sh index 556e158..efb4343 100644 --- a/scripts/ci/helm-lint.sh +++ b/scripts/ci/helm-lint.sh @@ -3,6 +3,8 @@ set -euo pipefail CHART_PATH="${1:-charts/rehydration-kernel}" DEV_VALUES="${CHART_PATH}/values.dev.yaml" +UNDERPASS_RUNTIME_VALUES="${CHART_PATH}/values.underpass-runtime.yaml" +UNDERPASS_RUNTIME_SECURE_VALUES="${CHART_PATH}/values.underpass-runtime.secure.example.yaml" DEFAULT_ERR="${TMPDIR:-/tmp}/rehydration-kernel-helm-default.err" helm lint "${CHART_PATH}" -f "${DEV_VALUES}" @@ -11,6 +13,10 @@ helm template rehydration-kernel "${CHART_PATH}" -f "${DEV_VALUES}" >/tmp/rehydr SERVER_TLS_VALUES="${TMPDIR:-/tmp}/rehydration-kernel-helm-server-tls.yaml" MUTUAL_TLS_VALUES="${TMPDIR:-/tmp}/rehydration-kernel-helm-mutual-tls.yaml" OUTBOUND_TLS_VALUES="${TMPDIR:-/tmp}/rehydration-kernel-helm-outbound-tls.yaml" +INGRESS_VALUES="${TMPDIR:-/tmp}/rehydration-kernel-helm-ingress.yaml" +NEO4J_TLS_VALUES="${TMPDIR:-/tmp}/rehydration-kernel-helm-neo4j-tls.yaml" +SERVICE_ANNOTATIONS_VALUES="${TMPDIR:-/tmp}/rehydration-kernel-helm-service-annotations.yaml" +PINNED_IMAGE_VALUES="${TMPDIR:-/tmp}/rehydration-kernel-helm-pinned-image.yaml" cat >"${SERVER_TLS_VALUES}" <<'EOF' image: @@ -80,11 +86,98 @@ EOF helm template rehydration-kernel "${CHART_PATH}" -f "${OUTBOUND_TLS_VALUES}" >/tmp/rehydration-kernel-helm-outbound-tls-template.yaml +cat >"${INGRESS_VALUES}" <<'EOF' +image: + tag: latest +ingress: + enabled: true + className: nginx + annotations: + nginx.ingress.kubernetes.io/backend-protocol: GRPC + hosts: + - host: rehydration-kernel.example.com + paths: + - path: / + pathType: Prefix +connections: + graphUri: neo4j://neo4j:7687 + detailUri: redis://valkey:6379 + snapshotUri: redis://valkey:6379 + runtimeStateUri: redis://valkey:6379 + natsUrl: nats://nats:4222 +development: + allowMutableImageTags: true + allowInlineConnections: true +EOF + +helm template rehydration-kernel "${CHART_PATH}" -f "${INGRESS_VALUES}" >/tmp/rehydration-kernel-helm-ingress-template.yaml + +cat >"${NEO4J_TLS_VALUES}" <<'EOF' +image: + tag: latest +neo4jTls: + enabled: true + existingSecret: neo4j-ca + keys: + ca: ca.crt +connections: + graphUri: bolt+s://neo4j:7687 + detailUri: redis://valkey:6379 + snapshotUri: redis://valkey:6379 + runtimeStateUri: redis://valkey:6379 + natsUrl: nats://nats:4222 +development: + allowMutableImageTags: true + allowInlineConnections: true +EOF + +helm template rehydration-kernel "${CHART_PATH}" -f "${NEO4J_TLS_VALUES}" >/tmp/rehydration-kernel-helm-neo4j-tls-template.yaml + +cat >"${SERVICE_ANNOTATIONS_VALUES}" <<'EOF' +image: + tag: latest +service: + annotations: + service.beta.kubernetes.io/aws-load-balancer-scheme: internal +connections: + graphUri: neo4j://neo4j:7687 + detailUri: redis://valkey:6379 + snapshotUri: redis://valkey:6379 + runtimeStateUri: redis://valkey:6379 + natsUrl: nats://nats:4222 +development: + allowMutableImageTags: true + allowInlineConnections: true +EOF + +helm template rehydration-kernel "${CHART_PATH}" -f "${SERVICE_ANNOTATIONS_VALUES}" >/tmp/rehydration-kernel-helm-service-annotations-template.yaml + +cat >"${PINNED_IMAGE_VALUES}" <<'EOF' +image: + tag: latest +development: + allowMutableImageTags: true +EOF + +helm template rehydration-kernel "${CHART_PATH}" -f "${UNDERPASS_RUNTIME_VALUES}" -f "${PINNED_IMAGE_VALUES}" >/tmp/rehydration-kernel-helm-underpass-runtime-template.yaml +helm template rehydration-kernel "${CHART_PATH}" -f "${UNDERPASS_RUNTIME_SECURE_VALUES}" -f "${PINNED_IMAGE_VALUES}" >/tmp/rehydration-kernel-helm-underpass-runtime-secure-template.yaml + grep -q "NATS_TLS_MODE" /tmp/rehydration-kernel-helm-outbound-tls-template.yaml grep -q "NATS_TLS_CERT_PATH" /tmp/rehydration-kernel-helm-outbound-tls-template.yaml grep -q "rediss://valkey:6379?tls_ca_path=/var/run/rehydration-kernel/valkey-tls/ca.crt&tls_cert_path=/var/run/rehydration-kernel/valkey-tls/tls.crt&tls_key_path=/var/run/rehydration-kernel/valkey-tls/tls.key" /tmp/rehydration-kernel-helm-outbound-tls-template.yaml grep -q "name: nats-tls" /tmp/rehydration-kernel-helm-outbound-tls-template.yaml grep -q "name: valkey-tls" /tmp/rehydration-kernel-helm-outbound-tls-template.yaml +grep -q "kind: Ingress" /tmp/rehydration-kernel-helm-ingress-template.yaml +grep -q "nginx.ingress.kubernetes.io/backend-protocol: GRPC" /tmp/rehydration-kernel-helm-ingress-template.yaml +grep -q "host: \"rehydration-kernel.example.com\"" /tmp/rehydration-kernel-helm-ingress-template.yaml +grep -q "bolt+s://neo4j:7687?tls_ca_path=/var/run/rehydration-kernel/neo4j-tls/ca.crt" /tmp/rehydration-kernel-helm-neo4j-tls-template.yaml +grep -q "name: neo4j-tls" /tmp/rehydration-kernel-helm-neo4j-tls-template.yaml +grep -q "service.beta.kubernetes.io/aws-load-balancer-scheme: internal" /tmp/rehydration-kernel-helm-service-annotations-template.yaml +grep -q "host: \"rehydration-kernel.underpassai.com\"" /tmp/rehydration-kernel-helm-underpass-runtime-template.yaml +grep -q "nginx.ingress.kubernetes.io/backend-protocol: GRPC" /tmp/rehydration-kernel-helm-underpass-runtime-template.yaml +grep -q "neo4j+s://neo4j:underpassai@neo4j.swe-ai-fleet.svc.cluster.local:7687?tls_ca_path=/var/run/rehydration-kernel/neo4j-tls/ca.crt" /tmp/rehydration-kernel-helm-underpass-runtime-secure-template.yaml +grep -q "secretName: rehydration-kernel-ingress-tls" /tmp/rehydration-kernel-helm-underpass-runtime-secure-template.yaml +grep -q "secretName: \"rehydration-kernel-neo4j-tls\"" /tmp/rehydration-kernel-helm-underpass-runtime-secure-template.yaml if helm template rehydration-kernel "${CHART_PATH}" > /dev/null 2>"${DEFAULT_ERR}"; then echo "default chart render unexpectedly succeeded" >&2