Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 19 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,13 @@ What is already in place:
- frozen node-centric gRPC and async contracts
- contract CI with `buf breaking`, AsyncAPI checks, and boundary naming policy
- container-backed integration tests
- full kernel journey end-to-end coverage:
- projection -> query -> compatibility -> command -> admin
- full TLS variant across gRPC, NATS, Valkey, and Neo4j in the test harness
- agentic end-to-end proofs:
- pull-driven runtime flow
- pull-driven runtime flow against a narrow runtime contract shape
- event-driven runtime trigger flow
- cluster-verifiable starship journey demo for a production-like graph case
- runtime integration reference docs and runnable client example

What is intentionally out of scope for this repo:
Expand Down Expand Up @@ -122,8 +126,13 @@ bash scripts/ci/integration-nats-compatibility.sh
bash scripts/ci/integration-grpc-compatibility.sh
bash scripts/ci/integration-agentic-context.sh
bash scripts/ci/integration-agentic-event-context.sh
bash scripts/ci/integration-kernel-full-journey.sh
bash scripts/ci/integration-kernel-full-journey-tls.sh
```

The `agentic_*` suites prove a reusable runtime integration shape. The strongest
kernel-owned end-to-end paths are the `kernel-full-journey*` suites above.

For deployed kernels, the generic projection runtime is enabled separately from
legacy compatibility NATS and persists its own state in Valkey through
`REHYDRATION_RUNTIME_STATE_URI`.
Expand Down Expand Up @@ -183,7 +192,7 @@ See:
The kernel now owns a standalone OCI image intended for external download and
evaluation.

Planned public location:
Public location:

- `ghcr.io/underpass-ai/rehydration-kernel`

Expand All @@ -199,13 +208,20 @@ for environment variables, tags, and usage.
Helm chart:

- source chart: [`charts/rehydration-kernel`](./charts/rehydration-kernel)
- planned OCI location: `oci://ghcr.io/underpass-ai/charts/rehydration-kernel`
- OCI location: `oci://ghcr.io/underpass-ai/charts/rehydration-kernel`

The default chart values are intentionally secure:

- no implicit `latest` image tag
- no inline backend URIs by default
- production-style installs should use `image.digest` or a pinned tag plus `secrets.existingSecret`
- optional `ingress.enabled` can expose the gRPC service through a controller-managed ingress
- optional `neo4jTls.*` can mount a custom Neo4j CA for secure `graphUri` values

The sibling-runtime deployment profiles are:

- [`charts/rehydration-kernel/values.underpass-runtime.yaml`](./charts/rehydration-kernel/values.underpass-runtime.yaml) for the current cluster wiring, including the NGINX gRPC ingress host `rehydration-kernel.underpassai.com`
- [`charts/rehydration-kernel/values.underpass-runtime.secure.example.yaml`](./charts/rehydration-kernel/values.underpass-runtime.secure.example.yaml) for the staged Neo4j TLS target once the shared graph service publishes a CA-backed TLS endpoint

For local evaluation only, use [`values.dev.yaml`](./charts/rehydration-kernel/values.dev.yaml).

Expand Down
22 changes: 18 additions & 4 deletions charts/rehydration-kernel/templates/NOTES.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,21 +2,35 @@

kubectl get svc {{ include "rehydration-kernel.fullname" . }}

2. Provide an immutable image reference for non-development installs:
{{- if .Values.ingress.enabled }}
2. The chart also rendered an Ingress for external gRPC access:

kubectl get ingress {{ include "rehydration-kernel.fullname" . }}

{{- else }}
2. If you need controller-managed external access, enable `ingress.enabled` and
provide gRPC-compatible controller annotations such as an NGINX
`backend-protocol: GRPC` annotation.
{{- end }}

3. Provide an immutable image reference for non-development installs:

- set `image.digest`
- or set `image.tag` to a non-`latest` value

3. Provide connection settings through a secret for non-development installs:
4. Provide connection settings through a secret for non-development installs:

- set `secrets.existingSecret`
- keys default to `graphUri`, `detailUri`, `snapshotUri`, `runtimeStateUri`, and `natsUrl`
- if your `graphUri` uses Neo4j private trust, mount `neo4jTls.existingSecret`
and include the final `tls_ca_path` in the secret-backed URI

4. Development-only installs may override this through:
5. Development-only installs may override this through:

- `development.allowMutableImageTags=true`
- `development.allowInlineConnections=true`
- values under `connections`
- optional `neo4jTls.*`, `natsTls.*`, and `valkeyTls.*` when testing secure transports

5. The chart deploys the standalone kernel service. Product adapters and
6. The chart deploys the standalone kernel service. Product adapters and
product-specific event bridges stay outside this chart.
45 changes: 45 additions & 0 deletions charts/rehydration-kernel/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,12 @@ app.kubernetes.io/instance: {{ .Release.Name }}
{{- $natsTlsCaKey := default "" .Values.natsTls.keys.ca -}}
{{- $natsTlsCertKey := default "" .Values.natsTls.keys.cert -}}
{{- $natsTlsKeyKey := default "" .Values.natsTls.keys.key -}}
{{- $ingressEnabled := default false .Values.ingress.enabled -}}
{{- $ingressHosts := default (list) .Values.ingress.hosts -}}
{{- $neo4jTlsEnabled := default false .Values.neo4jTls.enabled -}}
{{- $neo4jTlsSecret := default "" .Values.neo4jTls.existingSecret -}}
{{- $neo4jTlsMountPath := default "" .Values.neo4jTls.mountPath -}}
{{- $neo4jTlsCaKey := default "" .Values.neo4jTls.keys.ca -}}
{{- $valkeyTlsEnabled := default false .Values.valkeyTls.enabled -}}
{{- $valkeyTlsSecret := default "" .Values.valkeyTls.existingSecret -}}
{{- $valkeyTlsMountPath := default "" .Values.valkeyTls.mountPath -}}
Expand All @@ -73,6 +79,18 @@ app.kubernetes.io/instance: {{ .Release.Name }}
{{- if and (eq (default "" .Values.secrets.existingSecret) "") (not $allowInlineConnections) -}}
{{- fail "set secrets.existingSecret for connection URIs or explicitly enable development.allowInlineConnections=true" -}}
{{- end -}}
{{- if and $ingressEnabled (eq (len $ingressHosts) 0) -}}
{{- fail "ingress.hosts must contain at least one host when ingress.enabled=true" -}}
{{- end -}}
{{- if and $neo4jTlsEnabled (eq $neo4jTlsSecret "") (ne $neo4jTlsCaKey "") -}}
{{- fail "neo4jTls.existingSecret is required when neo4jTls.keys.ca is configured" -}}
{{- end -}}
{{- if and (ne $neo4jTlsSecret "") (eq $neo4jTlsMountPath "") -}}
{{- fail "neo4jTls.mountPath is required when neo4jTls.existingSecret is set" -}}
{{- end -}}
{{- if and $neo4jTlsEnabled (eq $neo4jTlsCaKey "") -}}
{{- fail "neo4jTls.keys.ca is required when neo4jTls.enabled=true" -}}
{{- end -}}
{{- if ne $grpcTlsMode "disabled" -}}
{{- if eq (default "" .Values.tls.existingSecret) "" -}}
{{- fail "tls.existingSecret is required when tls.mode is server or mutual" -}}
Expand Down Expand Up @@ -127,6 +145,11 @@ app.kubernetes.io/instance: {{ .Release.Name }}
{{- if eq (default "" .Values.connections.runtimeStateUri) "" -}}
{{- fail "connections.runtimeStateUri is required when development.allowInlineConnections=true" -}}
{{- end -}}
{{- if $neo4jTlsEnabled -}}
{{- if not (or (hasPrefix "bolt+s://" .Values.connections.graphUri) (hasPrefix "bolt+ssc://" .Values.connections.graphUri) (hasPrefix "neo4j+s://" .Values.connections.graphUri) (hasPrefix "neo4j+ssc://" .Values.connections.graphUri)) -}}
{{- fail "neo4jTls.enabled requires connections.graphUri to use bolt+s://, bolt+ssc://, neo4j+s://, or neo4j+ssc:// when development.allowInlineConnections=true" -}}
{{- end -}}
{{- end -}}
{{- if $valkeyTlsEnabled -}}
{{- range $connection := list .Values.connections.detailUri .Values.connections.snapshotUri .Values.connections.runtimeStateUri -}}
{{- if not (or (hasPrefix "redis://" $connection) (hasPrefix "valkey://" $connection) (hasPrefix "rediss://" $connection) (hasPrefix "valkeys://" $connection)) -}}
Expand Down Expand Up @@ -169,6 +192,28 @@ app.kubernetes.io/instance: {{ .Release.Name }}
{{- end -}}
{{- end -}}

{{- define "rehydration-kernel.inlineGraphUri" -}}
{{- $uri := .uri -}}
{{- $tls := .tls -}}
{{- if not $tls.enabled -}}
{{- $uri -}}
{{- else -}}
{{- $params := list -}}
{{- if and (ne (default "" $tls.existingSecret) "") (ne (default "" $tls.keys.ca) "") -}}
{{- $params = append $params (printf "tls_ca_path=%s/%s" $tls.mountPath $tls.keys.ca) -}}
{{- end -}}
{{- if gt (len $params) 0 -}}
{{- if contains "?" $uri -}}
{{- printf "%s&%s" $uri (join "&" $params) -}}
{{- else -}}
{{- printf "%s?%s" $uri (join "&" $params) -}}
{{- end -}}
{{- else -}}
{{- $uri -}}
{{- end -}}
{{- end -}}
{{- end -}}

{{- define "rehydration-kernel.image" -}}
{{- $repository := .Values.image.repository -}}
{{- $tag := default "" .Values.image.tag -}}
Expand Down
16 changes: 13 additions & 3 deletions charts/rehydration-kernel/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ spec:
key: {{ .Values.secrets.keys.natsUrl | quote }}
{{- else if .Values.development.allowInlineConnections }}
- name: REHYDRATION_GRAPH_URI
value: {{ .Values.connections.graphUri | quote }}
value: {{ include "rehydration-kernel.inlineGraphUri" (dict "uri" .Values.connections.graphUri "tls" .Values.neo4jTls) | quote }}
- name: REHYDRATION_DETAIL_URI
value: {{ include "rehydration-kernel.inlineValkeyUri" (dict "uri" .Values.connections.detailUri "tls" .Values.valkeyTls) | quote }}
- name: REHYDRATION_SNAPSHOT_URI
Expand All @@ -123,7 +123,7 @@ spec:
{{- with .Values.extraEnv }}
{{- toYaml . | nindent 12 }}
{{- end }}
{{- if or (ne .Values.tls.mode "disabled") .Values.natsTls.existingSecret .Values.valkeyTls.existingSecret }}
{{- if or (ne .Values.tls.mode "disabled") .Values.natsTls.existingSecret .Values.valkeyTls.existingSecret .Values.neo4jTls.existingSecret }}
volumeMounts:
{{- if ne .Values.tls.mode "disabled" }}
- name: grpc-tls
Expand All @@ -140,10 +140,15 @@ spec:
mountPath: {{ .Values.valkeyTls.mountPath | quote }}
readOnly: true
{{- end }}
{{- if .Values.neo4jTls.existingSecret }}
- name: neo4j-tls
mountPath: {{ .Values.neo4jTls.mountPath | quote }}
readOnly: true
{{- end }}
{{- end }}
resources:
{{- toYaml .Values.resources | nindent 12 }}
{{- if or (ne .Values.tls.mode "disabled") .Values.natsTls.existingSecret .Values.valkeyTls.existingSecret }}
{{- if or (ne .Values.tls.mode "disabled") .Values.natsTls.existingSecret .Values.valkeyTls.existingSecret .Values.neo4jTls.existingSecret }}
volumes:
{{- if ne .Values.tls.mode "disabled" }}
- name: grpc-tls
Expand All @@ -160,6 +165,11 @@ spec:
secret:
secretName: {{ .Values.valkeyTls.existingSecret | quote }}
{{- end }}
{{- if .Values.neo4jTls.existingSecret }}
- name: neo4j-tls
secret:
secretName: {{ .Values.neo4jTls.existingSecret | quote }}
{{- end }}
{{- end }}
{{- with .Values.nodeSelector }}
nodeSelector:
Expand Down
35 changes: 35 additions & 0 deletions charts/rehydration-kernel/templates/ingress.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{{- if .Values.ingress.enabled }}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ include "rehydration-kernel.fullname" . }}
labels:
{{- include "rehydration-kernel.labels" . | nindent 4 }}
{{- with .Values.ingress.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
{{- with .Values.ingress.className }}
ingressClassName: {{ . | quote }}
{{- end }}
rules:
{{- range .Values.ingress.hosts }}
- host: {{ .host | quote }}
http:
paths:
{{- range .paths }}
- path: {{ .path | quote }}
pathType: {{ .pathType | quote }}
backend:
service:
name: {{ include "rehydration-kernel.fullname" $ }}
port:
number: {{ $.Values.service.grpcPort }}
{{- end }}
{{- end }}
{{- with .Values.ingress.tls }}
tls:
{{- toYaml . | nindent 4 }}
{{- end }}
{{- end }}
4 changes: 4 additions & 0 deletions charts/rehydration-kernel/templates/service.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ metadata:
name: {{ include "rehydration-kernel.fullname" . }}
labels:
{{- include "rehydration-kernel.labels" . | nindent 4 }}
{{- with .Values.service.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
type: {{ .Values.service.type }}
selector:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Secure sibling-runtime profile once the shared Neo4j endpoint exposes TLS and
# a namespace-local CA secret has been provisioned for the kernel release.
image:
pullPolicy: Always

ingress:
enabled: true
className: nginx
annotations:
nginx.ingress.kubernetes.io/backend-protocol: GRPC
hosts:
- host: rehydration-kernel.underpassai.com
paths:
- path: /
pathType: Prefix
tls:
- hosts:
- rehydration-kernel.underpassai.com
secretName: rehydration-kernel-ingress-tls

neo4jTls:
enabled: true
existingSecret: rehydration-kernel-neo4j-tls
keys:
ca: ca.crt

config:
enableNats: false
enableProjectionNats: true

connections:
graphUri: neo4j+s://neo4j:underpassai@neo4j.swe-ai-fleet.svc.cluster.local:7687
detailUri: redis://valkey.underpass-runtime.svc.cluster.local:6379
snapshotUri: redis://valkey.underpass-runtime.svc.cluster.local:6379
runtimeStateUri: redis://valkey.underpass-runtime.svc.cluster.local:6379
natsUrl: nats://nats.underpass-runtime.svc.cluster.local:4222

development:
allowInlineConnections: true
11 changes: 11 additions & 0 deletions charts/rehydration-kernel/values.underpass-runtime.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,17 @@
image:
pullPolicy: Always

ingress:
enabled: true
className: nginx
annotations:
nginx.ingress.kubernetes.io/backend-protocol: GRPC
hosts:
- host: rehydration-kernel.underpassai.com
paths:
- path: /
pathType: Prefix

config:
enableNats: false
enableProjectionNats: true
Expand Down
15 changes: 15 additions & 0 deletions charts/rehydration-kernel/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,14 @@ securityContext:
service:
type: ClusterIP
grpcPort: 50054
annotations: {}

ingress:
enabled: false
className: ""
annotations: {}
hosts: []
tls: []

tls:
mode: disabled
Expand Down Expand Up @@ -64,6 +72,13 @@ valkeyTls:
cert: ""
key: ""

neo4jTls:
enabled: false
existingSecret: ""
mountPath: /var/run/rehydration-kernel/neo4j-tls
keys:
ca: ""

config:
serviceName: rehydration-kernel
grpcBind: 0.0.0.0:50054
Expand Down
2 changes: 1 addition & 1 deletion docs/migration/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Phase 0 status:
- runtime integration reference for external consumers: complete
- runnable runtime reference client outside tests: complete
- LLM response determinism strategy: planned and documented
- transport security v1: planned and documented
- transport security v1: implemented for gRPC, outbound NATS, outbound Valkey, and Neo4j CA wiring; Neo4j client identity remains open
- repo closeout and handoff to integrating products: complete
- shadow mode specification for `swe-ai-fleet`: complete as documentation
- deferred kernel maintenance milestone: consolidate the integration harness
Expand Down
11 changes: 7 additions & 4 deletions docs/migration/context-service-rust-roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ Reference:

## Strategic Security Milestone

Status: `planned`
Status: `mostly_complete`

Title:

Expand All @@ -134,9 +134,12 @@ Title:
Why it exists:

- the kernel is already strong in contract, CI, packaging, and deployment, but
still assumes plaintext trusted-network transport
- production-grade standalone deployment needs first-class TLS and mTLS support
for gRPC
still needed first-class transport hardening for standalone deployment
- inbound gRPC TLS and mTLS are now delivered
- outbound NATS TLS and outbound Valkey TLS are now delivered
- Neo4j custom CA wiring in Helm is now delivered
- the remaining transport gap is any future Neo4j client identity plus admin
hardening if those become necessary

Reference:

Expand Down
4 changes: 4 additions & 0 deletions docs/migration/kernel-repo-closeout.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,10 @@ Its non-responsibilities are:

- container-backed gRPC compatibility tests
- container-backed generic NATS tests
- full kernel journey E2E across projection, query, compatibility, command, and
admin
- full TLS kernel journey E2E across gRPC, NATS, Valkey, and Neo4j in the test
harness
- agentic end-to-end proof using runtime tool execution
- event-driven agentic end-to-end proof triggered from
`context.bundle.generated`
Expand Down
Loading
Loading