Add shadow linking lab for Kubernetes disaster recovery

docs/modules/kubernetes/pages/shadow-linking.adoc

@@ -0,0 +1 @@
+../../../../kubernetes/shadow-linking/README.adoc

kubernetes/shadow-linking/.gitignore

@@ -0,0 +1 @@
+*.pid

kubernetes/shadow-linking/README.adoc

@@ -0,0 +1,218 @@
+= Explore Shadow Linking for Disaster Recovery
+:env-kubernetes: true
+:page-categories: Disaster Recovery, Data Replication
+:description: Deploy two Redpanda clusters on Kubernetes and configure shadow linking to continuously replicate topics, consumer group offsets, and Schema Registry data.
+:page-layout: lab
+
+Shadow linking is a built-in disaster recovery mechanism that continuously replicates topics, consumer group offsets, and Schema Registry data from a source cluster to a shadow cluster running on Kubernetes. This lab demonstrates how shadow linking works and how you can use it to fail over to a shadow cluster.
+
+In this lab, you deploy two single-node Redpanda clusters on Kubernetes and configure a shadow link between them. You then explore basic topic shadowing, Schema Registry shadowing, and consumer group failover.
+
+== Prerequisites
+
+You must have the following:
+
+* https://kubernetes.io/docs/tasks/tools/[kubectl^] CLI installed
+* https://helm.sh/docs/intro/install/[Helm 3+^] installed
+* xref:ROOT:get-started:rpk-install.adoc[rpk] CLI installed
+* https://kind.sigs.k8s.io/docs/user/quick-start/#installation[kind^] installed (for local testing)
+* https://docs.docker.com/get-docker/[Docker^] installed and running
+
+== What gets deployed
+
+The setup script installs the following components:
+
+* *cert-manager* (namespace: `cert-manager`): Manages TLS certificates
+* *Redpanda Operator* (namespace: `rp-operator`): Manages Redpanda cluster lifecycle
+* *Source cluster* (namespace: `source`): Primary cluster where you write data
+* *Shadow cluster* (namespace: `shadow`): Replication target cluster
+* *ShadowLink resource*: Defines the replication relationship between clusters
+
+The shadow link configuration mirrors all topics using wildcard filtering, replicates consumer group offsets, and synchronizes the Schema Registry, all at 5-second intervals.
+
+== Run the lab
+
+. Clone this repository:
++
+[,bash]
+----
+git clone https://github.com/redpanda-data/redpanda-labs.git
+cd redpanda-labs/kubernetes/shadow-linking
+----
+
+. Create a local Kubernetes cluster using kind:
++
+[,bash]
+----
+kind create cluster --name redpanda-shadow
+----
+
+. Run the setup script to deploy both clusters and configure shadow linking:
++
+[,bash]
+----
+./setup.sh
+----
++
+The script installs cert-manager, deploys the Redpanda Operator, creates the source and shadow clusters, configures the shadow link, sets up port forwarding for local access, and creates rpk profiles for both clusters.
+
+. Verify both clusters are healthy:
++
+[,bash]
+----
+rpk --profile source cluster health
+rpk --profile shadow cluster health
+----
+
+== Explore basic topic shadowing
+
+This section shows how topics and messages automatically replicate from the source to the shadow cluster.
+
+. Create a topic on the source cluster:
++
+[,bash]
+----
+rpk --profile source topic create basic -p1 -r1
+----
+
+. Produce a message to the source cluster:
++
+[,bash]
+----
+echo "Hello, world" | rpk --profile source topic produce basic
+----
+
+. Wait a few seconds for replication, then list topics on the shadow cluster:
++
+[,bash]
+----
+rpk --profile shadow topic list
+----
++
+You should see the `basic` topic appear on the shadow cluster.
+
+. Consume messages from both clusters to verify data replication:
++
+[,bash]
+----
+rpk --profile source topic consume basic -n1
+rpk --profile shadow topic consume basic -n1
+----
++
+The same message appears on both clusters.
+
+== Explore Schema Registry shadowing
+
+This section shows how Avro schemas registered on the source cluster replicate to the shadow cluster, allowing shadow-side message deserialization.
+
+. Register an Avro schema on the source cluster:
++
+[,bash]
+----
+rpk --profile source registry schema create syslog-value --schema resources/syslog.avsc --type avro
+----
+
+. Create a topic for schema-encoded messages:
++
+[,bash]
+----
+rpk --profile source topic create syslog -p1 -r1
+----
+
+. Produce Avro-encoded syslog records to the source cluster:
++
+[,bash]
+----
+cat resources/syslogs.json | rpk --profile source topic produce syslog --schema-id=topic
+----
+
+. Wait a few seconds, then list schemas on both clusters:
++
+[,bash]
+----
+rpk --profile source registry subject list
+rpk --profile shadow registry subject list
+----
++
+The `syslog-value` schema appears on both clusters.
+
+. Consume decoded messages from the shadow cluster using the replicated schema:
++
+[,bash]
+----
+rpk --profile shadow topic consume syslog -n1 --use-schema-registry -f'%v\n' | jq
+----
++
+The shadow cluster can deserialize messages using the replicated schema.
+
+== Explore consumer group failover
+
+This section shows how consumer group offsets replicate to the shadow cluster, allowing consumers to resume from their exact position after a failover.
+
+. Create a topic on the source cluster:
++
+[,bash]
+----
+rpk --profile source topic create foo -p1 -r1
+----
+
+. Produce records and consume them on the source cluster:
++
+[,bash]
+----
+seq 0 2 | rpk --profile source topic produce foo
+rpk --profile source topic consume foo -n3 -g consumer-group-foo
+----
++
+This creates a consumer group and processes the messages.
+
+. View consumer group offsets on both clusters:
++
+[,bash]
+----
+rpk --profile source group describe consumer-group-foo
+rpk --profile shadow group describe consumer-group-foo
+----
++
+The consumer group and its offsets are replicated to the shadow cluster.
+
+. Trigger a failover to the shadow cluster:
++
+[,bash]
+----
+rpk --profile shadow shadow failover shadow-link --all --no-confirm
+rpk --profile shadow shadow status shadow-link
+----
++
+This command promotes the shadow cluster to become writable.
+
+. Produce more records and consume on the shadow cluster:
++
+[,bash]
+----
+seq 3 5 | rpk --profile shadow topic produce foo
+rpk --profile shadow topic consume foo -n3 -g consumer-group-foo
+----
++
+The consumer resumes from the last committed offset (3) and consumes the new messages.
+
+== Clean up
+
+To stop port forwarding, terminate the background processes:
+
+[,bash]
+----
+kill $(cat local-port-forward.pid)
+----
+
+To delete the kind cluster and all resources:
+
+[,bash]
+----
+kind delete cluster --name redpanda-shadow
+----
+
+== Suggested reading
+
+* xref:ROOT:manage:disaster-recovery/shadowing/index.adoc[Shadowing for Disaster Recovery]
+* xref:ROOT:deploy:deployment-option/self-hosted/kubernetes/index.adoc[Redpanda on Kubernetes]

kubernetes/shadow-linking/config

@@ -0,0 +1,4 @@
+export CERT_MANAGER_NAMESPACE=cert-manager
+export OPERATOR_NAMESPACE=rp-operator
+export SOURCE_REDPANDA_NAMESPACE=source
+export SHADOW_REDPANDA_NAMESPACE=shadow

kubernetes/shadow-linking/resources/shadow-cluster.yaml

@@ -0,0 +1,43 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: shadow
+---
+apiVersion: cluster.redpanda.com/v1alpha2
+kind: Redpanda
+metadata:
+  name: redpanda
+  namespace: shadow
+spec:
+  clusterSpec:
+    image:
+      tag: v25.3.4
+    external:
+      enabled: true
+      service:
+        enabled: false
+      addresses:
+        - localhost
+    listeners:
+      kafka:
+        external:
+          default:
+            enabled: true
+            port: 9094
+            advertisedPorts:
+              - 29094
+    statefulset:
+      replicas: 1
+    config:
+      cluster:
+        default_topic_replications: 1
+        enable_shadow_linking: true
+    storage:
+      tiered:
+        config:
+          cloud_storage_enabled: false
+    tls:
+      enabled: false
+    auth:
+      sasl:
+        enabled: false

kubernetes/shadow-linking/resources/shadow-link.yaml

@@ -0,0 +1,46 @@
+apiVersion: cluster.redpanda.com/v1alpha2
+kind: ShadowLink
+metadata:
+  name: shadow-link
+  namespace: shadow
+spec:
+
+  # What cluster are we writing to?
+  shadowCluster:
+    staticConfiguration:
+      kafka:
+        brokers: ["redpanda.shadow.svc.cluster.local:9093"]
+      admin:
+        urls: ["http://redpanda.shadow.svc.cluster.local:9644/"]
+      schemaRegistry:
+        urls: ["http://redpanda.shadow.svc.cluster.local:8081/"]
+
+  # What cluster are we reading from?
+  sourceCluster:
+    staticConfiguration:
+      kafka:
+        brokers: ["redpanda.source.svc.cluster.local:9093"]
+      admin:
+        urls: ["http://redpanda.source.svc.cluster.local:9644/"]
+      schemaRegistry:
+        urls: ["http://redpanda.source.svc.cluster.local:8081/"]
+
+  # Replicate all topics (except for any that are pre-filtered)
+  topicMetadataSyncOptions:
+    interval: 5s
+    autoCreateShadowTopicFilters:
+      - name: '*'
+        filterType: include
+        patternType: literal
+
+  # Replicate all consumer groups
+  consumerOffsetSyncOptions:
+    interval: 5s
+    groupFilters:
+      - patternType: literal
+        filterType: include
+        name: '*'
+
+  # Replicate schema registry (using _schemas topic replication)
+  schemaRegistrySyncOptions:
+    schema_registry_shadowing_mode: "topic"

kubernetes/shadow-linking/resources/source-cluster.yaml

@@ -0,0 +1,43 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: source
+---
+apiVersion: cluster.redpanda.com/v1alpha2
+kind: Redpanda
+metadata:
+  name: redpanda
+  namespace: source
+spec:
+  clusterSpec:
+    image:
+      tag: v25.3.4
+    external:
+      enabled: true
+      service:
+        enabled: false
+      addresses:
+        - localhost
+    listeners:
+      kafka:
+        external:
+          default:
+            enabled: true
+            port: 9094
+            advertisedPorts:
+              - 19094
+    statefulset:
+      replicas: 1
+    config:
+      cluster:
+        default_topic_replications: 1
+        enable_shadow_linking: true
+    storage:
+      tiered:
+        config:
+          cloud_storage_enabled: false
+    tls:
+      enabled: false
+    auth:
+      sasl:
+        enabled: false

kubernetes/shadow-linking/resources/syslog.avsc

@@ -0,0 +1,24 @@
+{
+  "type": "record",
+  "name": "syslog",
+  "fields": [
+    {"name": "name", "type": "string"},
+    {"name": "type", "type": "string"},
+    {"name": "message", "type": "string"},
+    {"name": "host", "type": "string"},
+    {"name": "version", "type": "string"},
+    {"name": "tag", "type": "string"},
+    {"name": "level", "type": "int"},
+    {"name": "facility", "type": "string"},
+    {"name": "severity", "type": "int"},
+    {"name": "appName", "type": "string"},
+    {"name": "remoteAddress", "type": "string"},
+    {"name": "rawMessage", "type": "string"},
+    {"name": "processId", "type": "string"},
+    {"name": "messageId", "type": "string"},
+    {"name": "deviceVendor", "type": "string"},
+    {"name": "deviceProduct", "type": "string"},
+    {"name": "deviceVersion", "type": "string"},
+    {"name": "ts", "type": "int"}
+  ]
+}

kubernetes/shadow-linking/resources/syslogs.json

@@ -0,0 +1,3 @@
+{"name":"log-001","type":"RFC5424","message":"System startup complete","host":"192.0.2.10","version":"3.25.1","tag":".source.s_src","level":6,"facility":"syslog","severity":6,"appName":"SYSTEM","remoteAddress":"198.51.100.1","rawMessage":"System startup complete","processId":"1001","messageId":"10001","deviceVendor":"example","deviceProduct":"server","deviceVersion":"1.0","ts":1}
+{"name":"log-002","type":"RFC5424","message":"User authentication successful","host":"192.0.2.11","version":"3.25.1","tag":".source.s_src","level":6,"facility":"authpriv","severity":6,"appName":"AUTH","remoteAddress":"198.51.100.2","rawMessage":"User authentication successful","processId":"1002","messageId":"10002","deviceVendor":"example","deviceProduct":"server","deviceVersion":"1.0","ts":2}
+{"name":"log-003","type":"RFC5424","message":"Configuration updated","host":"192.0.2.12","version":"3.25.1","tag":".source.s_src","level":5,"facility":"syslog","severity":5,"appName":"CONFIG","remoteAddress":"198.51.100.3","rawMessage":"Configuration updated","processId":"1003","messageId":"10003","deviceVendor":"example","deviceProduct":"server","deviceVersion":"1.0","ts":3}