From d94a0ed9e48c47027f120a7eb79aed0d0e1c3876 Mon Sep 17 00:00:00 2001 From: Nir Soffer Date: Wed, 10 Jan 2024 20:36:36 +0200 Subject: [PATCH 01/14] Upgrade CDI to latest release https://github.com/kubevirt/containerized-data-importer/releases/tag/v1.58.1 Signed-off-by: Nir Soffer --- test/addons/cdi/cr/kustomization.yaml | 2 +- test/addons/cdi/operator/kustomization.yaml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/test/addons/cdi/cr/kustomization.yaml b/test/addons/cdi/cr/kustomization.yaml index 3fd3d08eb5..f0e1900585 100644 --- a/test/addons/cdi/cr/kustomization.yaml +++ b/test/addons/cdi/cr/kustomization.yaml @@ -4,4 +4,4 @@ # yamllint disable rule:line-length --- resources: -- https://github.com/kubevirt/containerized-data-importer/releases/download/v1.57.0/cdi-cr.yaml +- https://github.com/kubevirt/containerized-data-importer/releases/download/v1.58.1/cdi-cr.yaml diff --git a/test/addons/cdi/operator/kustomization.yaml b/test/addons/cdi/operator/kustomization.yaml index b894cb9790..10ff5811f9 100644 --- a/test/addons/cdi/operator/kustomization.yaml +++ b/test/addons/cdi/operator/kustomization.yaml @@ -4,4 +4,4 @@ # yamllint disable rule:line-length --- resources: -- https://github.com/kubevirt/containerized-data-importer/releases/download/v1.57.0/cdi-operator.yaml +- https://github.com/kubevirt/containerized-data-importer/releases/download/v1.58.1/cdi-operator.yaml From dab930fc1caad2af22f3a3a08fe0855fb92ef828 Mon Sep 17 00:00:00 2001 From: Nir Soffer Date: Wed, 10 Jan 2024 20:39:05 +0200 Subject: [PATCH 02/14] Update kubvirt to latest release https://github.com/kubevirt/kubevirt/releases/tag/v1.2.0 Signed-off-by: Nir Soffer --- test/addons/kubevirt/cr/kustomization.yaml | 2 +- test/addons/kubevirt/operator/kustomization.yaml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/test/addons/kubevirt/cr/kustomization.yaml b/test/addons/kubevirt/cr/kustomization.yaml index 2f02b94a99..7efcb47dd5 100644 --- a/test/addons/kubevirt/cr/kustomization.yaml +++ b/test/addons/kubevirt/cr/kustomization.yaml @@ -4,4 +4,4 @@ # yamllint disable rule:line-length --- resources: -- https://github.com/kubevirt/kubevirt/releases/download/v1.0.1/kubevirt-cr.yaml +- https://github.com/kubevirt/kubevirt/releases/download/v1.2.0/kubevirt-cr.yaml diff --git a/test/addons/kubevirt/operator/kustomization.yaml b/test/addons/kubevirt/operator/kustomization.yaml index 2846de42f8..688334562a 100644 --- a/test/addons/kubevirt/operator/kustomization.yaml +++ b/test/addons/kubevirt/operator/kustomization.yaml @@ -4,4 +4,4 @@ # yamllint disable rule:line-length --- resources: -- https://github.com/kubevirt/kubevirt/releases/download/v1.0.1/kubevirt-operator.yaml +- https://github.com/kubevirt/kubevirt/releases/download/v1.2.0/kubevirt-operator.yaml From c0289b487a6f50ec03280add2214e12de6547b9f Mon Sep 17 00:00:00 2001 From: Nir Soffer Date: Wed, 10 Jan 2024 20:26:44 +0200 Subject: [PATCH 03/14] Use the standard cirros image in CDI test There is no point in using tow versions of the same image. Using this image in the CDI test can save time in the kubvirt tests later, using the cached image. Signed-off-by: Nir Soffer --- test/addons/cdi/disk/source.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/addons/cdi/disk/source.yaml b/test/addons/cdi/disk/source.yaml index cdd40c0646..c320d64541 100644 --- a/test/addons/cdi/disk/source.yaml +++ b/test/addons/cdi/disk/source.yaml @@ -9,4 +9,4 @@ metadata: spec: source: registry: - url: "docker://quay.io/alitke/cirros:latest" + url: "docker://quay.io/nirsof/cirros:0.6.2-1" From 25e2c3a44b89234cf5c7dbb1c1746b33755c201a Mon Sep 17 00:00:00 2001 From: Nir Soffer Date: Wed, 10 Jan 2024 21:36:20 +0200 Subject: [PATCH 04/14] Fix waiting for CDI CDI may become available before it is ready to use. If we try to use it while it is progressing we may fail with errors about missing CRDS. Wait until the progressing condition becomes false. Example run showing the issue: 2024-01-10 21:42:24,080 DEBUG [kubevirt/1] Deploying cdi cr 2024-01-10 21:42:25,674 DEBUG [kubevirt/1] Waiting until cdi cr is available 2024-01-10 21:42:26,005 DEBUG [kubevirt/1] cdi.cdi.kubevirt.io/cdi condition met We stopped waiting here... 2024-01-10 21:42:26,007 DEBUG [kubevirt/1] Waiting until cdi cr finished progressing 2024-01-10 21:42:39,472 DEBUG [kubevirt/1] cdi.cdi.kubevirt.io/cdi condition met But CDI finished progressing 13 seconds later. Signed-off-by: Nir Soffer --- test/addons/cdi/start | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/test/addons/cdi/start b/test/addons/cdi/start index f9bbf0fa0c..86dde20bd6 100755 --- a/test/addons/cdi/start +++ b/test/addons/cdi/start @@ -37,6 +37,14 @@ def wait(cluster): "--timeout=600s", context=cluster, ) + print("Waiting until cdi cr finished progressing") + kubectl.wait( + "cdi.cdi.kubevirt.io/cdi", + "--for=condition=progressing=False", + f"--namespace={NAMESPACE}", + "--timeout=300s", + context=cluster, + ) if len(sys.argv) != 2: From 43266c52d171406ba9417346084594d67fe1efef Mon Sep 17 00:00:00 2001 From: Nir Soffer Date: Sun, 19 Nov 2023 13:18:47 +0200 Subject: [PATCH 05/14] Trim kubevirt environment We cannot use volsync with ramen yet, and the kubevirt environment is already too big. Without volsync we can remove the volumesnapshot addon and submariner, which does not handle well suspending of the machine running the minikube VMs. With this change we should be able to start an environment, suspend the laptop, and resume it in an environment with unreliable network or no network access. This will be useful for live demo in conferences. Keep volsync enabled in `regional-dr` and `regional-dr-hubless` to keep the submariner and volsync addons functional. Signed-off-by: Nir Soffer --- ramenctl/ramenctl/config.py | 9 ++++++--- ramenctl/ramenctl/resources/configmap.yaml | 2 ++ test/envs/regional-dr-hubless.yaml | 2 ++ test/envs/regional-dr-kubevirt.yaml | 8 ++------ test/envs/regional-dr.yaml | 2 ++ 5 files changed, 14 insertions(+), 9 deletions(-) diff --git a/ramenctl/ramenctl/config.py b/ramenctl/ramenctl/config.py index 977d286bba..8f84257ee3 100644 --- a/ramenctl/ramenctl/config.py +++ b/ramenctl/ramenctl/config.py @@ -25,7 +25,7 @@ def run(args): cloud_secret = generate_cloud_credentials_secret(env["clusters"][0], args) if env["hub"]: - hub_cm = generate_config_map("hub", env["clusters"], args) + hub_cm = generate_config_map("hub", env, args) wait_for_ramen_hub_operator(env["hub"], args) @@ -38,7 +38,7 @@ def run(args): wait_for_dr_clusters(env["hub"], env["clusters"], args) wait_for_dr_policy(env["hub"], args) else: - dr_cluster_cm = generate_config_map("dr-cluster", env["clusters"], args) + dr_cluster_cm = generate_config_map("dr-cluster", env, args) for cluster in env["clusters"]: create_ramen_s3_secrets(cluster, s3_secrets) @@ -89,7 +89,9 @@ def create_cloud_credentials_secret(cluster, yaml): kubectl.apply("--filename=-", input=yaml, context=cluster, log=command.debug) -def generate_config_map(controller, clusters, args): +def generate_config_map(controller, env, args): + clusters = env["clusters"] + volsync = env["features"].get("volsync", True) template = drenv.template(command.resource("configmap.yaml")) return template.substitute( name=f"ramen-{controller}-operator-config", @@ -98,6 +100,7 @@ def generate_config_map(controller, clusters, args): cluster2=clusters[1], minio_url_cluster1=minio.service_url(clusters[0]), minio_url_cluster2=minio.service_url(clusters[1]), + volsync_disabled="false" if volsync else "true", namespace=args.ramen_namespace, ) diff --git a/ramenctl/ramenctl/resources/configmap.yaml b/ramenctl/ramenctl/resources/configmap.yaml index c82136f1f7..53b2b7fb44 100644 --- a/ramenctl/ramenctl/resources/configmap.yaml +++ b/ramenctl/ramenctl/resources/configmap.yaml @@ -34,6 +34,8 @@ data: clusterServiceVersionName: ramen-dr-cluster-operator.v0.0.1 kubeObjectProtection: veleroNamespaceName: velero + volSync: + disabled: $volsync_disabled s3StoreProfiles: - s3ProfileName: minio-on-$cluster1 s3Bucket: bucket diff --git a/test/envs/regional-dr-hubless.yaml b/test/envs/regional-dr-hubless.yaml index fff2a57f74..2928f7747a 100644 --- a/test/envs/regional-dr-hubless.yaml +++ b/test/envs/regional-dr-hubless.yaml @@ -9,6 +9,8 @@ ramen: hub: null clusters: [dr1, dr2] topology: regional-dr + features: + volsync: true templates: - name: "dr-cluster" diff --git a/test/envs/regional-dr-kubevirt.yaml b/test/envs/regional-dr-kubevirt.yaml index eeb7fdbced..a45eca833e 100644 --- a/test/envs/regional-dr-kubevirt.yaml +++ b/test/envs/regional-dr-kubevirt.yaml @@ -9,6 +9,8 @@ ramen: hub: hub clusters: [dr1, dr2] topology: regional-dr + features: + volsync: false templates: - name: "dr-cluster" @@ -25,7 +27,6 @@ templates: extra_disks: 1 disk_size: "50g" addons: - - volumesnapshots - csi-hostpath-driver workers: - addons: @@ -57,8 +58,6 @@ templates: - name: ocm-controller - name: cert-manager - name: olm - - name: submariner - args: ["hub", "dr1", "dr2"] profiles: - name: "dr1" @@ -72,6 +71,3 @@ workers: - addons: - name: rbd-mirror args: ["dr1", "dr2"] - - addons: - - name: volsync - args: ["dr1", "dr2"] diff --git a/test/envs/regional-dr.yaml b/test/envs/regional-dr.yaml index 516ae921cc..74c1646621 100644 --- a/test/envs/regional-dr.yaml +++ b/test/envs/regional-dr.yaml @@ -9,6 +9,8 @@ ramen: hub: hub clusters: [dr1, dr2] topology: regional-dr + features: + volsync: true templates: - name: "dr-cluster" From 1ab991dec7e33e11207b3b163fc4916ed732a7fc Mon Sep 17 00:00:00 2001 From: Nir Soffer Date: Mon, 20 Nov 2023 01:02:06 +0200 Subject: [PATCH 06/14] Add drenv --skip-addons option This is useful for starting a stopped working environment quickly without trying to redeploy everything. The main motivation is using a pre created environment in location with weak network like a conference. Other use cases are working around bugs in addons that do not work well when starting a stopped cluster, for example clusteradm. With `--skip-addons` we skip the `start` and `stop` hooks, but we do run the `test` hooks. This is useful for starting a stopped environment faster but testing that the environment works. To skip all hooks run with both `--skip-addons` and `--skip-tests`. Example run: $ drenv start --skip-addons --skip-tests $env 2023-11-20 00:59:25,341 INFO [rdr-kubevirt] Starting environment 2023-11-20 00:59:25,464 INFO [dr1] Starting minikube cluster 2023-11-20 00:59:29,566 INFO [hub] Starting minikube cluster 2023-11-20 00:59:29,578 INFO [dr2] Starting minikube cluster 2023-11-20 01:00:23,402 INFO [dr1] Cluster started in 57.94 seconds 2023-11-20 01:00:23,402 INFO [dr1] Configuring containerd 2023-11-20 01:00:24,936 INFO [dr1] Waiting until all deployments are available 2023-11-20 01:00:28,749 INFO [hub] Cluster started in 59.18 seconds 2023-11-20 01:00:28,750 INFO [hub] Waiting until all deployments are available 2023-11-20 01:00:53,834 INFO [dr2] Cluster started in 84.26 seconds 2023-11-20 01:00:53,834 INFO [dr2] Configuring containerd 2023-11-20 01:00:55,042 INFO [dr2] Waiting until all deployments are available 2023-11-20 01:01:01,063 INFO [hub] Deployments are available in 32.31 seconds 2023-11-20 01:01:09,482 INFO [dr1] Deployments are available in 44.55 seconds 2023-11-20 01:01:34,661 INFO [dr2] Deployments are available in 39.62 seconds 2023-11-20 01:01:34,661 INFO [rdr-kubevirt] Dumping ramen e2e config to '/home/nsoffer/.config/drenv/rdr-kubevirt' 2023-11-20 01:01:34,827 INFO [rdr-kubevirt] Environment started in 129.49 seconds Signed-off-by: Nir Soffer --- test/drenv/__main__.py | 39 +++++++++++++++++++++++++++++++-------- 1 file changed, 31 insertions(+), 8 deletions(-) diff --git a/test/drenv/__main__.py b/test/drenv/__main__.py index a7b1819012..70e806e6ee 100644 --- a/test/drenv/__main__.py +++ b/test/drenv/__main__.py @@ -30,7 +30,16 @@ def main(): p = argparse.ArgumentParser(prog="drenv") p.add_argument("-v", "--verbose", action="store_true", help="Be more verbose") p.add_argument( - "--skip-tests", dest="run_tests", action="store_false", help="Skip self tests" + "--skip-tests", + dest="run_tests", + action="store_false", + help="Skip addons 'test' hooks", + ) + p.add_argument( + "--skip-addons", + dest="run_addons", + action="store_false", + help="Skip addons 'start' and 'stop' hooks", ) p.add_argument("command", choices=commands, help="Command to run") p.add_argument("--name-prefix", help="Prefix profile names") @@ -57,7 +66,12 @@ def main(): def cmd_start(env, args): start = time.monotonic() logging.info("[%s] Starting environment", env["name"]) - hooks = ["start", "test"] if args.run_tests else ["start"] + + hooks = [] + if args.run_addons: + hooks.append("start") + if args.run_tests: + hooks.append("test") # Delaying `minikube start` ensures cluster start order. execute( @@ -67,7 +81,9 @@ def cmd_start(env, args): hooks=hooks, args=args, ) - execute(run_worker, env["workers"], hooks=hooks) + + if hooks: + execute(run_worker, env["workers"], hooks=hooks) if "ramen" in env: ramen.dump_e2e_config(env) @@ -82,7 +98,8 @@ def cmd_start(env, args): def cmd_stop(env, args): start = time.monotonic() logging.info("[%s] Stopping environment", env["name"]) - execute(stop_cluster, env["profiles"]) + hooks = ["stop"] if args.run_addons else [] + execute(stop_cluster, env["profiles"], hooks=hooks) logging.info( "[%s] Environment stopped in %.2f seconds", env["name"], @@ -152,17 +169,23 @@ def start_cluster(profile, hooks=(), args=None, **options): if is_restart: wait_for_deployments(profile) - execute(run_worker, profile["workers"], max_workers=args.max_workers, hooks=hooks) + if hooks: + execute( + run_worker, + profile["workers"], + max_workers=args.max_workers, + hooks=hooks, + ) -def stop_cluster(profile, **options): +def stop_cluster(profile, hooks=(), **options): cluster_status = cluster.status(profile["name"]) - if cluster_status == cluster.READY: + if cluster_status == cluster.READY and hooks: execute( run_worker, profile["workers"], - hooks=["stop"], + hooks=hooks, reverse=True, allow_failure=True, ) From de261b10849a279762cac5d98c6e033c3f146b2f Mon Sep 17 00:00:00 2001 From: Nir Soffer Date: Tue, 23 Jan 2024 01:50:02 +0200 Subject: [PATCH 07/14] Add drenv suspend and resume commands Suspend or resume the underlying virtual machines. Assuming kvm2 driver to keep it simple for now, need to implement it better later so it works also with qemu2 driver. The use case is building the environment with good network, suspending it, and resuming it in an environment with flaky network for demo. Signed-off-by: Nir Soffer --- test/drenv/__main__.py | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/test/drenv/__main__.py b/test/drenv/__main__.py index 70e806e6ee..cad6c86d23 100644 --- a/test/drenv/__main__.py +++ b/test/drenv/__main__.py @@ -124,6 +124,18 @@ def cmd_delete(env, args): ) +def cmd_suspend(env, args): + logging.info("[%s] Suspending environment", env["name"]) + for profile in env["profiles"]: + run("virsh", "-c", "qemu:///system", "suspend", profile["name"]) + + +def cmd_resume(env, args): + logging.info("[%s] Resuming environment", env["name"]) + for profile in env["profiles"]: + run("virsh", "-c", "qemu:///system", "resume", profile["name"]) + + def cmd_dump(env, args): yaml.dump(env, sys.stdout) From 6f8e6a3f4ef2eb450dd46c519ac4ae41f0af0f37 Mon Sep 17 00:00:00 2001 From: Nir Soffer Date: Wed, 10 Jan 2024 15:07:37 +0200 Subject: [PATCH 08/14] Allow pulling from local insecure registry Configure CDI to allow pulling from a local insecure registry. This is useful for demos in an environment with unreliable network, or for CI environment when we want to avoid random failures due to flaky network. The image must be pushed to the local registry, this is easy using standard podman push command. Signed-off-by: Nir Soffer --- test/addons/cdi/cr/kustomization.yaml | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/test/addons/cdi/cr/kustomization.yaml b/test/addons/cdi/cr/kustomization.yaml index f0e1900585..d6597e2c87 100644 --- a/test/addons/cdi/cr/kustomization.yaml +++ b/test/addons/cdi/cr/kustomization.yaml @@ -4,4 +4,18 @@ # yamllint disable rule:line-length --- resources: -- https://github.com/kubevirt/containerized-data-importer/releases/download/v1.58.1/cdi-cr.yaml + - https://github.com/kubevirt/containerized-data-importer/releases/download/v1.58.1/cdi-cr.yaml +patches: + # Allow pulling from local insecure registry. + - target: + kind: CDI + name: cdi + patch: |- + apiVersion: cdi.kubevirt.io/v1beta1 + kind: CDI + metadata: + name: not-used + spec: + config: + insecureRegistries: + - host.minikube.internal:5000 From 1b8e44a747e66a61d48bf2997675044024c83c63 Mon Sep 17 00:00:00 2001 From: Nir Soffer Date: Fri, 26 Jan 2024 23:56:34 +0200 Subject: [PATCH 09/14] Extend kubvirt certificate lifetime to 1 week To avoid certificate renewals during testing. Without this I experienced this error: drenv.commands.Error: Command failed: command: ('kubectl', 'apply', '--context', 'dr1', '--kustomize=cr') exitcode: 1 error: Error from server (InternalError): error when applying patch: {"spec":{"configuration":{"developerConfiguration":{"featureGates":[]}}}} to: Resource: "kubevirt.io/v1, Resource=kubevirts", GroupVersionKind: "kubevirt.io/v1, Kind=KubeVirt" Name: "kubevirt", Namespace: "kubevirt" for: "cr": error when patching "cr": Internal error occurred: failed calling webhook "kubevirt-update-validator.kubevirt.io": failed to call webhook: Post "https://kubevirt-operator-webhook.kubevirt.svc:443/kubevirt-validate-update?timeout=10s": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2024-01-26T19:05:52Z is after 2024-01-26T16:24:46Z Thanks: Michael Henriksen Signed-off-by: Nir Soffer --- test/addons/kubevirt/cr/kustomization.yaml | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/test/addons/kubevirt/cr/kustomization.yaml b/test/addons/kubevirt/cr/kustomization.yaml index 7efcb47dd5..71ea633afe 100644 --- a/test/addons/kubevirt/cr/kustomization.yaml +++ b/test/addons/kubevirt/cr/kustomization.yaml @@ -4,4 +4,22 @@ # yamllint disable rule:line-length --- resources: -- https://github.com/kubevirt/kubevirt/releases/download/v1.2.0/kubevirt-cr.yaml + - https://github.com/kubevirt/kubevirt/releases/download/v1.2.0/kubevirt-cr.yaml +patches: + # Incrase certificate duration to avoid certificates renewals while a cluster + # is suspended and resumed. + - target: + kind: KubeVirt + name: kubevirt + patch: |- + apiVersion: kubevirt.io/v1 + kind: Kubevirt + metadata: + name: not-used + spec: + certificateRotateStrategy: + selfSigned: + ca: + duration: 168h + server: + duration: 168h From dbb8031349f7f3747a36fc99c3d6c4b38f9f1281 Mon Sep 17 00:00:00 2001 From: Nir Soffer Date: Mon, 11 Mar 2024 15:21:32 +0200 Subject: [PATCH 10/14] Extend CDI certificate lifetime to 1 week To avoid certificate renewals during testing. Without this I experienced this error when starting a stopped environment after a day: drenv.commands.Error: Command failed: command: ('kubectl', 'apply', '--context', 'dr2', '--kustomize=disk') exitcode: 1 error: Error from server (InternalError): error when creating "disk": Internal error occurred: failed calling webhook "populator-validate.cdi.kubevirt.io": failed to call webhook: Post "https://cdi-api.cdi.svc:443/populator-validate?timeout=30s": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2024-01-28T14:08:01Z is after 2024-01-27T19:15:20Z Signed-off-by: Nir Soffer --- test/addons/cdi/cr/kustomization.yaml | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/test/addons/cdi/cr/kustomization.yaml b/test/addons/cdi/cr/kustomization.yaml index d6597e2c87..7763740f7a 100644 --- a/test/addons/cdi/cr/kustomization.yaml +++ b/test/addons/cdi/cr/kustomization.yaml @@ -19,3 +19,19 @@ patches: config: insecureRegistries: - host.minikube.internal:5000 + # Incrase certificate duration to avoid certificates renewals while a cluster + # is suspended and resumed. + - target: + kind: CDI + name: cdi + patch: |- + apiVersion: cdi.kubevirt.io/v1beta1 + kind: CDI + metadata: + name: not-used + spec: + certConfig: + ca: + duration: 168h + server: + duration: 168h From 327f5f605d0d538cf8c92d7ffefae342d3029289 Mon Sep 17 00:00:00 2001 From: Nir Soffer Date: Wed, 10 Jan 2024 23:44:18 +0200 Subject: [PATCH 11/14] How to configure local git server Using a local git server we can deploy ocm applications without network access to github. This is useful for demos when the network is unreliable, for example in a conference. Signed-off-by: Nir Soffer --- test/gitlap/README.md | 88 +++++++++++++++++++++++++++++++++++++++++ test/gitlap/gitlap.conf | 17 ++++++++ 2 files changed, 105 insertions(+) create mode 100644 test/gitlap/README.md create mode 100644 test/gitlap/gitlap.conf diff --git a/test/gitlap/README.md b/test/gitlap/README.md new file mode 100644 index 0000000000..d71460c68e --- /dev/null +++ b/test/gitlap/README.md @@ -0,0 +1,88 @@ +# Setting up a local git server + +## Initial setup + +1. Install lighttpd + + ``` + sudo dnf install lighttpd + ``` + +1. Create the git repo + + Create a directory where the git repositories will be served: + + ``` + sudo mkdir /var/www/gitlap + cd /var/www/gitlap + sudo git clone --bare https://github.com/nirs/ocm-kubevirt-samples.git + ``` + + Set git repo permissions so you can push changes, and the web server + can serve the repo. + + ``` + sudo chown -R $USER:lighttpd /var/www/gitlap + ``` + +1. Copy the vhost configuration + + ``` + sudo cp gitlap.conf /etc/lighttpd/vhosts.d/ + ``` + +1. Uncomment the vhost include in /etc/lighttpd/lighttpd.conf + + ``` + include conf_dir + "/vhosts.d/*.conf" + ``` + +1. Enable and start the service + + ``` + sudo systemctl enable --now lighttpd + ``` + +1. Allow http access in the libvirt zone + + ``` + sudo firewall-cmd --zone=libvirt --add-service=http --permanent + sudo firewall-cmd --reload + ``` + +## Testing the server + +1. Add entry in /etc/hosts for testing locally + + ``` + 192.168.122.1 host.minikube.internal + ``` + +1. Check that git clone works + + ``` + git clone http://host.minikube.internal/ocm-kubevirt-samples.git + rm -rf ocm-kubevirt-samples + ``` + +1. Check git clone in a minikube cluster + + ``` + minikube ssh -p dr1 + git clone http://host.minikube.internal/ocm-kubevirt-samples.git + rm -rf ocm-kubevirt-samples + ``` + +## Updating the git repo + +1. Add a remote to your working repo + + ``` + git remote add gitlap file:///var/www/gitlap/ocm-kubevirt-samples.git + ``` + +1. Push changes to the remote + + ``` + git push -f gitlap main + ``` diff --git a/test/gitlap/gitlap.conf b/test/gitlap/gitlap.conf new file mode 100644 index 0000000000..bfe98fc706 --- /dev/null +++ b/test/gitlap/gitlap.conf @@ -0,0 +1,17 @@ +# Minimal configuration for local git server for minikube clusters. +# +# For more options see: +# https://redmine.lighttpd.net/projects/lighttpd/wiki/How_to_set_up_a_git_server_over_http(s) + +server.modules += ("mod_setenv", "mod_cgi", "mod_alias") + +# `host.minikube.internal` is a special DNS name injected by minikube to all clusters. +# https://minikube.sigs.k8s.io/docs/handbook/host-access/ +$HTTP["host"] == "host.minikube.internal" { + alias.url = ( "" => "/usr/libexec/git-core/git-http-backend" ) + setenv.set-environment = ( + "GIT_PROJECT_ROOT" => "/var/www/gitlap/", + "GIT_HTTP_EXPORT_ALL" => "1" + ) + cgi.assign = ( "" => "" ) +} From d7e9d8b21dd13329c0f4ac7e319d735c07877017 Mon Sep 17 00:00:00 2001 From: Nir Soffer Date: Thu, 11 Jan 2024 00:16:13 +0200 Subject: [PATCH 12/14] How to use a local registry for minikube clusters Using local registry is useful for demos when network is unreliable, for example in a conference. It can also be used to avoid random failures when the network is flaky, by caching remove images locally. Signed-off-by: Nir Soffer --- test/registry/README.md | 73 +++++++++++++++++++++++ test/registry/host.minikube.internal.conf | 4 ++ 2 files changed, 77 insertions(+) create mode 100644 test/registry/README.md create mode 100644 test/registry/host.minikube.internal.conf diff --git a/test/registry/README.md b/test/registry/README.md new file mode 100644 index 0000000000..f172558e80 --- /dev/null +++ b/test/registry/README.md @@ -0,0 +1,73 @@ +# Using local registry for minikube clusters + +## Initial setup + +1. Install podman + + ``` + sudo dnf install podman + ``` + +1. Run the registry container + + ``` + podman run --name registry \ + --publish 5000:5000 \ + --volume registry:/var/lib/registry:Z \ + --detach \ + --replace \ + docker.io/library/registry:latest + ``` + + Use `--replace` to replace an existing container, typically left + after reboot the host. + +1. Allow access to port 5000 in the libvirt zone + + ``` + sudo firewall-cmd --zone=libvirt --add-port=5000/tcp --permanent + sudo firewall-cmd --reload + ``` + +1. Configure podman to allow insecure access + + ``` + sudo cp host.minikube.internal.conf /etc/containers/registries.conf.d/ + ``` + +1. Testing the registry + + ``` + $ curl host.minikube.internal:5000/v2/_catalog + {} + ``` + +## Pushing to the local registry + +1. Pull the image from a remote registry + + ``` + podman pull quay.io/nirsof/cirros:0.6.2-1 + ``` + +1. Push to the local registry + + ``` + podman push quay.io/nirsof/cirros:0.6.2-1 host.minikube.internal:5000/nirsof/cirros:0.6.2-1 + ``` + +## Using images from the local registry + +Example source.yaml: + +``` +--- +apiVersion: cdi.kubevirt.io/v1beta1 +kind: VolumeImportSource +metadata: + name: cirros-source +spec: + source: + registry: + url: "docker://host.minikube.internal:5000/nirsof/cirros:0.6.2-1" +``` diff --git a/test/registry/host.minikube.internal.conf b/test/registry/host.minikube.internal.conf new file mode 100644 index 0000000000..03bc08aa31 --- /dev/null +++ b/test/registry/host.minikube.internal.conf @@ -0,0 +1,4 @@ +# Local registry for serving minikube clusters +[[registry]] +location = 'host.minikube.internal:5000' +insecure = true From 73b4e4dffb0727652664eda8c6d1489ff16166c0 Mon Sep 17 00:00:00 2001 From: Nir Soffer Date: Sun, 28 Jan 2024 23:20:34 +0200 Subject: [PATCH 13/14] Add systemd service for the registry With this you can run the local registry as a systemd service starting at boot, instead of starting the registry manually when you want to use it. Signed-off-by: Nir Soffer --- test/registry/README.md | 17 +++++++++++++++++ test/registry/systemd/registry.container | 12 ++++++++++++ test/registry/systemd/registry.volume | 5 +++++ 3 files changed, 34 insertions(+) create mode 100644 test/registry/systemd/registry.container create mode 100644 test/registry/systemd/registry.volume diff --git a/test/registry/README.md b/test/registry/README.md index f172558e80..6e3c2ea3e3 100644 --- a/test/registry/README.md +++ b/test/registry/README.md @@ -22,6 +22,9 @@ Use `--replace` to replace an existing container, typically left after reboot the host. + To run the registry as system service see + [systemd service](#systemd-service). + 1. Allow access to port 5000 in the libvirt zone ``` @@ -71,3 +74,17 @@ spec: registry: url: "docker://host.minikube.internal:5000/nirsof/cirros:0.6.2-1" ``` + +## Systemd service + +To create a registry service running at boot, install the provided +systemd units and start the service. + +``` +sudo cp systemd/registry.* /etc/containers/systemd/ +sudo systemctl daemon-reload +sudo systemctl start registry.service +``` + +> [!NOTE] +> The service does not need to be enabled. diff --git a/test/registry/systemd/registry.container b/test/registry/systemd/registry.container new file mode 100644 index 0000000000..e7c63bbfe8 --- /dev/null +++ b/test/registry/systemd/registry.container @@ -0,0 +1,12 @@ +[Unit] +Description=Registry container + +[Container] +Label=app=registry +ContainerName=registry +Image=docker.io/library/registry:latest +PublishPort=5000:5000 +Volume=registry.volume:/var/lib/registry + +[Install] +WantedBy=multi-user.target default.target diff --git a/test/registry/systemd/registry.volume b/test/registry/systemd/registry.volume new file mode 100644 index 0000000000..d256357a6a --- /dev/null +++ b/test/registry/systemd/registry.volume @@ -0,0 +1,5 @@ +[Unit] +Description=Registry Volume + +[Volume] +Label=app=registry From 02ce48693b47a4f118442e13c73df7570b5ded0f Mon Sep 17 00:00:00 2001 From: Nir Soffer Date: Wed, 13 Mar 2024 20:08:58 +0200 Subject: [PATCH 14/14] Add PVC based VM sample configuration MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Since we plan multiple configuration for kubevirt, we use the same layout as ocm-ramen-samples subscription/ directory: configs/ ├── deployment-k8s-regional-rbd.yaml └── kubevirt └── vm-pvc-k8s-regional.yaml To run basic tests using a vm use: basic-test/run -c configs/kubevirt/vm-pvc-k8s-regional.yaml $env Signed-off-by: Nir Soffer --- test/configs/kubevirt/vm-pvc-k8s-regional.yaml | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 test/configs/kubevirt/vm-pvc-k8s-regional.yaml diff --git a/test/configs/kubevirt/vm-pvc-k8s-regional.yaml b/test/configs/kubevirt/vm-pvc-k8s-regional.yaml new file mode 100644 index 0000000000..f3a07827ee --- /dev/null +++ b/test/configs/kubevirt/vm-pvc-k8s-regional.yaml @@ -0,0 +1,11 @@ +# SPDX-FileCopyrightText: The RamenDR authors +# SPDX-License-Identifier: Apache-2.0 + +--- +repo: https://github.com/ramendr/ocm-ramen-samples.git +path: subscription/kubevirt/vm-pvc-k8s-regional +branch: main +name: vm-pvc +namespace: vm-pvc +dr_policy: dr-policy +pvc_label: vm