From 063b6ea33000306aef3094ce15f4c1252b343334 Mon Sep 17 00:00:00 2001 From: Obed N Munoz Date: Mon, 21 Oct 2019 17:22:35 -0500 Subject: [PATCH] Add support for cri-resource-manager project This commit adds support for installing and configuring the [cri-resource-manager](https://github.com/intel/cri-resource-manager) project as an systemd-based service. It also adds the automation to configure `kubelet` service to consume it as a remote container runtime. Finally, it's providing the automation for cleaning up the `kubelet` service configuration to its original state without `cri-resource-manager`. Binary installation will be temporally consumed from a personal fork that is currently hosting `cri-resource-manager` binaries in the meantime that `cri-resource-manager` generates its packaging strategy. Signed-off-by: Obed N Munoz --- .../10-cri-resource-manager/README.md | 70 +++++++++++++++++++ .../10-cri-resource-manager/clean.sh | 23 ++++++ .../10-cri-resource-manager/install.sh | 36 ++++++++++ .../10-cri-resource-manager/setup.sh | 29 ++++++++ clr-k8s-examples/README.md | 3 + 5 files changed, 161 insertions(+) create mode 100644 clr-k8s-examples/10-cri-resource-manager/README.md create mode 100755 clr-k8s-examples/10-cri-resource-manager/clean.sh create mode 100755 clr-k8s-examples/10-cri-resource-manager/install.sh create mode 100755 clr-k8s-examples/10-cri-resource-manager/setup.sh diff --git a/clr-k8s-examples/10-cri-resource-manager/README.md b/clr-k8s-examples/10-cri-resource-manager/README.md new file mode 100644 index 00000000..d9401767 --- /dev/null +++ b/clr-k8s-examples/10-cri-resource-manager/README.md @@ -0,0 +1,70 @@ +# CRI Resource Manager +CRI Resource Manager serves as a relay/proxy between kubelet and the container runtime, relaying requests and responses back and forth between these two, potentially altering requests as they fly by. + +This document explains a very simple use case for the `cri-resource-manager`, for more details and tweaks +on CRI Resource Manager service, you can go to https://github.com/intel/cri-resource-manager. + +## Install + +[`install.sh`](install.sh) script will download the binary and install it as an `systemd` service unit. This script will be executed in all nodes where `cri-resmgr` is required. + +Below you can see the available variables you can use to customize the usage of your CRI Resource Manager service. + +| Variable | Description | Default Value | +|-----------------------------|-------------------------------------------|--------------------------------------------------| +| `RUNNER` | Default Container Runtime | `containerd` | +| `CRI_RESMGR_POLICY` | CRI Resource Manager Policy type | `null` | +| `CRI_RESMGR_POLICY_OPTIONS` | CRI Resource Manager extra policy options | `-dump='reset,full:.*' -dump-file=/tmp/cri.dump` | +| `CRI_RESMGR_DEBUG_OPTIONS` | CRI Resource Manager debugging options | `` | + +**Example:** +```bash +$ RUNNER=containerd ./install.sh +``` + +Verify that the cri-resource-manager service is actually running. + +```bash +$ systemctl status cri-resource-manager +``` + +Verify that the `cri-resmgr` socket is created, it will indicate that `cri-resource-manager` is ready to receive requests. +```bash +$ sudo ls -la /var/run/cri-resmgr/cri-resmgr.sock +``` + +## Setup as a container runtime in `kubelet` + +The [`setup.sh`](setup.sh) script will configure the `kubelet` service to use the `cri-resource-manager` relay as its remote container runtime. This script will be executed in all nodes where `cri-resmgr` is being configured. + +**Example:** +```bash +$ ./setup.sh +``` + +Kubelet service should be restarted and now using `cri-resource-manager` as its container runtime + +```bash +$ ps aux | grep kubelet | grep container-runtime +root 28703 1.7 2.0 1246348 83088 ? Ssl 20:03 0:06 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime remote --container-runtime-endpoint unix:///var/run/cri-resmgr/cri-resmgr.sock +``` + +`cri-resource-manager` service's logs will be located at `/tmp/cri.dump` + +```bash +$ tail /tmp/cri.dump +``` + +## Cleanup + +The [`clean.sh`](clean.sh) will first clean the `kubelet` service as it was before the `cri-resource-manager` and restarts `kubelet` service. This script will be executed in all nodes where `cri-resmgr` is being uninstalled. +Then, it will proceed to stop the `cri-resource-manager` service. + +**Example:** +```bash +$ ./clean.sh +``` + +## More kubernetes native approach (experimental) + +In case that you're interested in a more Kubernetes native way of deploying the CRI Resource manager, take a look on: https://github.com/intel/cri-resource-manager/pull/55 \ No newline at end of file diff --git a/clr-k8s-examples/10-cri-resource-manager/clean.sh b/clr-k8s-examples/10-cri-resource-manager/clean.sh new file mode 100755 index 00000000..7423ff91 --- /dev/null +++ b/clr-k8s-examples/10-cri-resource-manager/clean.sh @@ -0,0 +1,23 @@ +#!/bin/bash +# Copyright (c) 2019 Intel Corporation +# +# SPDX-License-Identifier: Apache-2.0 +# +# Uninstall and stop the CRI Resource Manager service + +set -o errexit +set -o nounset + +# Kubelet +KUBEADM_FLAGS="/var/lib/kubelet/kubeadm-flags.env" +sudo rm -f /etc/systemd/system/kubelet.service.d/99-cri-resource-manager.conf +sudo systemctl daemon-reload +sudo systemctl restart kubelet + +if sudo test -f "$KUBEADM_FLAGS.bkp" ; then + sudo mv $KUBEADM_FLAGS.bkp $KUBEADM_FLAGS +fi + +# CRI Resource Manager +sudo systemctl stop cri-resource-manager +sudo systemctl disable cri-resource-manager diff --git a/clr-k8s-examples/10-cri-resource-manager/install.sh b/clr-k8s-examples/10-cri-resource-manager/install.sh new file mode 100755 index 00000000..56ea23e6 --- /dev/null +++ b/clr-k8s-examples/10-cri-resource-manager/install.sh @@ -0,0 +1,36 @@ +#!/bin/bash +# Copyright (c) 2019 Intel Corporation +# +# SPDX-License-Identifier: Apache-2.0 +# +# Install and start the CRI Resource Manager service + +set -o errexit +set -o nounset + +RUNNER=${RUNNER:-"containerd"} +CRI_RESMGR_POLICY=${CRI_RESMGR_POLICY:-"null"} +CRI_RESMGR_POLICY_OPTIONS=${CRI_RESMGR_POLICY_OPTIONS:-"-dump='reset,full:.*' -dump-file=/tmp/cri.dump"} +CRI_RESMGR_DEBUG_OPTIONS=${CRI_RESMGR_DEBUG_OPTIONS:-""} + +curl https://raw.githubusercontent.com/obedmr/cri-resource-manager/master/godownloader.sh | bash +sudo cp ./bin/* /usr/bin/ + +runtime_socket=$(sudo find /run/ -iname $RUNNER.sock | head -1) +CRI_RESMGR_POLICY_OPTIONS+=" -runtime-socket=$runtime_socket -image-socket=$runtime_socket" + +sudo mkdir -p /etc/sysconfig/ +cat <