1 GitHub runner manages remote kubernetes cluster

.github/workflows/kubectl.yml

@@ -0,0 +1,40 @@
+name: Run kubectl against remote cluster
+
+on:
+  workflow_dispatch: # Allows manual start of workflows
+  push:
+    branches:
+      - "1-github-runner-manages-remote"
+    paths-ignore:
+      - *.md
+      - *.example
+
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Install kubectl
+        run: |
+          mkdir $HOME/bin
+          curl -Lf "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" -o $HOME/bin/kubectl
+          chmod +x $HOME/bin/kubectl
+          echo "$HOME/bin" >> $GITHUB_PATH
+
+      - name: Check kubectl is available on PATH
+        run: kubectl version --client
+
+      - name: Checkout repo
+        uses: actions/checkout@v4
+
+      - name: Set kubeconfig with kubectl
+        run: |
+          kubectl config set-cluster "${{ vars.KUBE_REMOTE_CLUSTER }}" --server "${{ secrets.KUBE_API_SERVER_ADDR }}"
+          kubectl config set-credentials "${{ vars.KUBE_REMOTE_USER }}" --token "${{ secrets.KUBE_JWT_AUTH_TOKEN }}"
+          kubectl config set-context "${{ vars.KUBE_REMOTE_CONTEXT }}" --cluster "${{ vars.KUBE_REMOTE_CLUSTER }}" --user "${{ vars.KUBE_REMOTE_USER }}"
+          kubectl config use-context "${{ vars.KUBE_REMOTE_CONTEXT }}"
+
+      - name: Run kubectl command against remote API
+        run: kubectl get namespaces
+
+      - name: kubectl apply with a file
+        run: kubectl apply -f "${GITHUB_WORKSPACE}/manifests/nginx-test.yml"

.gitignore

@@ -1,4 +1,3 @@
 kube-config
 *.crt
 *.key
-kubectl

README.md

@@ -1,5 +1,11 @@
 # github-actions-k8s
 
+## A quick note
+
+- This repo is an **example** of how to do this, and for that it uses minikube. This repo does **not** create a production ready environment, it merely displays the concepts and methods for remotely managing a cluster with kubectl.
+
+## Get started
+
 - Running `kubectl` against a K8s cluster using GitHub Actions
 - This example runs a minikube cluster inside a VM managed by incus, then runs a `kubectl apply` against that cluster from a Github Actions runner
   - A publicly accessibly IP address/domain is required

kubectl-gh-actions.md

@@ -0,0 +1,98 @@
+- Assuming you followed [manual-remote-access.md](./manual-remote-access.md), you should now have remote access to your minikube cluster set up
+
+# Steps
+
+1. Add secrets for the server address and JWT token, and other variables
+
+  - Note that generated JWT tokens are relatively short-lived, but you can extend their validity by passing `--duration=<timespan>` to `kubectl create token`
+    - e.g. `kubectl create-token remote-dev --duration=12h` for a token valid for 12 hours
+    - We probably don't want to use these in production, your kubernetes provider (e.g. EKS) may offer a better means of authentication
+
+  - On the webpage for your repo:
+    - Settings -> Secrets and Variables -> Actions -> New Repository Secret
+    - Set the name to `KUBE_JWT_AUTH_TOKEN`
+    - Set the value to the JWT token you generated
+  - Add another secret called `KUBE_API_SERVER_ADDR` with the value of your public-facing API server address
+
+  - We'll also add some variables for the cluster, remote username, and remote context
+  - On the webpage for your repo:
+    - Settings -> Secrets and Variables -> Actions -> Variables -> New Repository Variable
+    - Add three variables with the names and values:
+      - KUBE_REMOTE_CLUSTER = minikube
+      - KUBE_REMOTE_USER = remote-dev
+      - KUBE_REMOTE_CONTEXT = remote-context
+
+2. Access the secrets and variables in the action
+
+  - Github actions can access repository secrets using the syntax `${{ secrets.<secret> }}` and variables with `${{ vars.<variable> }}`
+  - We'll create a step in our action that sets the correct kubeconfig
+
+  ```yaml
+    # Other steps... #
+
+    - name: Set kubeconfig with kubectl
+      run: |
+        kubectl config set-cluster "${{ vars.KUBE_REMOTE_CLUSTER }}" --server "${{ secrets.KUBE_API_SERVER_ADDR }}"
+        kubectl config set-credentials "${{ vars.KUBE_REMOTE_USER }}" --token "${{ secrets.KUBE_JWT_AUTH_TOKEN }}"
+        kubectl config set-context "${{ vars.KUBE_REMOTE_CONTEXT }}" --cluster "${{ vars.KUBE_REMOTE_CLUSTER }}" --user "${{ vars.KUBE_REMOTE_USER }}"
+        kubectl config use-context "${{ vars.KUBE_REMOTE_CONTEXT }}"
+
+    # kubectl command steps ... #
+  ```
+
+  - Using these variables and secrets makes it easier to update them in the future, without modifying the workflow file directly
+
+3. Create the full workflow
+
+  - So we need to:
+    1. Make sure the `kubectl` binary is available
+    2. Checkout the repo
+    3. Configure authentication with kubectl
+    4. Run `kubectl` commands against the remote API
+
+  ```yaml
+  # File: .github/workflows/kubectl.yaml
+
+  name: Run kubectl against remote cluster
+  on:
+    workflow_dispatch: # Allows manual start of workflows
+    push:
+      branches:
+        - "main"
+  jobs:
+    deploy:
+      runs-on: ubuntu-latest
+      steps:
+        - name: Install kubectl
+          run: |
+            mkdir "$HOME/bin"
+            curl -Lf "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" -o "$HOME/bin/kubectl"
+            chmod +x $HOME/bin/kubectl
+            echo "$HOME/bin" >> $GITHUB_PATH
+
+        - name: Check kubectl is available on PATH
+          run: kubectl version --client
+
+        - name: Checkout repo
+          uses: actions/checkout@v4
+
+        - name: Set kubeconfig with kubectl
+          run: |
+            kubectl config set-cluster "${{ vars.KUBE_REMOTE_CLUSTER }}" --server "${{ secrets.KUBE_API_SERVER_ADDR }}"
+            kubectl config set-credentials "${{ vars.KUBE_REMOTE_USER }}" --token "${{ secrets.KUBE_JWT_AUTH_TOKEN }}"
+            kubectl config set-context "${{ vars.KUBE_REMOTE_CONTEXT }}" --cluster "${{ vars.KUBE_REMOTE_CLUSTER }}" --user "${{ vars.KUBE_REMOTE_USER }}"
+            kubectl config use-context "${{ vars.KUBE_REMOTE_CONTEXT }}"
+
+        - name: Run kubectl command against remote API
+          run: kubectl get namespaces
+  ```
+
+  - With this, we have remote access to the API in according with the RBAC rules we created earlier
+  - If you wanted to `kubectl apply -f` in this action, you could do so like below:
+
+  ```yaml
+    # previous setup steps #
+
+    - name: kubectl apply with a file
+      run: kubectl apply -f "${GITHUB_WORKSPACE}/manifests/nginx-test.yml"
+  ```

manifests/nginx-test.yml

@@ -0,0 +1,14 @@
+# Create an nginx pod
+
+apiVersion: v1
+kind: Pod
+metadata:
+  name: nginx-pod
+  labels:
+    app: nginx
+spec:
+  containers:
+    - name: nginx-container
+      image: nginx:latest
+      ports:
+        - containerPort: 80

manual-remote-access.md

@@ -138,8 +138,8 @@ $ curl -X GET <your-server-url>/api
 - On the minikube host:
   1. Create a service account, `ClusterRoleBinding`, and token
 
-  - We create a service account called remote-dev that we will authenticate as
-  - We then create a `ClusterRoleBinding` referencing the `cluster-admin` role (created by default which provides full access to everything in the cluster) and bind the remote-dev account we just created to it
+    - We create a service account called remote-dev that we will authenticate as
+    - We then create a `ClusterRoleBinding` referencing the `cluster-admin` role (created by default which provides full access to everything in the cluster) and bind the remote-dev account we just created to it
   ```shell
   ubuntu@kubectl-ghactions-test:~$ kubectl create serviceaccount remote-dev
   ubuntu@kubectl-ghactions-test:~$ kubectl create clusterrolebinding remote-dev-binding \
@@ -177,7 +177,7 @@ current-context: remote-context
 - Test access by making a request
 
 ```shell
-$ KUBECONFIG=$(pwd)/kube-config ./kubectl get ns
+$ KUBECONFIG=$(pwd)/kube-config kubectl get ns
 NAME              STATUS   AGE
 default           Active   7m57s
 kube-node-lease   Active   7m57s