From K8s Tutorial

Author: Programmer-RD-AI

K8s/01-history-and-motivation/README.md

@@ -0,0 +1,42 @@
+# Section 1: History and Motivation
+
+## 2000s: Traditional Deployment Era
+
+During the 2000s, we experienced what is known as the "Traditional Deployment Era". This period was characterized by:
+
+- **On-premises Deployments**: Companies managed their own data centers or used colocation services.
+- **Teams of Sysadmins**: Dedicated teams of system administrators were responsible for provisioning and managing fleets of servers, which was often a labor-intensive and immature process.
+- **Bare Metal Servers**: Applications ran directly on physical servers.
+- **Monolithic Architecture**: The prevalent architectural style was monolithic, where applications were built as single, indivisible units.
+- **Homegrown Monitoring Tools**: Monitoring and managing applications required custom-built tools due to the lack of standardized solutions.
+
+## 2010s: Virtualized Deployment Era
+
+The 2010s marked the transition to the "Virtualized Deployment Era". Key developments during this time included:
+
+- **Cloud Computing**: The advent of cloud computing allowed Virtual Machines (VMs) to be created and destroyed in minutes, providing greater flexibility and scalability.
+- **Configuration Management Tools**: Tools like Puppet and Chef became popular for managing infrastructure as code, simplifying the configuration and management of large-scale deployments.
+- **Manual Bin-Packing**: Applications were manually allocated to VMs, optimizing resource usage but still requiring significant manual effort.
+- **Improved Tooling**: The emergence of better tooling made it practical to manage a larger number of applications and cloud resources.
+- **Challenges with Scale**: Despite the improvements, managing large numbers of cloud resources remained a significant challenge.
+
+## 2020s: Container Deployment Era
+
+In the 2020s, we entered the "Container Deployment Era", which brought about transformative changes in how workloads are managed:
+
+- **Workload Orchestrators**: Tools like Kubernetes enabled treating clusters of machines as a single resource, simplifying management and scaling.
+- **Standard Interfaces and Utilities**: These orchestrators provided a range of utilities and interfaces to handle:
+  - **Efficient Scheduling**: Optimally distributing workloads across instances.
+  - **Health Checks**: Monitoring the health and status of applications.
+  - **Service Discovery**: Automating the detection of service locations within the cluster.
+  - **Configuration Management**: Standardizing the way configurations are managed and applied.
+  - **Autoscaling**: Automatically adjusting the number of running instances based on demand.
+  - **Persistent Storage**: Managing storage that persists beyond the lifecycle of individual containers.
+  - **Networking**: Ensuring reliable and scalable networking between services.
+
+## Kubernetes History
+
+For a deeper dive into the history and development of Kubernetes, you can check out the two-part documentary by Honeypot on YouTube:
+
+- [Kubernetes: The Documentary [PART 1]](https://www.youtube.com/watch?v=BE77h7dmoQU&).
+- [Kubernetes: The Documentary [PART 2]](https://www.youtube.com/watch?v=318elIq37PE)

K8s/02-technology-overview/README.md

@@ -0,0 +1,31 @@
+# Technology Overview
+
+## Planes and Nodes
+
+The first concepts to understand with regard to kubernetes are:
+
+- **Node:** A "node" is a computer/server. Multiple nodes are joined together to form a "cluster".
+- **Control Plane:** A subset of nodes in the cluster dedicated to performing system tasks. Nodes that are part of the control plane are referred to as "control plane nodes".
+- **Data Plane:** A subset of nodes in the cluster dedicated to running user worklods. Nodes that are part of the data plane are referred to as "worker nodes".
+
+![control-and-data-planes](readme-assets/control-and-data-planes.jpg)
+
+## Kubernetes System Components
+
+Kubernetes is comprised of many smaller components:
+
+- **etcd**: Key-value store used for storing all cluster data. It serves as the source of truth for the cluster state and configuration.
+
+- **kube-apiserver**: The front end for the Kubernetes control plane.
+
+- **kube-scheduler**: Schedules pods onto the appropriate nodes based on resource availability and other constraints.
+
+- **kube-controller-manager**: Runs controller processes. Each controller is a separate process that manages routine tasks such as maintaining the desired state of resources, managing replication, handling node operations, etc...
+
+- **cloud-controller-manager**: Integrates with the underlying cloud provider (if running in one) to manage cloud-specific resources. It handles tasks such as managing load balancers, storage, and networking.
+
+- **kubelet**: An agent that runs on each worker node and ensures that containers are running in pods and manages the lifecycle of containers.
+
+- **kube-proxy**: This network proxy runs on each node and maintains network rules to allow communication to and from pods.
+
+- ![k8s-architecture](readme-assets/k8s-architecture.jpg)

K8s/02-technology-overview/readme-assets/control-and-data-planes.jpg

@@ -0,0 +1 @@
+kind-config.yaml

K8s/02-technology-overview/readme-assets/k8s-architecture.jpg

@@ -0,0 +1,96 @@
+# Installation and Setup
+
+## Dependencies
+
+### Docker Desktop
+
+Please install Docker Desktop via their instructions (Install Docker Desktop on [Mac](https://docs.docker.com/desktop/install/mac-install/), [Windows](https://docs.docker.com/desktop/install/windows-install/), or [Linux](https://docs.docker.com/desktop/install/linux-install/).)
+
+### Devbox
+
+All other software dependencies for the course are defined in the `devbox.json` and `devbox.lock` files in the root directory.
+
+Please install Devbox according to their instructions: https://www.jetify.com/devbox/docs/installing_devbox/
+
+Once installed you can run:
+
+```
+devbox shell
+```
+
+from anywhere in the repo and devbox will use Nix package manager to install a copy of all of the required software in an isolated environment.
+
+### Aliases
+
+I suggest creating the following aliases:
+
+```
+k=kubectl
+t=task
+tl='task --list-all'
+```
+
+### Autocomplete:
+
+Setting up tab completion for your shell of choice makes life much nicer:
+
+- kubectl: https://kubernetes.io/docs/reference/kubectl/generated/kubectl_completion/
+- task: https://taskfile.dev/installation/#setup-completions
+
+## Cluster Set Up
+
+This directory contains configurations and commands for setting up 3 kubernetes clusters. Any of the 3 can be used for most of the examples, with a few exceptions.
+
+### KinD
+
+Runs a kubernetes cluster locally within Docker! Great for testing and development purposes (and doesn't cost any additional money). Nearly all examples in the course can be run within this cluster, with the exception of things demonstrating cloud specific features or exposing services via public DNS.
+
+To start the cluster run:
+
+```
+devbox shell # if you haven't already
+task kind:01-generate-config
+task kind:02-create-cluster
+```
+
+### Civo Cloud
+
+Civo cloud offers a simple managed Kubernetes cluster. Their clusters provision quickly and they offer a 1 month, $250 credit for new users: https://dashboard.civo.com/signup.
+
+To create a cluster run:
+
+```
+devbox shell # if you haven't already
+civo:01-authenticate-cli
+civo:04-create-all
+civo:05-get-kubeconfig
+```
+
+To destroy the cluster run:
+
+```
+civo:06-clean-up
+```
+
+🚨 NOTE: If you have deployed additional resources (such as load balancers) that reference the VPC/Subnets/Etc... you may need to manually clean up those resources in order for this command to succeed. You should also verify that you have cleaned up all resources to avoid unwanted costs!
+
+### Google Kubernetes Engine (GKE)
+
+Google has been operating managed kubernetes clusters longer than any other cloud provider and the polish of the GKE experience shows that. They offer a 90-day, $300 free trial for new users: https://cloud.google.com/free.
+
+To create a cluster run:
+
+```
+devbox shell # if you haven't already
+gcp:01-init-cli
+gcp:07-create-all
+gcp:08-connect-to-cluster
+```
+
+To destroy the cluster run:
+
+```
+gcp:09-clean-up
+```
+
+🚨 NOTE: If you have deployed additional resources (such as load balancers) that reference the VPC/Subnets/Etc... you may need to manually clean up those resources in order for this command to succeed. You should also verify that you have cleaned up all resources to avoid unwanted costs!

K8s/03-installation-and-setup/.gitignore

@@ -0,0 +1,188 @@
+version: "3"
+
+env:
+  CLUSTER_NAME: devops-directive-kubernetes-course
+  CIVO_REGION: NYC1
+  GCP_REGION: us-central1
+  GCP_ZONE: us-central1-a
+  # Set default gum style options
+  BORDER: double
+  BORDER_FOREGROUND: "212"
+  PADDING: "1 1"
+  MARGIN: "1 1"
+
+tasks:
+  civo:00-authenticate-cli:
+    cmds:
+      - cmd: |
+          gum style "$(cat <<EOF
+          To get an API key you need to:
+          ---
+          1. Log in or create an account at https://dashboard.civo.com/
+          2. Create a team at https://dashboard.civo.com/teams
+          3. Add yourself to the team
+          4. Navigate to https://dashboard.civo.com/security to get the api key
+
+          🚨🚨🚨 NOTE: Sometimes account verification required for new accounts 
+          (so sign up before you want to use it!) 🚨🚨🚨
+          EOF
+          )"
+        silent: true
+      - civo apikey save
+      - civo apikey ls
+      - cmd: gum style "run \`civo apikey current <KEY_NAME>\` to set the current key as the default (if it is not already)"
+        silent: true
+    desc: Authenticate the Civo CLI
+
+  civo:01-create-network:
+    cmds:
+      - civo network create ${CLUSTER_NAME} --region ${CIVO_REGION}
+    desc: Create a Civo network
+
+  civo:02-create-firewall:
+    cmds:
+      - |
+        civo firewall create ${CLUSTER_NAME} \
+        --network ${CLUSTER_NAME} \
+        --create-rules false \
+        --region ${CIVO_REGION}
+      - |
+        ingress_rule_ids=$(civo firewall rule ls --region ${CIVO_REGION} ${CLUSTER_NAME} -o json | jq -r '.[] | select(.direction == "ingress") | .id')
+        for rule_id in $ingress_rule_ids; do
+          civo firewall rule remove ${CLUSTER_NAME} $rule_id -y --region ${CIVO_REGION}
+        done
+      - civo firewall rule create ${CLUSTER_NAME} --startport 80 --endport 80 --cidr 0.0.0.0/0 --protocol TCP --region ${CIVO_REGION}
+      - civo firewall rule create ${CLUSTER_NAME} --startport 443 --endport 443 --cidr 0.0.0.0/0 --protocol TCP --region ${CIVO_REGION}
+      - civo firewall rule create ${CLUSTER_NAME} --startport 6443 --endport 6443 --cidr 0.0.0.0/0 --protocol TCP --region ${CIVO_REGION}
+      - cmd: gum style "🚨 If you wanted to lock down access to the k8s api, you could instead only allow traffic on 6443 from your IP (or that of a bastion host)"
+        silent: true
+    desc: Create a Civo firewall and set up rules
+
+  civo:03-create-cluster:
+    cmds:
+      - |
+        civo kubernetes create ${CLUSTER_NAME} \
+          --region ${CIVO_REGION} \
+          --network ${CLUSTER_NAME} \
+          --existing-firewall ${CLUSTER_NAME} \
+          --nodes 2 \
+          --size g4s.kube.medium \
+          --remove-applications "traefik2-nodeport" \
+          --wait
+    desc: Create a Civo Kubernetes cluster
+
+  civo:04-create-all:
+    cmds:
+      - task: civo:01-create-network
+      - task: civo:02-create-firewall
+      - task: civo:03-create-cluster
+    desc: Create the Civo network, firewall, and cluster in sequence
+
+  civo:05-get-kubeconfig:
+    cmds:
+      - civo kubernetes config ${CLUSTER_NAME} --region ${CIVO_REGION} --save --switch
+    desc: Get kubeconfig for the cluster
+
+  civo:06-clean-up:
+    cmds:
+      - civo kubernetes delete ${CLUSTER_NAME} --region ${CIVO_REGION} -y
+      - cmd: gum style "There is some delay on the civo side from cluster being deleted to it being removed from the firewall rule usage"
+        silent: true
+      - sleep 10
+      - civo firewall delete ${CLUSTER_NAME} --region ${CIVO_REGION} -y
+      - civo network delete ${CLUSTER_NAME} --region ${CIVO_REGION} -y
+    desc: Clean up the Civo Kubernetes cluster and associated resources
+
+  gcp:01-init-cli:
+    cmds:
+      - gcloud init
+    desc: "Authenticate and configure the gcloud CLI"
+
+  gcp:02-enable-apis:
+    cmds:
+      - |
+        gcloud services enable \
+          compute.googleapis.com \
+          container.googleapis.com \
+          cloudresourcemanager.googleapis.com \
+          iam.googleapis.com \
+          secretmanager.googleapis.com \
+          servicemanagement.googleapis.com \
+          serviceusage.googleapis.com
+    desc: "Enable necessary APIs"
+
+  gcp:03-set-region-and-zone:
+    cmds:
+      - gcloud config set compute/region ${GCP_REGION}
+      - gcloud config set compute/zone ${GCP_ZONE}
+    desc: "Set default region and zone"
+
+  gcp:04-create-vpc:
+    cmds:
+      - gcloud compute networks create ${CLUSTER_NAME} --subnet-mode=custom
+    desc: "Create VPC"
+
+  gcp:05-create-subnet:
+    cmds:
+      - |
+        gcloud compute networks subnets create subnet-1 \
+          --network=${CLUSTER_NAME} \
+          --region=${GCP_REGION} \
+          --range=10.0.0.0/20
+    desc: "Create subnet"
+
+  gcp:06-create-cluster:
+    desc: "Create GKE cluster"
+    vars:
+      GCP_PROJECT_ID: kubernetes-course-424917
+    cmds:
+      - |
+        gcloud container clusters create ${CLUSTER_NAME} \
+          --zone ${GCP_ZONE} \
+          --network ${CLUSTER_NAME} \
+          --subnetwork subnet-1 \
+          --machine-type e2-standard-2 \
+          --num-nodes 2 \
+          --gateway-api=standard \
+          --workload-pool={{.GCP_PROJECT_ID}}.svc.id.goog
+
+  gcp:07-create-all:
+    cmds:
+      - task: gcp:02-enable-apis
+      - task: gcp:03-set-region-and-zone
+      - task: gcp:04-create-vpc
+      - task: gcp:05-create-subnet
+      - task: gcp:06-create-cluster
+    desc: Create the GCP network, subnet, firewall rules, and cluster in sequence
+
+  gcp:09-clean-up:
+    cmds:
+      - gcloud container clusters delete ${CLUSTER_NAME} --zone ${GCP_ZONE} --quiet
+      - gcloud compute networks subnets delete subnet-1 --region=${GCP_REGION} --quiet
+      - gcloud compute networks delete ${CLUSTER_NAME} --quiet
+    desc: Delete the GCP network, subnet, firewall rules, and cluster in reverse sequence
+
+  gcp:08-connect-to-cluster:
+    cmds:
+      - gcloud container clusters get-credentials ${CLUSTER_NAME} --zone ${GCP_ZONE}
+    desc: "Connect to the GKE cluster"
+
+  kind:01-generate-config:
+    cmds:
+      - REPLACE_WITH_ABSOLUTE_PATH=${PWD} envsubst < kind-config.yaml.TEMPLATE > kind-config.yaml
+    desc: "Generate kind config with local absolute paths for PV mounts"
+
+  kind:02-create-cluster:
+    cmds:
+      - kind create cluster --config kind-config.yaml
+    desc: Create a Kubernetes cluster using kind
+
+  kind:03-run-cloud-provider-kind:
+    desc: "Run sigs.k8s.io/cloud-provider-kind@latest to enable load balancer services with KinD"
+    cmds:
+      - sudo cloud-provider-kind
+
+  kind:04-delete-cluster:
+    cmds:
+      - kind delete cluster
+    desc: Delete and existing a kind Kubernetes cluster

K8s/03-installation-and-setup/README.md

@@ -0,0 +1 @@
+👋 (#1)

K8s/03-installation-and-setup/Taskfile.yaml

@@ -0,0 +1 @@
+👋 (#2)

K8s/03-installation-and-setup/kind-bind-mount-1/hello-from-host

@@ -0,0 +1,13 @@
+# three node (two workers) cluster config
+kind: Cluster
+apiVersion: kind.x-k8s.io/v1alpha4
+nodes:
+  - role: control-plane
+  - role: worker
+    extraMounts:
+      - hostPath: ${REPLACE_WITH_ABSOLUTE_PATH}/kind-bind-mount-1
+        containerPath: /some/path/in/container
+  - role: worker
+    extraMounts:
+      - hostPath: ${REPLACE_WITH_ABSOLUTE_PATH}/kind-bind-mount-2
+        containerPath: /some/path/in/container