Terraform IaC for 3edges on AWS

.gitignore

@@ -1,6 +1,6 @@
 .terraform*
 terraform.*
-!terraform.tfvars
+!terraform.tfvars.sample
 plan.bkp
 localhost/.env
 .vscode

README.md

@@ -1,13 +1,18 @@
 # 3Edges Deployment
 
-This repository contains Terraform scripts to deploy the 3Edges software across cloud provider. These scripts automate the infrastructure setup, ensuring a seamless deployment process of 3Edges into your cloud environment. 
+This repository contains Terraform scripts to deploy the 3Edges software across cloud providers. These scripts automate the infrastructure setup, ensuring a seamless deployment process of 3Edges into your cloud environment.
+
+## Table of Contents
+1. [Features](#features)
+2. [Customization](#customization)
+3. [Deployment Documentation](#deployment-documentation)
 
 ## Features
 - Automated Cloud Infrastructure: Deploys and configures cloud resources for 3Edges, such as VPCs, load balancers, Kubernetes clusters, databases, and more. 
 
 - Scalable and Secure: Includes cloud-native best practices for security, scalability, and reliability.
 
-- Easy Configuration: Customize settings like region, cluster names, and database credentials through a simple configuration file (terraform.tfvars).
+- Easy Configuration: Customize settings like region, cluster names, and database credentials through a simple configuration file (`terraform.tfvars`).
 
 ## Customization
 You can modify various settings, just to name a few:
@@ -16,8 +21,8 @@ You can modify various settings, just to name a few:
 - Networking settings
 - Admin emails and passwords
 
-These can be set in the ```terraform.tfvars file```
+These can be set in the `terraform.tfvars` file.
 
 ## Deployment Documentation
 
-[AWS](terraform/aws/README.md)
+[AWS](terraform/aws/README.md)

terraform/aws/README.md

@@ -1,12 +1,33 @@
 # AWS Deployment
 
+## Table of Contents
+1. [Prerequisites](#prerequisites)
+2. [Configuring Domain with Route 53 Name Servers](#configuring-domain-with-route-53-name-servers)
+3. [On AWS](#on-aws)
+   - [EC2 Instance Creation (Terraform Host)](#ec2-instance-creation-terraform-host)
+   - [Install Prerequisites on EC2 Instance](#install-prerequisites-on-ec2-instance)
+   - [Create an S3 Bucket for Terraform Statefile](#create-an-s3-bucket-for-terraform-to-store-the-statefile)
+   - [Create an IAM User for Terraform](#create-an-iam-user-for-terraform)
+   - [Attach Policies to IAM User](#attach-policies-to-iam-user)
+   - [Create Access Keys for IAM User](#create-access-keys-for-the-iam-user)
+4. [On Terraform Host (EC2 instance)](#on-terraform-host-ec2-instance)
+   - [Configure AWS CLI](#configure-aws-cli)
+   - [Clone the GitHub Repository](#clone-the-github-repository)
+   - [Modify the Backend Configuration](#modify-the-backend-configuration)
+   - [Modify the Terraform Configuration](#modify-the-terraform-configuration)
+5. [Deploy 3Edges to AWS](#deploy-3edges-to-aws)
+6. [Decommission the 3Edges deployment](#decommission-the-3edges-deployment)
+7. [Release Notes - Post Release Actions](#release-notes---post-release-actions)
+
 ## Prerequisites
 - Terraform
 
 - AWS CLI (For AWS deployment)
 
 - Docker 
 
+- kubectl
+
 - Git
 
 - Neo4j Database access
@@ -116,6 +137,14 @@ sudo usermod -aG docker $USER
 ```
 **Make sure to log out and log back in or restart your terminal session for the changes to take effect**
 
+---
+[kubectl Installation Link](https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/#install-using-other-package-management)
+
+```
+snap install kubectl --classic
+kubectl version --client
+```
+
 ---
 Git installation : Usually Ubuntu EC2 instance comes with git installed in the OS. In-case if its not installed run the following command  
 ```
@@ -124,7 +153,6 @@ sudo apt-get install git
 git help
 ```
 
-
 ## Create an S3 Bucket for Terraform to store the statefile 
 
 NOTE: ca-west-1 (Calgary) region is not supported as S3 Bucket Backend. Try to use ca-central-1 (Canada Central)
@@ -256,22 +284,19 @@ Clone the repository containing the Terraform configuration:
 
 ## Modify the Backend Configuration 
 
-Update the `backend.tf` file with your S3 bucket details
+Rename backend.tf.sample to backend.tf. Update the `backend.tf` file with your S3 bucket details
 
 ```
-3edges-deployments/terraform/aws/backend.tf
+cp /path/to/3edges-deployments/terraform/aws/backend.tf.sample /path/to/3edges-deployments/terraform/aws/backend.tf 
 ```
 
 ## Modify the Terraform Configuration
 
-Update the `terraform.tfvars` file with your environment details
+Rename terraform.tfvars.sample to terraform.tfvars. Update the `terraform.tfvars` file with your environment details
 
 ```
-3edges-deployments/terraform/aws/terraform.tfvars
+cp /path/to/3edges-deployments/terraform/aws/terraform.tfvars.sample /path/to/3edges-deployments/terraform/aws/terraform.tfvars
 ```
-If you are deploying 3Edges on a sub-domain
-
-1. Make sure you have Route53 record for your root domain and Nameservers added to your Domain Controller.
 
 ## Deploy 3Edges to AWS
 
@@ -282,7 +307,59 @@ After making the necessary changes, run the deployment script
 # ./run.sh
 ```
 
-
 Once the DNS propagation is successful, open your configured domain in the browser, you will a see Login Page
 
-![](./docs/images/login_page.png)
+![](./docs/images/login_page.png)
+
+## Decommission the 3Edges deployment
+
+Warning: Running terraform destroy is a destructive operation that will permanently delete all resources created by your Terraform configuration. Use this command with caution, and ensure you have backups or snapshots of any critical data before proceeding.
+
+#### Destroy Terraform-managed Resources
+
+Run the following command to remove all resources defined in the Terraform state. Double-check that you are in the correct environment before executing this command.
+
+```bash
+    cd /path/to/3edges-deployments/terraform/aws
+
+    terraform destroy
+```
+
+When prompted, confirm the destruction process by typing "yes."
+
+## Release Notes - Post Release Actions
+
+After a new release of 3Edges, follow these steps to update your deployment. This ensures that any updates to configuration, secrets, or deployments are applied to your environment.
+
+1. **SSH to EC2 Instance**  
+   - Access the EC2 instance that is running as the Terraform host by connecting via SSH.
+
+2. **Navigate to the Repository**  
+   - Once connected, navigate to the directory where the Git repository is located (e.g., `/home/ubuntu/3edges-deployments/`).
+   ```bash
+   cd /path/to/3edges-deployments/
+   ```
+3. **Pull the latest changes**
+    - Fetch the most recent updates from the Git repository to ensure you have the latest code, configurations, and scripts
+    ```bash
+      git pull origin main
+    ```
+4. Run the Terraform Bash Script
+   - Execute the Terraform script to apply any infrastructure updates required for the release.
+   ```bash
+   cd /path/to/3edges-deployments/terraform/aws
+   ./run.sh
+   ```
+
+5. **Authenticate to EKS Cluster**
+   - Set up the connection to the EKS cluster by executing the following script
+   ```bash
+   /path/to/3edges-deployments/terraform/aws/scripts/authn-k8s-cluster.sh
+   ```
+
+6. Restart the Deployments and ensure the pods are running successfully
+    ```bash
+    /path/to/3edges-deployments/terraform/aws/scripts/restart-3edges-controlplane.sh
+    ```
+
+

terraform/aws/main.tf

@@ -23,6 +23,7 @@ module "cluster" {
   source                                 = "./modules/cluster"
   eks_cluster                            = var.eks_cluster
   eks_node_group                         = var.eks_node_group
+  eks_security_group_id                  = module.vpc.eks_security_group_id
   eks_subnet                             = module.vpc.eks_subnet
   iam_role_arn                           = module.iam.iam_role_arn
   aws_iam_role                           = module.iam.aws_iam_role

terraform/aws/modules/cluster/main.tf

@@ -50,8 +50,6 @@ resource "aws_eks_node_group" "eks_node_group" {
 resource "aws_eks_addon" "vpc_cni" {
   cluster_name  = aws_eks_cluster.eks_cluster.name
   addon_name    = "vpc-cni"
-  # addon_version = "v1.18.1-eksbuild.3"
-  # addon_version = "v1.18.5-eksbuild.1"
 
   depends_on = [
     aws_eks_cluster.eks_cluster,
@@ -62,8 +60,6 @@ resource "aws_eks_addon" "vpc_cni" {
 resource "aws_eks_addon" "kube_proxy" {
   cluster_name  = aws_eks_cluster.eks_cluster.name
   addon_name    = "kube-proxy"
-  # addon_version = "v1.30.0-eksbuild.3"
-  # addon_version = "v1.31.0-eksbuild.5"
 
   depends_on = [
     aws_eks_addon.vpc_cni,
@@ -75,7 +71,6 @@ resource "aws_eks_addon" "kube_proxy" {
 resource "aws_eks_addon" "eks_pod_identity" {
   cluster_name  = aws_eks_cluster.eks_cluster.name
   addon_name    = "eks-pod-identity-agent"
-  # addon_version = "v1.3.0-eksbuild.1"
 
   depends_on = [
     aws_eks_addon.kube_proxy,
@@ -87,8 +82,6 @@ resource "aws_eks_addon" "eks_pod_identity" {
 resource "aws_eks_addon" "coredns" {
   cluster_name  = aws_eks_cluster.eks_cluster.name
   addon_name    = "coredns"
-  # addon_version = "v1.11.1-eksbuild.9"
-  # addon_version = "v1.11.3-eksbuild.1"
 
   depends_on = [
     aws_eks_addon.eks_pod_identity,

terraform/aws/modules/cluster/variables.tf

@@ -4,6 +4,8 @@ variable "eks_subnet" {
   type = list(any)
 }
 
+variable "eks_security_group_id" {}
+
 variable "eks_node_group" {}
 
 variable "iam_role_arn" {}

terraform/aws/modules/kubernetes/deployments/client/api_route53.tf

@@ -3,9 +3,6 @@ locals {
   domain_parts = split(".", var.hosted_zone)
   root_domain  = length(local.domain_parts) > 2 ? join(".", slice(local.domain_parts, length(local.domain_parts) - 2, length(local.domain_parts))) : var.hosted_zone
 
-  # Condition to determine if the hosted zone is a root domain or not
-  # For example, check if the hosted zone is equal to root_domain
-#   is_root_domain = var.hosted_zone == local.root_domain
 }
 
 # Look up the existing hosted zone for the parent domain
@@ -14,16 +11,9 @@ data "aws_route53_zone" "parent_domain" {
   private_zone = false
 }
 
-# # Conditionally create a new hosted zone if the parent domain doesn't exist
-# resource "aws_route53_zone" "hosted_zone" {
-#   # count = length(data.aws_route53_zone.parent_domain.id != "" ? [] : [1])  # Only create if not found
-#   count = length(data.aws_route53_zone.parent_domain) == 0 ? 1 : 0  # Only create if not found
-#   name = local.root_domain
-# }
 
 # Use the correct zone ID (either existing or newly created)
 locals {
-  # zone_id = length(data.aws_route53_zone.parent_domain) > 0 ? data.aws_route53_zone.parent_domain[0].zone_id : aws_route53_zone.hosted_zone[0].id
   zone_id = data.aws_route53_zone.parent_domain.zone_id
 }
 

terraform/aws/modules/kubernetes/deployments/client/pods.tf

@@ -2,6 +2,9 @@ resource "kubernetes_deployment" "deployment_dataproxy" {
   metadata {
     name      = "${local.api_name}-proxy"
     namespace = "3edges"
+    labels = {
+      type = "3edges_client_api"
+     }
   }
 
   spec {
@@ -23,7 +26,7 @@ resource "kubernetes_deployment" "deployment_dataproxy" {
       spec {
         container {
           name              = "${local.api_name}-proxy"
-          image             = "indykite/3edges-dataproxy:qa"
+          image             = "indykite/3edges-dataproxy:latest"
           image_pull_policy = "Always"
 
           volume_mount {
@@ -63,6 +66,9 @@ resource "kubernetes_deployment" "deployment_authorization" {
   metadata {
     name      = "${local.api_name}-authz"
     namespace = "3edges"
+    labels = {
+      type = "3edges_client_api"
+     }
   }
 
   spec {
@@ -84,7 +90,7 @@ resource "kubernetes_deployment" "deployment_authorization" {
       spec {
         container {
           name              = "${local.api_name}-authz"
-          image             = "indykite/3edges-authorization:qa"
+          image             = "indykite/3edges-authorization:latest"
           image_pull_policy = "Always"
 
           volume_mount {
@@ -125,6 +131,9 @@ resource "kubernetes_deployment" "deployment_authorization_csp" {
   metadata {
     name      = "${local.api_name}-authz-csp"
     namespace = "3edges"
+    labels = {
+      type = "3edges_client_api"
+     }
   }
 
   spec {
@@ -146,7 +155,7 @@ resource "kubernetes_deployment" "deployment_authorization_csp" {
       spec {
         container {
           name              = "${local.api_name}-authz-csp"
-          image             = "indykite/3edges-authorization-csp:qa"
+          image             = "indykite/3edges-authorization-csp:latest"
           image_pull_policy = "Always"
 
           volume_mount {
@@ -186,6 +195,9 @@ resource "kubernetes_deployment" "deployment_dashboard" {
   metadata {
     name      = "${local.api_name}-dashboard"
     namespace = "3edges"
+    labels = {
+      type = "3edges_client_api"
+     }
   }
 
   spec {
@@ -207,7 +219,7 @@ resource "kubernetes_deployment" "deployment_dashboard" {
       spec {
         container {
           name              = "${local.api_name}-dashboard"
-          image             = "indykite/3edges-dashboard:qa"
+          image             = "indykite/3edges-dashboard:latest"
           image_pull_policy = "Always"
 
           volume_mount {
@@ -249,6 +261,9 @@ resource "kubernetes_deployment" "deployment_client_idp" {
   metadata {
     name      = "${local.api_name}-idp"
     namespace = "3edges"
+    labels = {
+      type = "3edges_client_api"
+     }
   }
 
   spec {
@@ -270,7 +285,7 @@ resource "kubernetes_deployment" "deployment_client_idp" {
       spec {
         container {
           name              = "${local.api_name}-idp"
-          image = "indykite/3edges-idp:qa"
+          image = "indykite/3edges-idp:latest"
           image_pull_policy = "Always"
 
           volume_mount {

terraform/aws/modules/kubernetes/deployments/configmap.tf

@@ -162,6 +162,10 @@ resource "kubernetes_config_map" "cluster_config" {
     API_NAME              = local.api_name
     hostedZoneID          = var.aws_route53_zone_hosted_zone_id
     DOMAIN                = var.hosted_zone
+    REPO_PROD             = "indykite"
+    TAG_PROD              = "latest"
+    REPO_QA               = "us-docker.pkg.dev/edges-305901/gcr.io"
+    TAG_QA                = "qa"
 
   }