diff --git a/README.md b/README.md index 58db76a..0bee3f2 100644 --- a/README.md +++ b/README.md @@ -116,6 +116,7 @@ Security scanning is graciously provided by Prowler. Proowler is the leading ful | Name | Source | Version | |------|--------|---------| +| [backup\_restore](#module\_backup\_restore) | ./modules/db-backup-restore | n/a | | [cw\_sns\_slack](#module\_cw\_sns\_slack) | ./lambda | n/a | | [db](#module\_db) | terraform-aws-modules/rds/aws | 6.1.0 | | [db\_replica](#module\_db\_replica) | terraform-aws-modules/rds/aws | 6.1.0 | @@ -152,9 +153,12 @@ Security scanning is graciously provided by Prowler. Proowler is the leading ful | [availability\_zone](#input\_availability\_zone) | The Availability Zone of the RDS instance | `string` | `null` | no | | [backup\_retention\_period](#input\_backup\_retention\_period) | The number of days to retain backups for | `number` | `5` | no | | [backup\_window](#input\_backup\_window) | The time window during which database backups are performed | `string` | `"03:00-06:00"` | no | +| [bucket\_provider\_type](#input\_bucket\_provider\_type) | Choose what type of provider you want (s3, gcs) | `string` | `"s3"` | no | | [cloudwatch\_log\_group\_retention\_in\_days](#input\_cloudwatch\_log\_group\_retention\_in\_days) | The number of days to retain CloudWatch logs for the database instance | `number` | `7` | no | | [cloudwatch\_metric\_alarms\_enabled](#input\_cloudwatch\_metric\_alarms\_enabled) | Boolean flag to enable/disable CloudWatch metrics alarms | `bool` | `false` | no | +| [cluster\_name](#input\_cluster\_name) | Specifies the name of the EKS cluster to deploy the MySQL application on. | `string` | `""` | no | | [create\_db\_subnet\_group](#input\_create\_db\_subnet\_group) | Whether to create a database subnet group | `bool` | `true` | no | +| [create\_namespace](#input\_create\_namespace) | Specify whether or not to create the namespace if it does not already exist. Set it to true to create the namespace. | `string` | `false` | no | | [create\_random\_password](#input\_create\_random\_password) | Whether to create a random password for the primary database cluster | `bool` | `false` | no | | [custom\_user\_password](#input\_custom\_user\_password) | Custom password for the RDS master user | `string` | `""` | no | | [cw\_sns\_topic\_arn](#input\_cw\_sns\_topic\_arn) | The username to use when sending notifications to Slack. | `string` | `""` | no | @@ -177,7 +181,12 @@ Security scanning is graciously provided by Prowler. Proowler is the leading ful | [master\_username](#input\_master\_username) | The username for the RDS primary cluster | `string` | `""` | no | | [max\_allocated\_storage](#input\_max\_allocated\_storage) | The Maximum storage capacity for the database value after autoscaling | `number` | `null` | no | | [multi\_az](#input\_multi\_az) | Enables multi-AZ for disaster recovery | `bool` | `false` | no | +| [mysqldb\_backup\_config](#input\_mysqldb\_backup\_config) | configuration options for MySQL database backups. It includes properties such as the S3 bucket URI, the S3 bucket region, and the cron expression for full backups. | `map(string)` | 
{
  "bucket_uri": "",
  "cron_for_full_backup": "",
  "mysql_database_name": "",
  "s3_bucket_region": ""
}
| no | +| [mysqldb\_backup\_enabled](#input\_mysqldb\_backup\_enabled) | Specifies whether to enable backups for MySQL database. | `bool` | `false` | no | +| [mysqldb\_restore\_config](#input\_mysqldb\_restore\_config) | Configuration options for restoring dump to the MySQL database. | `any` | 
{
  "bucket_uri": "",
  "file_name": "",
  "s3_bucket_region": ""
}
| no | +| [mysqldb\_restore\_enabled](#input\_mysqldb\_restore\_enabled) | Specifies whether to enable restoring dump to the MySQL database. | `bool` | `false` | no | | [name](#input\_name) | The name of the RDS instance | `string` | `""` | no | +| [namespace](#input\_namespace) | Name of the Kubernetes namespace where the MYSQL deployment will be deployed. | `string` | `"mysqldb"` | no | | [ok\_actions](#input\_ok\_actions) | The list of actions to execute when this alarm transitions into an OK state from any other state. Each action is specified as an Amazon Resource Number (ARN) | `list(string)` | `[]` | no | | [port](#input\_port) | The port for the database | `number` | `3306` | no | | [publicly\_accessible](#input\_publicly\_accessible) | Specifies whether the database is publicly accessible over the internet | `bool` | `false` | no | diff --git a/examples/complete-mysql/README.md b/examples/complete-mysql/README.md index c60d0d7..ccc8a68 100644 --- a/examples/complete-mysql/README.md +++ b/examples/complete-mysql/README.md @@ -26,7 +26,7 @@ This example will be very useful for users who are new to a module and want to q | Name | Source | Version | |------|--------|---------| | [kms](#module\_kms) | terraform-aws-modules/kms/aws | n/a | -| [rds-mysql](#module\_rds-mysql) | terraform-aws-modules/rds/aws | n/a | +| [rds-mysql](#module\_rds-mysql) | squareops/rds-mysql/aws | 1.1.7 | | [vpc](#module\_vpc) | squareops/vpc/aws | n/a | ## Resources @@ -34,6 +34,8 @@ This example will be very useful for users who are new to a module and want to q | Name | Type | |------|------| | [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source | +| [aws_eks_cluster.cluster](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster) | data source | +| [aws_eks_cluster_auth.cluster](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) | data source | | [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source | ## Inputs diff --git a/examples/complete-mysql/helm/values.yaml b/examples/complete-mysql/helm/values.yaml index 7371739..bfd3de9 100644 --- a/examples/complete-mysql/helm/values.yaml +++ b/examples/complete-mysql/helm/values.yaml @@ -45,4 +45,4 @@ restorejob: cpu: 50m limits: memory: 200Mi - cpu: 100m \ No newline at end of file + cpu: 100m diff --git a/examples/complete-mysql/main.tf b/examples/complete-mysql/main.tf index 6a4cae3..7ef54d5 100644 --- a/examples/complete-mysql/main.tf +++ b/examples/complete-mysql/main.tf @@ -9,7 +9,8 @@ locals { mysql_instance_class = "db.t3.micro" mysql_engine_version = "8.0.32" major_engine_version = "8.0" - allowed_security_groups = ["sg-xxxxxxxxxxxxxx"] + cluster_name = "" + allowed_security_groups = ["sg-xxxxxxxxxxx"] vpc_cidr = "10.10.0.0/16" current_identity = data.aws_caller_identity.current.arn custom_user_password = "" @@ -101,6 +102,7 @@ module "vpc" { module "rds-mysql" { source = "squareops/rds-mysql/aws" + version = "1.1.7" name = local.name vpc_id = module.vpc.vpc_id family = local.family @@ -134,21 +136,19 @@ module "rds-mysql" { slack_channel = "mysql-notification" slack_webhook_url = "https://hooks/xxxxxxxx" custom_user_password = local.custom_user_password - cluster_name = "" # cluster name - namespace = local.namespace - create_namespace = local.create_namespace + cluster_name = local.cluster_name # cluster name + namespace = local.namespace # namespace + create_namespace = local.create_namespace # create namespace mysqldb_backup_enabled = false bucket_provider_type = "s3" mysqldb_backup_config = { - mysql_database_name = "" - s3_bucket_region = "us-west-1" - cron_for_full_backup = "0 */6 * * *" - bucket_uri = "s3://mysql-rds-backup-store/" + mysql_database_name = "" # If you want to backup all databases, leave it empty or specify the database name + cron_for_full_backup = "*/2 * * * *" # Cron expression for full backup + bucket_uri = "s3://my-backup-dumps-databases/" # S3 bucket URI } mysqldb_restore_enabled = false mysqldb_restore_config = { - bucket_uri = "s3://mysql-rds-backup-store/mysqldump_20240723_074237.zip" - file_name = "mysqldump_20240723_074237.zip" - s3_bucket_region = "us-west-1" + bucket_uri = "s3://my-backup-dumps-databases/mysqldump_20250303_141601.zip" # S3 bucket URI with the backup file + file_name = "mysqldump_20250303_141601.zip" # Backup file name } } diff --git a/helm/values/backup/values.yaml b/helm/values/backup/values.yaml index 676ffd8..71326d0 100644 --- a/helm/values/backup/values.yaml +++ b/helm/values/backup/values.yaml @@ -1,7 +1,6 @@ ## Enable Full backup backup: bucket_uri: ${bucket_uri} - aws_default_region: ${s3_bucket_region} cron_for_full_backup: "${cron_for_full_backup}" database_name: "${mysql_database_name}" database_endpoint: "${db_endpoint}" diff --git a/helm/values/restore/values.yaml b/helm/values/restore/values.yaml index 16af1f6..1543865 100644 --- a/helm/values/restore/values.yaml +++ b/helm/values/restore/values.yaml @@ -1,11 +1,10 @@ restore: file_name: ${file_name} bucket_uri: ${bucket_uri} - aws_default_region: ${s3_bucket_region} - bucket_provider : ${bucket_provider_type} database_endpoint: "${db_endpoint}" database_password: "${db_password}" database_user: "${db_username}" + provider: "${bucket_provider_type}" auth: username: "${custom_user_username}" diff --git a/main.tf b/main.tf index a1fee2d..5021f6b 100644 --- a/main.tf +++ b/main.tf @@ -327,7 +327,6 @@ module "backup_restore" { db_username = module.db.db_instance_username db_password = var.custom_user_password != "" ? var.custom_user_password : nonsensitive(random_password.master[0].result) mysql_database_name = var.mysqldb_backup_config.mysql_database_name - s3_bucket_region = var.mysqldb_backup_config.s3_bucket_region cron_for_full_backup = var.mysqldb_backup_config.cron_for_full_backup bucket_uri = var.mysqldb_backup_config.bucket_uri db_endpoint = replace(module.db.db_instance_endpoint, ":3306", "") @@ -335,11 +334,10 @@ module "backup_restore" { mysqldb_restore_enabled = var.mysqldb_restore_enabled mysqldb_restore_config = { - db_endpoint = replace(module.db.db_instance_endpoint, ":3306", "") - db_username = module.db.db_instance_username - db_password = var.custom_user_password != "" ? var.custom_user_password : nonsensitive(random_password.master[0].result) - bucket_uri = var.mysqldb_restore_config.bucket_uri - file_name = var.mysqldb_restore_config.file_name - s3_bucket_region = var.mysqldb_restore_config.s3_bucket_region + db_endpoint = replace(module.db.db_instance_endpoint, ":3306", "") + db_username = module.db.db_instance_username + db_password = var.custom_user_password != "" ? var.custom_user_password : nonsensitive(random_password.master[0].result) + bucket_uri = var.mysqldb_restore_config.bucket_uri + file_name = var.mysqldb_restore_config.file_name } } diff --git a/modules/db-backup-restore/README.md b/modules/db-backup-restore/README.md new file mode 100644 index 0000000..91715f2 --- /dev/null +++ b/modules/db-backup-restore/README.md @@ -0,0 +1,57 @@ +# db-backup-restore + + +## Requirements + +No requirements. + +## Providers + +| Name | Version | +|------|---------| +| [aws](#provider\_aws) | n/a | +| [helm](#provider\_helm) | n/a | +| [kubernetes](#provider\_kubernetes) | n/a | + +## Modules + +No modules. + +## Resources + +| Name | Type | +|------|------| +| [aws_iam_role.mysql_backup_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource | +| [aws_iam_role.mysql_restore_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource | +| [helm_release.mysqldb_backup](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) | resource | +| [helm_release.mysqldb_restore](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) | resource | +| [kubernetes_namespace.mysqldb](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace) | resource | +| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source | +| [aws_eks_cluster.kubernetes_cluster](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster) | data source | + +## Inputs + +| Name | Description | Type | Default | Required | +|------|-------------|------|---------|:--------:| +| [azure\_container\_name](#input\_azure\_container\_name) | Azure container name | `string` | `""` | no | +| [azure\_storage\_account\_key](#input\_azure\_storage\_account\_key) | Azure storage account key | `string` | `""` | no | +| [azure\_storage\_account\_name](#input\_azure\_storage\_account\_name) | Azure storage account name | `string` | `""` | no | +| [bucket\_provider\_type](#input\_bucket\_provider\_type) | Choose what type of provider you want (s3, gcs) | `string` | `"s3"` | no | +| [cluster\_name](#input\_cluster\_name) | Specifies the name of the EKS cluster to deploy the MySQL application on. | `string` | `""` | no | +| [create\_namespace](#input\_create\_namespace) | Specify whether or not to create the namespace if it does not already exist. Set it to true to create the namespace. | `string` | `false` | no | +| [iam\_role\_arn\_backup](#input\_iam\_role\_arn\_backup) | IAM role ARN for backup (AWS) | `string` | `""` | no | +| [iam\_role\_arn\_restore](#input\_iam\_role\_arn\_restore) | IAM role ARN for restore (AWS) | `string` | `""` | no | +| [mysqldb\_backup\_config](#input\_mysqldb\_backup\_config) | configuration options for MySQL database backups. It includes properties such as the S3 bucket URI, the S3 bucket region, and the cron expression for full backups. | `map(string)` | 
{
  "bucket_uri": "",
  "cron_for_full_backup": "",
  "mysql_database_name": "",
  "s3_bucket_region": ""
}
| no | +| [mysqldb\_backup\_enabled](#input\_mysqldb\_backup\_enabled) | Specifies whether to enable backups for MySQL database. | `bool` | `false` | no | +| [mysqldb\_permission](#input\_mysqldb\_permission) | access | `bool` | `false` | no | +| [mysqldb\_restore\_config](#input\_mysqldb\_restore\_config) | Configuration options for restoring dump to the MySQL database. | `any` | 
{
  "bucket_uri": "",
  "file_name": "",
  "s3_bucket_region": ""
}
| no | +| [mysqldb\_restore\_enabled](#input\_mysqldb\_restore\_enabled) | Specifies whether to enable restoring dump to the MySQL database. | `bool` | `false` | no | +| [name](#input\_name) | Name identifier for module to be added as suffix to resources | `string` | `"test"` | no | +| [namespace](#input\_namespace) | Name of the Kubernetes namespace where the MYSQL deployment will be deployed. | `string` | `"mysqldb"` | no | +| [service\_account\_backup](#input\_service\_account\_backup) | Service account for backup (GCP) | `string` | `""` | no | +| [service\_account\_restore](#input\_service\_account\_restore) | Service account for restore (GCP) | `string` | `""` | no | + +## Outputs + +No outputs. + diff --git a/modules/db-backup-restore/backup/templates/cronjob.yaml b/modules/db-backup-restore/backup/templates/cronjob.yaml index bf41c83..8f6ef92 100644 --- a/modules/db-backup-restore/backup/templates/cronjob.yaml +++ b/modules/db-backup-restore/backup/templates/cronjob.yaml @@ -20,7 +20,7 @@ spec: serviceAccountName: sa-mysql-backup containers: - name: backup-mysqldb - image: amanmall/mysqldb-backup:v1 + image: squareops01/rds-mysql-backup:v1 imagePullPolicy: Always env: - name: MYSQL_HOST @@ -34,7 +34,5 @@ spec: - name: MYSQL_BUCKET_URI value: {{ .Values.backup.bucket_uri }} - name: BUCKET_PROVIDER - value: {{ .Values.backup.provider }} - - name: AWS_DEFAULT_REGION - value: {{ .Values.backup.aws_default_region}} + value: {{ .Values.backup.provider }} resources: {{ .Values.backupjob.resources | toYaml | nindent 12 }} diff --git a/modules/db-backup-restore/main.tf b/modules/db-backup-restore/main.tf index d553ea6..83ab75f 100644 --- a/modules/db-backup-restore/main.tf +++ b/modules/db-backup-restore/main.tf @@ -20,7 +20,6 @@ resource "helm_release" "mysqldb_backup" { db_endpoint = var.bucket_provider_type == "s3" ? var.mysqldb_backup_config.db_endpoint : "", db_password = var.bucket_provider_type == "s3" ? var.mysqldb_backup_config.db_password : "", db_username = var.bucket_provider_type == "s3" ? var.mysqldb_backup_config.db_username : "", - s3_bucket_region = var.bucket_provider_type == "s3" ? var.mysqldb_backup_config.s3_bucket_region : "", cron_for_full_backup = var.mysqldb_backup_config.cron_for_full_backup, custom_user_username = "admin", bucket_provider_type = var.bucket_provider_type, @@ -45,7 +44,6 @@ resource "helm_release" "mysqldb_restore" { templatefile("${path.module}/../../helm/values/restore/values.yaml", { bucket_uri = var.mysqldb_restore_config.bucket_uri, file_name = var.mysqldb_restore_config.file_name, - s3_bucket_region = var.bucket_provider_type == "s3" ? var.mysqldb_restore_config.s3_bucket_region : "", db_endpoint = var.bucket_provider_type == "s3" ? var.mysqldb_restore_config.db_endpoint : "", db_password = var.bucket_provider_type == "s3" ? var.mysqldb_restore_config.db_password : "", db_username = var.bucket_provider_type == "s3" ? var.mysqldb_restore_config.db_username : "", @@ -58,7 +56,3 @@ resource "helm_release" "mysqldb_restore" { }) ] } - - - - diff --git a/modules/db-backup-restore/restore/templates/job.yaml b/modules/db-backup-restore/restore/templates/job.yaml index 18411aa..65ac580 100644 --- a/modules/db-backup-restore/restore/templates/job.yaml +++ b/modules/db-backup-restore/restore/templates/job.yaml @@ -9,7 +9,7 @@ spec: serviceAccountName: sa-mysql-restore containers: - name: restore-mysqldb - image: squareops/mysqldb-restore:v5 + image: squareops01/rds-mysql-restore:v1 imagePullPolicy: Always env: - name: MYSQL_HOST @@ -28,9 +28,7 @@ spec: - name: RESTORE_FILE_NAME value: {{ .Values.restore.file_name}} - name: RESTORE_FROM - value: {{ .Values.restore.bucket_provider}} - - name: AWS_DEFAULT_REGION - value: {{ .Values.restore.aws_default_region}} + value: {{ .Values.restore.provider }} resources: {{ .Values.restorejob.resources | toYaml | nindent 12 }} initContainers: - name: grant-system-variable-admin diff --git a/modules/db-backup-restore/roles.tf b/modules/db-backup-restore/roles.tf index 5917379..175bfd7 100644 --- a/modules/db-backup-restore/roles.tf +++ b/modules/db-backup-restore/roles.tf @@ -13,8 +13,8 @@ data "aws_eks_cluster" "kubernetes_cluster" { } resource "aws_iam_role" "mysql_backup_role" { - count = var.mysqldb_backup_enabled ? 1 : 0 - name = format("%s-%s-%s", var.cluster_name, var.name, "mysql-backup-rds") + count = var.mysqldb_backup_enabled ? 1 : 0 + name = format("%s-%s-%s", var.cluster_name, var.name, "mysql-backup-rds") assume_role_policy = jsonencode({ Version = "2012-10-17", Statement = [ @@ -57,8 +57,8 @@ resource "aws_iam_role" "mysql_backup_role" { resource "aws_iam_role" "mysql_restore_role" { - count = var.mysqldb_restore_enabled ? 1 : 0 - name = format("%s-%s-%s", var.cluster_name, var.name, "mysql-restore") + count = var.mysqldb_restore_enabled ? 1 : 0 + name = format("%s-%s-%s", var.cluster_name, var.name, "mysql-restore") assume_role_policy = jsonencode({ Version = "2012-10-17", Statement = [ @@ -97,4 +97,4 @@ resource "aws_iam_role" "mysql_restore_role" { ] }) } -} \ No newline at end of file +} diff --git a/modules/db-backup-restore/variables.tf b/modules/db-backup-restore/variables.tf index 18e6594..ff859b1 100644 --- a/modules/db-backup-restore/variables.tf +++ b/modules/db-backup-restore/variables.tf @@ -66,9 +66,9 @@ variable "cluster_name" { } variable "mysqldb_permission" { - default = false + default = false description = "access" - type = bool + type = bool } variable "bucket_provider_type" { diff --git a/variables.tf b/variables.tf index 700dc4b..a85fe48 100644 --- a/variables.tf +++ b/variables.tf @@ -373,4 +373,4 @@ variable "cluster_name" { type = string default = "" description = "Specifies the name of the EKS cluster to deploy the MySQL application on." -} \ No newline at end of file +}