Terraform enables you to manage your Amazon Relational Database Service (RDS) instances over their lifecycle. Using Terraform's built-in lifecycle arguments, you can manage the dependency and upgrade ordering for tightly coupled resources like RDS instances and their parameter groups. You will also use Terraform to securely set your database password and store it in AWS Systems Manager (SSM).
In this tutorial, you will configure an RDS instance with Terraform, storing the password in SSM. Next, you will perform a major version upgrade on your RDS instance using Terraform and review how Terraform handles dependencies when you use it to manage resources that depend on other resources in your configuration.
This tutorial assumes that you are familiar with the standard Terraform workflow. If you are new to Terraform, complete the Get Started tutorials first.
For this tutorial, you will need:
psql command line utility for PostgreSQLjq utility installed and in your PATHClone the example repository for this tutorial, which contains configuration for an RDS instance and parameter group.
$ git clone https://github.com/hashicorp-education/learn-terraform-rds-upgrade
Change into the repository directory.
$ cd learn-terraform-rds-upgrade
Open main.tf in your code editor to review the resources you will provision. This configuration defines the following resources:
5432.aws_db_instance, configured with PostgreSQL 15.Note
The example configuration allows access to your RDS instance from the public internet, so that you can connect to it later in this tutorial. In production scenarios, we recommend you follow security best practices, such as placing your RDS instance in a private subnet and restricting access to it only from subnets you control.
The aws_db_parameter_group resource's family attribute configures the major version of your database instance. In this case, the parameter group family is postgres15, so the RDS engine will be PostgreSQL v15.
main.tf
resource "aws_db_parameter_group" "education" {
name_prefix = "${random_pet.name.id}-education"
family = "postgres15"
parameter {
name = "log_connections"
value = "1"
}
lifecycle {
create_before_destroy = true
}
}
While you could use a default AWS parameter group for your database, we recommend that you maintain a custom one for your RDS instances. You cannot modify the parameters on the default parameter groups maintained by AWS. If you need to update an RDS setting in the future, you can modify your custom parameter group rather than creating a new one at that time.
The configuration generates a random password for your RDS instance using an ephemeral resource. The configuration then sets this password for your database using a write-only argument. The configuration also stores and encrypts the generated password in AWS SSM using another write-only argument. for your RDS instance, and stores the password as a parameter in AWS SSM, encrypted with your default SSM key. The configuration sets the password for your database and stores it in SSM using write-only arguments.
Review the ephemeral resource for the database password in main.tf.
main.tf
ephemeral "random_password" "db_password" {
length = 16
}
The random_password.db_password is an ephemeral resource. Terraform does not store ephemeral resources in its state or plan files.
Note
Ephemeral resources and values are available in Terraform 1.10 and later.
The configuration uses random_password.db_password to set the value of two write-only arguments. A resource's write-only arguments are only available during the current operation, and Terraform does not store the argument's values in state or plan files. Terraform providers define write-only arguments for values that you do not want to store in Terraform's state, such as passwords or other secrets.
The first write-only argument, aws_db_instance.password_wo, sets the password on the RDS instance. The second write-only argument, aws_ssm_parameter.value_wo, stores the password value as an AWS SSM secret. With this configuration, Terraform does not store your database password, and the only way to retrieve that password is by querying the AWS SSM parameter.
main.tf
locals {
# Increment db_password_version to update the DB password and store the new
# password in SSM.
db_password_version = 1
}
resource "aws_db_instance" "education" {
identifier = "${random_pet.name.id}-education"
instance_class = "db.t3.micro"
allocated_storage = 10
apply_immediately = true
engine = "postgres"
engine_version = "15"
username = "edu"
password_wo = ephemeral.random_password.db_password.result
password_wo_version = local.db_password_version
## ...
}
resource "aws_ssm_parameter" "secret" {
name = "/education/database/password/master"
description = "Password for RDS database."
type = "SecureString"
value_wo = ephemeral.random_password.db_password.result
value_wo_version = local.db_password_version
}
Because Terraform does not store the value of write-only arguments, it cannot detect if the value of a write-only argument changed in your configuration. To track whether a write-only argument changes, the AWS provider includes accompanying versioning arguments: password_wo uses password_wo_version and value_wo uses value_wo_version.
Versioning arguments are tracked in state. You can indicate to Terraform and providers that a write-only arguments has changed by incrementing the corresponding _version argument. For example, incrementing password_wo_version lets Terraform know the value of password_wo has changed. Terraform then records that change in its plan, notifying the provider that password_wo has a new value it can use.
The example configuration sets both password_wo_version and value_wo_version to the same local value, local.db_password_version. If the values were hard-coded, a user might update one of these values but not the other, and cause the database password to become out of sync with the password stored in SSM.
Note
Write-only arguments are available in Terraform 1.11 and later.
In your terminal, initialize the Terraform configuration to install the module and providers used in this tutorial.
$ terraform init
Initializing the backend...
Initializing modules...
Downloading registry.terraform.io/terraform-aws-modules/vpc/aws 5.19.0 for vpc...
- vpc in .terraform/modules/vpc
Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Reusing previous version of hashicorp/random from the dependency lock file
- Installing hashicorp/aws v5.88.0...
- Installed hashicorp/aws v5.88.0 (signed by HashiCorp)
- Installing hashicorp/random v3.7.0...
- Installed hashicorp/random v3.7.0 (signed by HashiCorp)
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
Next, apply your configuration to create your RDS instance and other resources. Enter yes when prompted to confirm the operation. Note that it can take up to 10 minutes to create an RDS instance.
$ terraform apply
ephemeral.random_password.db_password: Opening...
ephemeral.random_password.db_password: Opening complete after 0s
data.aws_availability_zones.available: Reading...
data.aws_availability_zones.available: Read complete after 1s [id=us-east-2]
ephemeral.random_password.db_password: Closing...
ephemeral.random_password.db_password: Closing complete after 0s
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# aws_db_instance.education will be created
+ resource "aws_db_instance" "education" {
+ address = (known after apply)
+ allocated_storage = 10
## ...
Plan: 19 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ random_pet_name = (known after apply)
+ rds_hostname = (sensitive value)
+ rds_port = (sensitive value)
+ rds_username = (sensitive value)
+ region = "us-east-2"
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
## ...
random_pet_name = "pelican"
rds_hostname = <sensitive>
rds_port = <sensitive>
rds_username = <sensitive>
region = "us-east-2"
Notice that the RDS hostname, port, and username are marked as sensitive. The example configuration sets the sensitive attribute to true for these outputs so that Terraform won't include those values in its output by default. For example, the rds_hostname output block is designated as sensitive.
outputs.tf
output "rds_hostname" {
description = "RDS instance hostname."
value = aws_db_instance.education.address
sensitive = true
}
Unlike ephemeral resources and write-only arguments, Terraform stores sensitive values in its state file, and will output these values as plain text if you specify the -raw flag for the terraform output command, or the -json flag to print out your workspace's output values in JSON format. Terraform stores sensitive values unencrypted in its state file, so you must keep this file secure.
Next, connect to the database with the psql command line utility, and seed it. The coffees.sql file in the repository contains commands that populate your database with mock data about HashiCorp-themed coffee beverages.
psql can access your password using thr PGPASSWORD environment variable. Set your PostgreSQL password as an environment variable by retrieving the parameter from AWS SSM and using jq to extract the password from the response.
$ export PGPASSWORD=$( \
aws ssm get-parameter \
--region=$(terraform output -raw region) \
--name=/education/database/password/master \
--with-decryption \
| jq --raw-output '.Parameter.Value' \
)
Note
The previous command will save your database password unencrypted in the PGPASSWORD environment variable in your shell session. For production use cases, you may wish to unset this value once you are done using it.
Then, execute the script.
$ psql -h $(terraform output -raw rds_hostname) -U $(terraform output -raw rds_username) postgres -f coffees.sql
SET
CREATE EXTENSION
CREATE TABLE
CREATE TABLE
CREATE TABLE
## ...
Connect to your database to inspect your records.
$ psql -h $(terraform output -raw rds_hostname) -U $(terraform output -raw rds_username) postgres
psql (16.3, server 15.5)
SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, compression: off)
Type "help" for help.
postgres=>
At the postgres prompt, list all of the coffees in your database.
$ SELECT * FROM coffees;
1 | Packer Spiced Latte | Packed with goodness to spice up your images | | 350 | /packer.png | 2021-08-30 00:00:00 | 2021-08-30 00:00:00 |
2 | Vaulatte | Nothing gives you a safe and secure feeling like a Vaulatte | | 200 | /vault.png | 2021-08-30 00:00:00 | 2021-08-30 00:00:00 |
3 | Nomadicano | Drink one today and you will want to schedule another | | 150 | /nomad.png | 2021-08-30 00:00:00 | 2021-08-30 00:00:00 |
4 | Terraspresso | Nothing kickstarts your day like a provision of Terraspresso | | 150 | /terraform.png | 2021-08-30 00:00:00 | 2021-08-30 00:00:00 |
5 | Vagrante espresso | Stdin is not a tty | | 200 | /vagrant.png | 2021-08-30 00:00:00 | 2021-08-30 00:00:00 |
6 | Connectaccino | Discover the wonders of our meshy service | | 250 | /consul.png | 2021-08-30 00:00:00 | 2021-08-30 00:00:00 |
Type exit to exit psql.
When managing an RDS instance, a common task is to upgrade the database to a new major version. To do so, you will upgrade the database engine version and the parameter group family in your Terraform configuration, and apply the change.
Take a snapshotBefore you upgrade your database, create a backup snapshot of your data. It is good practice to back up your data when you perform operations on your databases so that you have a point of recovery in the event of data loss or other error.
Add the following configuration to main.tf to create a snapshot of your database.
main.tf
resource "aws_db_snapshot" "pre_16_upgrade" {
db_instance_identifier = aws_db_instance.education.identifier
db_snapshot_identifier = "pre16upgradebackup"
}
Add the following to outputs.tf to report the identifier and status of your DB snapshot.
outputs.tf
output "rds_pre_16_backup_identifier" {
description = "Identifier of the snapshot created before upgrading RDS instance to PostgreSQL 16."
value = aws_db_snapshot.pre_16_upgrade.db_snapshot_identifier
}
output "rds_pre_16_backup_status" {
description = "Status of the snapshot created before upgrading RDS instance to PostgreSQL 16."
value = aws_db_snapshot.pre_16_upgrade.status
}
Apply your configuration to create the snapshot. Enter yes when prompted to confirm the operation.
$ terraform apply
random_pet.name: Refreshing state... [id=pelican]
ephemeral.random_password.db_password: Opening...
ephemeral.random_password.db_password: Opening complete after 0s
data.aws_availability_zones.available: Reading...
## ...
Terraform used the selected providers to generate the following execution plan.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# aws_db_snapshot.pre_16_upgrade will be created
+ resource "aws_db_snapshot" "pre_16_upgrade" {
+ allocated_storage = (known after apply)
+ availability_zone = (known after apply)
## ...
Plan: 1 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ rds_pre_16_backup_identifier = "pre16upgradebackup"
+ rds_pre_16_backup_status = (known after apply)
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
ephemeral.random_password.db_password: Opening...
ephemeral.random_password.db_password: Opening complete after 0s
aws_db_snapshot.pre_16_upgrade: Creating...
## ...
aws_db_snapshot.pre_16_upgrade: Still creating... [2m0s elapsed]
aws_db_snapshot.pre_16_upgrade: Creation complete after 2m1s [id=pre-16-upgrade-backup-pelican]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Outputs:
random_pet_name = "pelican"
rds_hostname = <sensitive>
rds_port = <sensitive>
rds_pre_16_backup_identifier = "pre16upgradebackup"
rds_pre_16_backup_status = "available"
rds_username = <sensitive>
region = "us-east-2"
In this Terraform configuration, the aws_db_instance resource references the aws_db_parameter_group, creating an implicit dependency between the two. As a result, Terraform would first try to upgrade the parameter group, but would error out because the destructive update would attempt to remove a parameter group associated with a running RDS instance.
Terraform offers lifecycle meta-arguments to help you manage more complex resource dependencies such as this one. In this case, the aws_db_parameter_group in the example configuration includes the create_before_destroy argument to ensure that Terraform provisions the new parameter group and upgrades your RDS instance before destroying the original parameter group.
In your main.tf file, make the following changes:
In the aws_rds_parameter_group resource definition, update the family argument to postgres16 as shown below.
main.tf
resource "aws_db_parameter_group" "education" {
name_prefix = "${random_pet.name.id}-education"
family = "postgres16"
parameter {
name = "log_connections"
value = "1"
}
lifecycle {
create_before_destroy = true
}
}
Update the version argument for the aws_db_instance resource to 16.
main.tf
resource "aws_db_instance" "education" {
identifier = "${random_pet.name.id}-education"
instance_class = "db.t3.micro"
allocated_storage = 10
apply_immediately = true
engine = "postgres"
engine_version = "16"
## ...
}
In your terminal, apply your configuration changes to replace the parameter group and upgrade the engine version of your RDS instance. Enter yes when prompted to approve the operation.
Note
Major version upgrades to RDS are a destructive change. AWS will remove the existing data from your database, and you will need to reload it.
$ terraform apply
## ...
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
~ update in-place
+/- create replacement and then destroy
Terraform will perform the following actions:
# aws_db_instance.education will be updated in-place
~ resource "aws_db_instance" "education" {
~ engine_version = "15" -> "16"
id = "db-RBX4IELIDWELBC3JXRPYXKANHU"
~ parameter_group_name = "halibut-education20240510184235754100000001" -> (known after apply)
tags = {}
# (52 unchanged attributes hidden)
}
# aws_db_parameter_group.education must be replaced
/- resource "aws_db_parameter_group" "education" {
~ arn = "arn:aws:rds:us-east-2:561656980159:pg:halibut-education20240510184235754100000001" -> (known after apply)
~ family = "postgres15" -> "postgres16" # forces replacement
~ id = "halibut-education20240510184235754100000001" -> (known after apply)
~ name = "halibut-education20240510184235754100000001" -> (known after apply)
- tags = {} -> null
# (3 unchanged attributes hidden)
# (1 unchanged block hidden)
}
Plan: 1 to add, 1 to change, 1 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
## ...
Apply complete! Resources: 1 added, 1 changed, 1 destroyed.
Outputs:
random_pet_name = "pelican"
rds_hostname = <sensitive>
rds_port = <sensitive>
rds_pre_16_backup_identifier = "pre16upgradebackup"
rds_pre_16_backup_status = "available"
rds_username = <sensitive>
region = "us-east-2"
Note
This upgrade may take up to 20 minutes.
Verify upgradeVerify that the RDS instance is using Postgres 16.
$ psql -h $(terraform output -raw rds_hostname) -U $(terraform output -raw rds_username) postgres -c "SELECT version()"
version
---------------------------------------------------------------------------------------------------------
PostgreSQL 16.3 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 7.3.1 20180712 (Red Hat 7.3.1-17), 64-bit
(1 row)
Once you have completed the tutorial, destroy your infrastructure to avoid incurring unnecessary costs. Type yes when prompted to confirm the operation.
$ terraform destroy
random_pet.name: Refreshing state... [id=pelican]
## ...
aws_db_snapshot.pre_16_upgrade: Refreshing state... [id=pre-16-upgrade-backup-pelican]
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
- destroy
Terraform will perform the following actions:
# aws_db_instance.education will be destroyed
- resource "aws_db_instance" "education" {
- address = "pelican-education.c0gk8jn5j2r9.us-east-2.rds.amazonaws.com" -> null
- allocated_storage = 10 -> null
- allow_major_version_upgrade = true -> null
- apply_immediately = true -> null
## ...
Plan: 0 to add, 0 to change, 20 to destroy.
Changes to Outputs:
- random_pet_name = "pelican" -> null
- rds_hostname = (sensitive value) -> null
- rds_port = (sensitive value) -> null
- rds_pre_16_backup_identifier = "pre-16-upgrade-backup-pelican" -> null
- rds_pre_16_backup_status = "available" -> null
- rds_username = (sensitive value) -> null
- region = "us-east-2" -> null
Do you really want to destroy all resources?
Terraform will destroy all your managed infrastructure, as shown above.
There is no undo. Only 'yes' will be accepted to confirm.
Enter a value: yes
module.vpc.aws_route_table_association.public[1]: Destroying... [id=rtbassoc-0f201820cea0f3d14]
module.vpc.aws_default_security_group.this[0]: Destroying... [id=sg-065248f22c8ea3239]
module.vpc.aws_route_table_association.public[2]: Destroying... [id=rtbassoc-00cba7a3c2fb4da35]
## ...
random_pet.name: Destroying... [id=pelican]
random_pet.name: Destruction complete after 0s
Destroy complete! Resources: 20 destroyed.
In this tutorial, you learned how you can use Terraform's lifecycle arguments to manage a major version upgrade of your RDS instances. To learn more about the concepts used in this configuration, review the following tutorials:
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4