Configuring Infrastructure-as-Code
Objective
Complete the necessary repository setup to begin utilizing OpenTofu (Terraform) and Terragrunt.
Background
The Panfactum stack uses the open-source Terraform fork, OpenTofu, to define the 100+ infrastructure modules that comprise the Panfactum stack. 1 We use a configuration-as-code manager for Terraform / OpenTofu called Terragrunt to deploy each module.
Because both OpenTofu (Terraform) and Terragrunt are unopinionated tools, every organization tends to implement infrastructure-as-code differently. While there is no "right" approach, it is tedious and error-prone to invent one from scratch.
As a result, the Panfactum stack provides standardization with a highly opinionated set of practices for deploying infrastructure-as-code that incorporates dozens of lessons learned over the past decade. It aligns with the recommendations provided by both Hashicorp (Terraform) and Gruntwork (Terragrunt).
We assume that you will build upon the framework we provide as you begin to deploy infrastructure. We provide CLI tooling that enables you to quickly scaffold out your project to align with our guides. As you become comfortable working in the stack, you may customize any part of our starting setup to fit your organization's evolving needs.
Setting up Your Repo
In order to provide a convenient and integrated experience for consumers of the Panfactum stack, we assume you will configure your stack repository inline with the following guidelines.
Version Control
One core axiom is that the stack repo will follow Github Flow for creating and integrating new changes.
The most important facets of GitHub flow are:
-
There is exactly one primary integration branch
-
That branch is always deployable
We extend that pattern with one more rule:
- That branch contains the configuration-as-code for all deployed infrastructure across every environment
This is a critically important concept. Again, one branch will define exactly what is deployed across all environments.
Pick the branch you want to use and make sure it is set as the PF_REPO_PRIMARY_BRANCH
(per the docs).
Choose Environments Folder
We store all configuration-as-code for all environment in a directory in the repo we canonically call the "environments folder" or "environments directory."
Pick the folder you want to use and make sure it is set as the PF_ENVIRONMENTS_DIR
(per the docs).
Configure Environments Folder
Your environments folder must have three levels of scoping for the configuration-as-code:
-
environment: A separation of your deployments according to your SDLC conventions (e.g.,
development
,staging
,production
, etc.) -
region: A separation of your deployments according to their physical location (e.g.,
us-west-2
) -
module: A separation of your deployments based on the specific infrastructures module being deployed
Accordingly, there are four levels of configuration for parameterizing the behavior of panfactum.hcl
:
-
global.yaml
: Found at the root of the directory -
environment.yaml
: Found at the root of each environment subdirectory -
region.yaml
: Found at the root of each region subdirectory -
module.yaml
: Found in each module subdirectory
A few rules about these files:
-
These files are optional, but you will use them to configure your providers and other behavior (described in detail later).
-
They can each contain any and every key found in this reference document.
-
You should commit these files to version control.
-
Conflicts are resolved via specificity (e.g., conflicting
module.yaml
values will overrideregion.yaml
values which in turn will overrideenvironment.yaml
values, etc.). -
Each file has an optional, user-specific counterpart named
*.user.yaml
(e.g.,region.user.yaml
).-
Values in these files override the values in their counterparts for local development purposes (e.g.,
region.user.yaml
overrides values inregion.yaml
). -
These files are not committed to version-control; each user will have their own set of values.
-
At this point, you should scaffold your environments folder until it looks something like this:
- environments/
- global.yaml
- panfactum.hcl
- providers/
- aws.tftpl
- kubernetes.tftpl
***
- [environment_1]/
- environment.yaml
- environment.user.yaml
- [region_1]/
- region.yaml
- region.user.yaml
- [module_1]/
- module.yaml
- [module_2]/
- module.yaml
- [region_2]/
***
- [environment_2]/
- environment.yaml
- [region_1]/
***
- [region_2]/
***
You should now have a fully scaffolded environments folder. Please validate the following statements about this folder:
-
It contains a
panfactum.hcl
. -
It contains a
global.yaml
. -
It contains a
providers
directory with several providers such asaws.tftpl
. -
You have a top-level directory for every environment we created in the previous Preparing AWS guide.
-
Each environment directory contains an
environment.yaml
file. -
Each environment directory contains a region directory for every AWS region you want to deploy infrastructure into. Additionally, you have a region directory called
global
inside of every environment directory. -
Each region directory contains a
region.yaml
file.
Finally, ensure that you do not receive any warnings about needing to run pf-terragrunt-update
when opening
the repository in your terminal.
Ultimately, your environments folder should closely resemble that of our reference architecture.
Configure Terragrunt Variables
In order to begin deploying infrastructure modules, we must first configure terragrunt via our terragrunt variables
which will be set in the global.yaml
, environment.yaml
, and region.yaml
files
(see reference docs).
Metadata
The following metadata fields are used for tagging and labeling deployed infrastructure:
-
In every
environment.yaml
file, set theenvironment
key to the name of the environment (typically the same as the directory name). -
In every
region.yaml
file, set theregion
key to the name of the region (typically the same as the directory name).
Panfactum Version
The following fields are used to define which versions of the Panfactum modules to deploy:
pf_stack_version
: Check here for the full list of available releases. This should be the same version as used for the Panfactum devenv. Set this in eachenvironment.yaml
so that you can test version updates in one environment at a time.
For more information about releases and support, see our guide to releases.
State Backend
Each environment will have its own, independent OpenTofu backend for storing information about the tracked infrastructure resources. We utilize the S3 backend and store the state for each environment inside of that environment's AWS account.
In every environment.yaml, set the following keys:
-
tf_state_account_id
: Set this to the account id for the environment's AWS account -
tf_state_profile
: Set this to the AWS profile name you created for accessing this account in the Preparing AWS guide -
tf_state_region
: Set this your primary AWS region (buckets have to have an assigned region even though we use it for deploying resources to all regions in the environment) -
tf_state_bucket
: Set the name of the S3 bucket you want to use. Should not exist yet and must be globally unique. Example:my-company-tf-state-development
. -
tf_state_lock_table
: Set the name of the DynamoDB table you want to use. Should not exist yet and must be globally unique. Example:my-company-tf-state-development
.
AWS Provider
These variables will configure how the OpenTofu (Terraform) AWS provider deploys resources into AWS and will vary its behavior based on the folder the module is deployed from.
In every environment.yaml
file, set the following keys:
-
aws_account_id
: Set this to the account id for the environment's AWS account. -
aws_profile
: Set this to the AWS profile name you created for accessing this account in the Preparing AWS guide. -
aws_secondary_account_id
: Set this to theaws_account_id
from above. -
aws_secondary_profile
: Set this to theaws_profile
from above.
In every region.yaml
file, set the following keys:
-
aws_region
: Set this to the AWS region code for the directory (e.g.,us-west-2
). For theglobal
region, set the region to your primary AWS region which would normally be what you used fortf_state_region
above. -
aws_secondary_region
: Set this to a different region thanaws_region
from above. This value is primarily used to create regional backups, so set this to the region where you want your backups to live. 2
The secondary values are used by some modules that configure resources in multiple AWS accounts or regions. What we have just done is default them to primary values, and we will override them on a per-module basis as needed.
Next Steps
In the next guide section, we will use this configuration to deploy your first infrastructure-as-code modules.
Footnotes
-
To learn more about why Terraform was forked, see the OpenTofu manifesto. ↩
-
You do not have to a have a region folder for the backup region. ↩