Configuring Infrastructure-as-Code

Objective

Complete the necessary repository setup to begin utilizing OpenTofu (Terraform) and Terragrunt.

Background

The Panfactum stack uses the open-source Terraform fork, OpenTofu, to define the 100+ infrastructure modules that comprise the Panfactum stack. 1 We use a configuration-as-code manager for Terraform / OpenTofu called Terragrunt to deploy each module.

Because both OpenTofu (Terraform) and Terragrunt are unopinionated tools, every organization tends to implement infrastructure-as-code differently. While there is no "right" approach, it is tedious and error-prone to invent one from scratch.

As a result, the Panfactum stack provides standardization with a highly opinionated set of practices for deploying infrastructure-as-code that incorporates dozens of lessons learned over the past decade. It aligns with the recommendations provided by both Hashicorp (Terraform) and Gruntwork (Terragrunt).

We assume that you will build upon the framework we provide as you begin to deploy infrastructure. We provide CLI tooling that enables you to quickly scaffold out your project to align with our guides. As you become comfortable working in the stack, you may customize any part of our starting setup to fit your organization's evolving needs.

Setting up Your Repo

In order to provide a convenient and integrated experience for consumers of the Panfactum stack, we assume you will configure your stack repository inline with the following guidelines.

Version Control

One core axiom is that the stack repo will follow Github Flow for creating and integrating new changes.

The most important facets of GitHub flow are:

• There is exactly one primary integration branch

• That branch is always deployable

We extend that pattern with one more rule:

• That branch contains the configuration-as-code for all deployed infrastructure across every environment

This is a critically important concept. Again, one branch will define exactly what is deployed across all environments.

Pick the branch you want to use and make sure it is set as the PF_REPO_PRIMARY_BRANCH (per the docs).

Choose Environments Folder

We store all configuration-as-code for all environment in a directory in the repo we canonically call the "environments folder" or "environments directory."

Pick the folder you want to use and make sure it is set as the PF_ENVIRONMENTS_DIR (per the docs).

Configure Environments Folder

Your environments folder must have three levels of scoping for the configuration-as-code:

• environment: A separation of your deployments according to your SDLC conventions (e.g., development, staging, production, etc.)

• region: A separation of your deployments according to their physical location (e.g., us-west-2)

• module: A separation of your deployments based on the specific infrastructures module being deployed

Accordingly, there are four levels of configuration for parameterizing the behavior of panfactum.hcl:

• global.yaml: Found at the root of the directory

• environment.yaml: Found at the root of each environment subdirectory

• region.yaml: Found at the root of each region subdirectory

• module.yaml: Found in each module subdirectory

A few rules about these files:

• These files are optional, but you will use them to configure your providers and other behavior (described in detail later).

• They can each contain any and every key found in this reference document.

• You should commit these files to version control.

• Conflicts are resolved via specificity (e.g., conflicting module.yaml values will override region.yaml values which in turn will override environment.yaml values, etc.).

• Each file has an optional, user-specific counterpart named *.user.yaml (e.g., region.user.yaml).

  • Values in these files override the values in their counterparts for local development purposes (e.g., region.user.yaml overrides values in region.yaml).

  • These files are not committed to version-control; each user will have their own set of values.

At this point, you should scaffold your environments folder until it looks something like this:

- environments/
    - global.yaml
    - panfactum.hcl
    - providers/
      - aws.tftpl
      - kubernetes.tftpl
      ***
    - [environment_1]/
      - environment.yaml
      - environment.user.yaml
      - [region_1]/
        - region.yaml
        - region.user.yaml
        - [module_1]/
          - module.yaml
        - [module_2]/
          - module.yaml
      - [region_2]/
        ***
    - [environment_2]/
      - environment.yaml
      - [region_1]/
         ***
      - [region_2]/
        ***

You should now have a fully scaffolded environments folder. Please validate the following statements about this folder:

• It contains a panfactum.hcl.

• It contains a global.yaml.

• It contains a providers directory with several providers such as aws.tftpl.

• You have a top-level directory for every environment we created in the previous Preparing AWS guide.

• Each environment directory contains an environment.yaml file.

• Each environment directory contains a region directory for every AWS region you want to deploy infrastructure into. Additionally, you have a region directory called global inside of every environment directory.

• Each region directory contains a region.yaml file.

Finally, ensure that you do not receive any warnings about needing to run pf-terragrunt-update when opening the repository in your terminal.

Ultimately, your environments folder should closely resemble that of our reference architecture.

Configure Terragrunt Variables

In order to begin deploying infrastructure modules, we must first configure terragrunt via our terragrunt variables which will be set in the global.yaml, environment.yaml, and region.yaml files (see reference docs).

Metadata

The following metadata fields are used for tagging and labeling deployed infrastructure:

• In every environment.yaml file, set the environment key to the name of the environment (typically the same as the directory name).

• In every region.yaml file, set the region key to the name of the region (typically the same as the directory name).

Panfactum Version

The following fields are used to define which versions of the Panfactum modules to deploy:

• pf_stack_version: Check here for the full list of available releases. This should be the same version as used for the Panfactum devenv. Set this in each environment.yaml so that you can test version updates in one environment at a time.

For more information about releases and support, see our guide to releases.

State Backend

Each environment will have its own, independent OpenTofu backend for storing information about the tracked infrastructure resources. We utilize the S3 backend and store the state for each environment inside of that environment's AWS account.

In every environment.yaml, set the following keys:

• tf_state_account_id: Set this to the account id for the environment's AWS account

• tf_state_profile: Set this to the AWS profile name you created for accessing this account in the Preparing AWS guide

• tf_state_region: Set this your primary AWS region (buckets have to have an assigned region even though we use it for deploying resources to all regions in the environment)

• tf_state_bucket: Set the name of the S3 bucket you want to use. Should not exist yet and must be globally unique. Example: my-company-tf-state-development.

• tf_state_lock_table: Set the name of the DynamoDB table you want to use. Should not exist yet and must be globally unique. Example: my-company-tf-state-development.

AWS Provider

These variables will configure how the OpenTofu (Terraform) AWS provider deploys resources into AWS and will vary its behavior based on the folder the module is deployed from.

In every environment.yaml file, set the following keys:

• aws_account_id: Set this to the account id for the environment's AWS account.

• aws_profile: Set this to the AWS profile name you created for accessing this account in the Preparing AWS guide.

• aws_secondary_account_id: Set this to the aws_account_id from above.

• aws_secondary_profile: Set this to the aws_profile from above.

In every region.yaml file, set the following keys:

• aws_region: Set this to the AWS region code for the directory (e.g., us-west-2). For the global region, set the region to your primary AWS region which would normally be what you used for tf_state_region above.

• aws_secondary_region: Set this to a different region than aws_region from above. This value is primarily used to create regional backups, so set this to the region where you want your backups to live. 2

The secondary values are used by some modules that configure resources in multiple AWS accounts or regions. What we have just done is default them to primary values, and we will override them on a per-module basis as needed.

Next Steps

In the next guide section, we will use this configuration to deploy your first infrastructure-as-code modules.