Panfactum LogoPanfactum
Infrastructure-as-CodeDebugging

Debugging

Unfortunately, there is no step-by-step guide that can resolve all issues with Terragrunt and OpenTofu (Terraform). However, most issues can be resolved through the following:

  • Ensure that you have the correct providers enabled by running pf-tf-init in the module directory.

  • Ensure that you are authenticated for the region / environment you are attempting to run the command in:

    • aws

      • Should succeed: aws --profile <profile> sts get-caller-identity
      • To fix: aws --profile <profile> sso login
    • kubernetes

      • Should succeed: kubectl --context <context> get cluster-info
      • To fix: pf-update-kube
    • vault

      • Should succeed: vault token lookup --address <vault-address>
      • To fix: pf-get-vault-token <vault-address>
  • Delete the .terragrunt-cache directory at the root of the repo to resolve issues caused by a stale cache.

  • Try to re-init the module causing issues by running terragrunt init.

  • Review the debugging documentation provided by the Terragrunt team.

  • Review the debugging documentation provided by the OpenTofu team.

Common Errors

sops Not Authenticated

Call to function "sops_decrypt_file" failed: Error getting data key: 0 successful groups required, got 0.

This means the AWS profile needed to access the secrets file by sops is not currently available. This usually occurs when your SSO session has expired. Login to the correct profile: aws --profile <profile> sso login.

sops File Not Encrypted

Error in function call; Call to function "sops_decrypt_file" failed: sops metadata not found.

This means your secrets file was not encrypted by sops. Encrypt it via sops -e -i <file> to resolve.

Invalid Provider Configuration

Error: Invalid provider configuration

This likely means the module is using one or more providers that you have not enabled. Run pf-tf-init in the module directory to automatically enable all the required providers.

Available checksums are for packages targeting different platforms

Error: Failed to install provider                                                                                                                                           │

Error while installing hashicorp/aws v5.39.1: the local package for                                                                                                         │
registry.opentofu.org/hashicorp/aws 5.39.1 doesn't match any of the                                                                                                         │
checksums previously recorded in the dependency lock file (this might be                                                                                                    │
because the available checksums are for packages targeting different                                                                                                        │
platforms); for more information:                                                                                                                                           │
https://opentofu.org/docs/language/files/dependency-lock/#checksum-verification

When first deployed, every IaC module generates a .terraform.lock.hcl file. This file contains checksum hashes for the Tofu provider binaries to ensure that the same provider is always used to deploy this module unless explicitly updated.

Unfortunately, the list of checksums only contain hashes for your current operating system (e.g., linux) and system architecture (e.g., amd64) by default. This will cause the above error to appear if you try to deploy the IaC from a different operating system or system architecture.

Run pf-tf-init to add checksums for all common systems.

This uses the providers lock command to enable the following platforms: darwin_amd64, darwin_arm64, linux_amd64, linux_arm64.

Inconsistent final plan

Error: Provider produced inconsistent final plan

When expanding the plan for
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
to include new values learned so far during apply, provider
"registry.opentofu.org/alekc/kubectl" changed the planned action from
Update to DeleteThenCreate.

This is a bug in the provider, which should be reported in the provider's
own issue tracker.

Issues like this can occasionally occur when non-breaking internal refactors are completed on resources that use the kubectl provider.

This is a transient error and can be resolved by simply re-running terragrunt apply.

Backend configuration block has changed

Error: Backend initialization required: please run "tofu init"

Reason: Backend configuration block has changed

The "backend" is the interface that OpenTofu uses to store state,
perform operations, etc. If this message is showing up, it means that the
OpenTofu configuration you're using is using a custom configuration for
the OpenTofu backend.

Changes to backend configurations require reinitialization. This allows
OpenTofu to set up the new configuration, copy existing state, etc. Please
run "tofu init" with either the "-reconfigure" or "-migrate-state" flags to
use the current configuration.

If the change reason above is incorrect, please verify your configuration
hasn't changed and try again. At this point, no changes to your existing
configuration or state have been made.

This issue can occur when changing your terragrunt variables in between runs of the same IaC module.

To resolve, delete your terragrunt cache (.terragrunt-cache) and retry.

Module not installed

Error: Module not installed

OpenTofu requires that module and submodule code be stored in certain locations on disk. This is done via a call to init (i.e., terragrunt init).

When you first apply a module, Terragrunt will "auto-init" your module as a convenience.

However, as the internals of modules change (such as between Panfactum releases), the submodules sometimes moves or new submodules are added. This requires the module be reinitialized. Unfortunately, Terragrunt does not have the capability to detect when module internals change and thus will not automatically init again. 1

As a result, when you run into this error, you just need to run terragrunt init manually. Generally, this can be across your entire environment by at once by running terragrunt run-all init.

Footnotes

  1. You may be wondering: Why not just automatically init before every apply? The answer is that init is a relatively expensive operation and can add 5-15 seconds of execution time. This would be annoying to deal with on every apply when it is only needed occasionally.