Deploying Workloads: Permissions

Overview

Often your workloads will need to interact with the control plane APIs. To do this, your workloads will need to be granted the permissions relevant to the particular control plane system (e.g., AWS). While this differs by system (AWS permissions are assigned in a different way than Kubernetes permissions), all permissions are assigned to workloads via Service Accounts.

All our controller submodules will automatically create a dedicated service account for the pods managed by the controller and provide its name via the service_account_name output.

We provide several submodules that allow you to easily assign permissions to service accounts:

Multiple of these submodules may be used with a single service account in order to grant a single workload access to multiple control plane systems.

Kubernetes Permissions

When a pod is created, a JWT is generated for its service account and mounted inside the pod at /var/run/secrets/kubernetes.io/serviceaccount/token. This JWT is a "Bearer" token and can be presented to the Kubernetes API to authenticate: 1

TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
curl -H "Authorization: Bearer $TOKEN" https://$KUBERNETES_API/api/v1/namespaces/default/pods/ --insecure

This token is bound to the lifetime of the pod and will no longer work once the pod is removed from the cluster. Additionally, the token will be rotated every hour while the pod is still alive. 2

While the token "authenticates" the workloads (e.g., provides an identity), it does not "authorize" the workload to access specific API endpoints, so you will receive 403 error codes.

To provide permissions to service accounts, you must use the Kubernetes RBAC system by creating Roles and RoleBindings. 3

For example, this code snippet gives a Deployment's service account permissions to get, list, and update PVCs in its namespace:

module "example_deployment" {
  source     = "github.com/Panfactum/stack.git//packages/infrastructure/kube_deployment?ref=edge.24-09-12" # pf-update

  name       = "example"
  namespace  = var.namespace

  ...

  # pf-generate: pass_vars
  pf_stack_version = var.pf_stack_version
  pf_stack_commit  = var.pf_stack_commit
  environment      = var.environment
  region           = var.region
  pf_root_module   = var.pf_root_module
  is_local         = var.is_local
  extra_tags       = var.extra_tags
  # end-generate
}

resource "kubernetes_role" "example" {
  metadata {
    name      = "example"
    namespace = var.namespace
    labels    = module.example_deployment.labels
  }
  rule {
    api_groups = [""]
    resources  = ["persistentvolumeclaims"]
    verbs      = ["get", "update", "list"]
  }
}

resource "kubernetes_role_binding" "example" {
  metadata {
    name      = "example"
    namespace = var.namespace
    labels    = module.example_deployment.labels
  }
  subject {
    kind      = "ServiceAccount"
    name      = module.example_deployment.service_account_name
    namespace = var.namespace
  }
  role_ref {
    api_group = "rbac.authorization.k8s.io"
    kind      = "Role"
    name      = kubernetes_role.example.metadata[0].name
  }
}