aws_eks
Elastic Kubernetes Service (EKS)
This module provides our standard setup for a configurable AWS EKS Cluster. It includes:
-
An EKS Cluster. This cluster defines the Kubernetes control plane (managed by AWS) and provisions it to the specified set of availability zones.
-
A KMS key for encrypting the control plane data at-rest.
-
Setup of EKS Access Entries.
-
A set of "controller" node groups with a static size for running cluster-critical controllers. Nodes use the Bottlerocket distribution. Autoscaled nodes are deployed via our kube_karpenter module.
-
Security groups for both the cluster control plane and for the node groups.
- The control plane accepts inbound traffic from the nodes and can make arbitrary outbound traffic.
- The nodes accept inbound traffic from the control plane, from each other, and can make arbitrary outbound traffic.
-
Subnet tags that controllers in our other modules depend upon.
-
The requisite infrastructure for using IAM roles for service accounts (IRSA).
Usage
Installation
Choose Control Plane Subnets
Control plane subnets are the subnets within which AWS will deploy the EKS-managed Kubernetes API servers.
By default, the control plane subnets will be any subnet named PUBLIC_A, PUBLIC_B, or PUBLIC_C in the VPC indicated
by the vpc_id input as these are the subnets created by the aws_vpc module.
If you need to overwrite the default module behavior, you can specify control_plane_subnets. This input takes at least 2 subnets (each in a different AZ).
Choose Node Subnets
Your node subnets cannot be changed without downtime in the future.
Node subnets are the subnets within which your actual workloads will run once deployed to the Kubernetes cluster.
By default, the node subnets will be any subnet named PRIVATE_A, PRIVATE_B, or PRIVATE_C in the VPC indicated
by the vpc_id input as these are the subnets created by the aws_vpc module.
If you need to overwrite the default module behavior, you can specify node_subnets.
For an SLA target of level 2 or above, you MUST provide at least 3 subnets (each in a different AZ).
Overriding the Service CIDR
Kubernetes requires that you specify a range of IP addresses that can be allocated to Services deployed in Kubernetes. This is called the Service CIDR.
We provide a default CIDR range of 172.20.0.0/16. We strongly discourage overriding this default unless you
have a demonstrated need.
If you do override with the service_cidr input, you MUST provide a private CIDR range that does not conflict with your VPC or any of its subnets. That
is because Kubernetes performs its own routing and networking independently of AWS.
You will also need to choose a dns_service_ip which must be in the service_cidr. If you use the
172.20.0.0/16 CIDR, then you should use 172.20.0.10 as this is the EKS default.
Post-install Steps
This module is intended to be installed as a part of this guide which includes manual steps that must be run after applying the module.
RBAC
This module configures access to the cluster via EKS Access Entries.
See the below table for our standard Kubernetes groups, the AWS principals linked to each group (configured through this module's input variables), and the description of the intended permission level:
| Kubernetes Group | Default AWS Principals Linked | Extra AWS Principals Linked | Permission Level |
|---|---|---|---|
pf:superusers | Superuser SSO Role, root IAM User | var.superuser_principal_arns | Full access to everything in the cluster. (AmazonEKSClusterAdminPolicy) |
pf:admins | Admin SSO Role | var.admin_princiapl_arns | Write access to everything besides core cluster utilities. (AmazonEKSAdminViewPolicy) |
pf:readers | Reader SSO Role | var.reader_principal_arns | Read access to all resources (including secrets). (AmazonEKSEditPolicy) |
pf:restricted-readers | RestrictedReader SSO Role | var.restricted_reader_principal_arns | Read access to all resources (not including secrets). (AmazonEKSViewPolicy) |
The SSO roles are installed into each account via aws_iam_identity_center_permissions and are automatically discovered by this module. Users with access to a particular AWS IAM SSO role will have the corresponding permissions in all Panfactum clusters in that AWS account.
You can explicitly grant additional AWS IAM principals (users and roles) access via the input variables outlined above (e.g., var.superuser_principal_arns).
Note that extra permissions are given to the pf:admins and pf:restricted-readers Kubernetes groups
in the kube_policies. AWS doesn't install
permissions that cover CRDs, so we add them ourselves once the cluster is instantiated.
Providers
The following providers are needed by this module:
Required Inputs
The following input variables are required:
cluster_description
Description: The purpose of the EKS cluster.
Type: string
cluster_name
Description: The name of the EKS cluster resource.
Type: string
node_subnets
Description: List of names for subnets that controller nodes should be deployed to
Type: list(string)
vpc_id
Description: The id for the VPC that the cluster should be deployed into
Type: string
Optional Inputs
The following input variables are optional (have default values):
bootstrap_cluster_creator_admin_privileges
Description: Whether to give cluster admin privileges to the cluster creator implicitly. Cannot be changed after cluster creation. For backwards compatibility purposes only.
Type: bool
Default: false
bootstrap_mode_enabled
Description: Whether the cluster is being bootstrapped and does not yet have the autoscaler enabled.
Type: bool
Default: false
control_plane_logging
Description: Which log streams to turn on for the control plane (will be sent to Cloudwatch and forwarded to DataDog)
Type: set(string)
Default: []
control_plane_subnets
Description: List of subnet names for the control plane. Must be in at least two different availability zones.
Type: set(string)
Default: []
dns_service_ip
Description: The IP address of the cluster's DNS service. Must be inside the service_cidr range.
Type: string
Default: "172.20.0.10"
enable_public_access
Description: Whether the cluster control plane should be available from the public internet.
Type: bool
Default: true
extended_support_enabled
Description: Whether to enable extended support for EOL Kubernetes versions.
Type: bool
Default: true
extra_admin_principal_arns
Description: Grants read-write access to most resources (not included top-level cluster configuration) to the indicated principals.
Type: list(string)
Default: []
extra_reader_principal_arns
Description: Grants read access to all resources (including secrets) to the indicated principals.
Type: list(string)
Default: []
extra_restricted_reader_principal_arns
Description: Grants read access to all resources (not including secrets) to the indicated principals.
Type: list(string)
Default: []
extra_superuser_principal_arns
Description: Grants read-write access to all resources to the indicated principals.
Type: list(string)
Default: []
kube_version
Description: Desired Kubernetes version to use for all subsystems. Use only major and minor specifiers as patches will be automatically applied.
Type: string
Default: "1.30"
node_ebs_volume_size_gb
Description: The size of the EBS volume in GiB to use for each node.
Type: number
Default: 40
node_security_groups
Description: Names of security groups allowed to communicate directly with the cluster nodes.
Type: set(string)
Default: []
public_access_cidrs
Description: IP address ranges that can access the public control plane API endpoint.
Type: set(string)
Default:
[
"0.0.0.0/0"
]
root_user_access_entry_enabled
Description: Whether to enable the root user access entry
Type: bool
Default: true
service_cidr
Description: CIDR block that kubernetes will use for assigning service and pod ID addresses.
Type: string
Default: "172.20.0.0/16"
Outputs
The following outputs are exported:
cluster_ca_data
Description: The CA certificate for the Kubernetes API server
cluster_name
Description: The name of the EKS cluster
cluster_region
Description: The AWS region to which the EKS cluster is deployed
cluster_url
Description: The URL for the Kubernetes API server
dns_service_ip
Description: The IP address of the cluster's DNS service.
node_instance_profile
Description: The instance profile assigned to controller nodes
node_role_arn
Description: The ARN of the role assigned to controller nodes
node_security_group_id
Description: ID of the security group assigned to controller nodes
user_data
Description: The user data file provided to the controller nodes
Usage
No notes