# Panfactum Changelog — edge.24-06-02

> Replaces EKS CoreDNS with a custom module, adds monitoring stack with Prometheus and Grafana, introduces Argo Workflow engine, and makes significant improvements to cluster resource utilization and stability.

## Highlights

- EKS CoreDNS addon replaced by [`kube_core_dns`](https://panfactum.com/docs/edge/reference/infrastructure-modules/direct/kubernetes/kube_core_dns) — manual migration required
- New monitoring stack with Prometheus, Thanos, and Grafana via [`kube_monitoring`](https://panfactum.com/docs/edge/reference/infrastructure-modules/direct/kubernetes/kube_monitoring) (Alpha)
- New Argo Workflow engine via [`kube_argo`](https://panfactum.com/docs/edge/reference/infrastructure-modules/direct/kubernetes/kube_argo) (Alpha)
- New [`kube_vault_proxy`](https://panfactum.com/docs/edge/reference/infrastructure-modules/submodule/kubernetes/kube_vault_proxy) to add SSO to web assets without integrated SSO
- `pgbouncer_read_only_enabled` now defaults to `false` — breaking change for read-only pooler users

## Breaking Changes

- The EKS CoreDNS addon has been removed and replaced with the [`kube_core_dns`](https://panfactum.com/docs/edge/reference/infrastructure-modules/direct/kubernetes/kube_core_dns) module in order to provide better guarantees about
  - Impacts: iac-module `kube_core_dns` — New module replacing EKS CoreDNS addon
- [`kube_pg_cluster`](https://panfactum.com/docs/edge/reference/infrastructure-modules/submodule/kubernetes/kube_pg_cluster) has two new flags, `pgbouncer_read_only_enabled` (default `false`) and `pgbouncer_read_write_enabled` (default `true`), which enable the `r` and `rw` poolers, respectively.
  - Impacts: iac-module `kube_pg_cluster` — New pgbouncer_read_only_enabled and pgbouncer_read_write_enabled flags
- Label selectors in [`kube_pod`](https://panfactum.com/docs/edge/reference/infrastructure-modules/submodule/kubernetes/kube_pod) have been stabilized but require a one-time manual deletion of already-deployed Deployments (e.g., `bastion/bastion`) before re-applying.
  - Impacts: iac-module `kube_pod` — Stabilized label selectors requiring one-time Deployment deletion

## Additions

- (Alpha) Added a monitoring stack [`kube_monitoring`](https://panfactum.com/docs/edge/reference/infrastructure-modules/direct/kubernetes/kube_monitoring) which includes HA Prometheus, Prometheus Operator, Thanos metrics storage on S3, Node Exporter, kube-state-metrics,
  - Impacts: iac-module `kube_monitoring` — New module with Prometheus, Thanos, and Grafana monitoring stack
- (Alpha) Added the Argo Workflow engine to the stack via the [`kube_argo`](https://panfactum.com/docs/edge/reference/infrastructure-modules/direct/kubernetes/kube_argo) module.
  - Impacts: iac-module `kube_argo` — New module providing Argo Workflow engine
- New module [`kube_vault_proxy`](https://panfactum.com/docs/edge/reference/infrastructure-modules/submodule/kubernetes/kube_vault_proxy) that can be used to add SSO to web assets that do not have integrated SSO.
  - Impacts: iac-module `kube_vault_proxy` — New module for adding SSO to web assets
- [`kube_redis_sentinel`](https://panfactum.com/docs/edge/reference/infrastructure-modules/submodule/kubernetes/kube_redis_sentinel) has a new flag, `lfu_cache_enabled`, to configure automatic LFU-based eviction under memory pressure.
  - Impacts: iac-module `kube_redis_sentinel` — New lfu_cache_enabled flag for LFU-based eviction

## Improvements

- Added the standard Restricted Reader role to Vault instances and updated [`vault_auth_oidc`](https://panfactum.com/docs/edge/reference/infrastructure-modules/direct/vault/vault_auth_oidc) to take `restricted_reader_groups`.
  - Impacts: iac-module `vault_auth_oidc` — New restricted_reader_groups input added
- Reduced controller node costs by ~40% by recommending `controller_node_count = 1` and `controller_node_instance_types = ["t3a.medium"]` after bootstrapping.
- Added scheduling constraints to prevent critical workloads from scheduling all pods on the same instance type.

## Fixes

- PVCs for postgres instances were inadvertently created with duplicated entries for accessModes. This has been fixed (existing PVCs are unaffected

## Upgrade Instructions

## Migrate EKS CoreDNS to `kube_core_dns`

The EKS CoreDNS addon has been replaced with the
[`kube_core_dns`](https://panfactum.com/docs/edge/reference/infrastructure-modules/direct/kubernetes/kube_core_dns) module.

1. Add the `dns_service_ip` input to [`aws_eks`](https://panfactum.com/docs/edge/reference/infrastructure-modules/direct/aws/aws_eks)
   deployments by following [this guide](https://panfactum.com/docs/edge/guides/bootstrapping/kubernetes-cluster#choose-a-service-cidr).
   Double check that the `dns_service_ip` is the same IP as defined by `kube-system/kube-dns`.

2. Additionally, set `core_dns_addon_enabled` to `true`.

3. Apply the updated `aws_eks` module.

4. Add the `cluster_dns_service_ip` input to your
   [`kube_karpenter_node_pools`](https://panfactum.com/docs/edge/reference/infrastructure-modules/direct/kubernetes/kube_karpenter_node_pools)
   module, and re-apply the module. Ensure that all of your nodes have been replaced with the new configuration.

5. Deploy `kube_core_dns` by following
   [this guide](https://panfactum.com/docs/main/guides/bootstrapping/internal-cluster-networking#deploy-coredns). Note that this deployment
   will fail as the original addon service is still running and the IP is already taken.

6. Delete `kube-system/kube-dns` and re-apply `kube_core_dns`. Note that while the service is deleted, DNS will be
   temporarily unavailable in your cluster.

7. Once you've validated that DNS is working in the cluster, remove the `core_dns_addon_enabled` input from the
   `aws_eks` module and re-apply.

## Fix Label Selectors

Label selectors in [`kube_pod`](https://panfactum.com/docs/edge/reference/infrastructure-modules/submodule/kubernetes/kube_pod) have been
stabilized, but require a one-time manual deletion of already-deployed Deployments before re-applying.

Delete the `bastion/bastion` deployment (and any other deployments created by
[`kube_deployment`](https://panfactum.com/docs/edge/reference/infrastructure-modules/submodule/kubernetes/kube_deployment)) before running
`terragrunt apply`.

## Related Resources

- [JSON Data](https://panfactum.com/docs/changelog/edge.24-06-02.json): Machine-readable data
- [Channel Release List](https://panfactum.com/docs/changelog/edge.json): All releases in this channel