Kubernetes Node Image Cache Controller
This controller has two different components:
- Pinner: A DaemonSet that creates a pod on each node filled with sleep containers that use the images that you wish to cache.
- Prepuller: A set of "prepull" pods (one per cached imaged) that get launched on every new node added to the cluster. Each pod has a noop container that does nothing except immediately exit. This serves to "seed" the node with images as soon as it is created and augments the pinner as the prepuller downloads all images in parallel while the pinner must download images serially (i.e., slowly). 1
Many of the Panfactum modules have built-in integrations with this controller, and you can add additional images to the cache by leveraging the kube_node_image_cache submodule.
Debugging
Image Pull Errors
The pods that this controller generates may create temporary ErrImagePull
or ImagePullBackoff
errors.
This occurs when rates limits for AWS ECR (which is used for every image since ECR is used as a pull through cache) are exceeded due to the number of images in the cache. Especially when new nodes are created, dozens of images may be downloaded from ECR at once.
You can resolve this by requesting a service quota increase from AWS for the following quotas:
- Rate of BatchGetImage requests
- Rate of GetDownloadUrlForLayer requests
Providers
The following providers are needed by this module:
-
http (3.4.5)
-
kubectl (2.1.3)
-
kubernetes (2.34.0)
-
pf (0.0.7)
-
random (3.6.3)
Required Inputs
No required inputs.
Optional Inputs
The following input variables are optional (have default values):
pull_through_cache_enabled
Description: Whether to use the ECR pull through cache for the deployed images
Type: bool
Default: true
Outputs
No outputs.
Usage
No notes
Footnotes
-
Multiple images in the same pod are downloaded serially. ↩