AWS Networking

Objective

Deploy the core AWS networking primitives including your Virtual Private Cloud (VPC) and Route 53 Hosted Zone.

A Quick Note

Up to now, we have focused on setting up all environments in each section. Moving forward, this guide will focus on setting up infrastructure for a single environment at a time. We recommend starting with your production environment and then returning here for additional environments as needed.

Additionally, unless otherwise specified, the following sections of this guide only apply to environments that will be hosting Kubernetes clusters. Those that do not host live infrastructure (e.g., management) do not need much further setup.

Background

If you are new to cloud networking, we recommend that your review the concept documentation before proceeding.

Create your VPC

We will use the aws_vpc module to deploy the VPC. This includes not only the VPC but also configuration of subnets, routing tables, flow logs, etc. See the module reference docs for a more comprehensive overview.

Choose your CIDR Blocks and Subnets

One of the most important decision that you make throughout this entire guide is deciding your CIDR blocks for subnet setup. This is very difficult to change later and will usually require redeploying your entire VPC and all the resources it contains.

We strongly recommend choosing the largest possible CIDR block for your VPC: 10.0.0/16. You want to ensure that you have at least 100 IPs available in each public subnet and 1,000 IPs available in each private / isolated subnet. Choosing a large VPC CIDR gives you the most flexibility. 1

We also strongly recommend that you set up at least three of each subnet type (nine total). This allows you to have one subnet type in at least three availability zones which is required to achieve a highly available setup.

More specifically, we recommend you use the following subnet configuration): 2 3

If you are choosing a different network layout, we recommend this site for helping to divide your network.

Understanding NAT

If you are unfamiliar with NAT, you should stop now to review the NAT concept documentation.

NAT is the one component of the VPC configuration that we have enhanced beyond the typical AWS-recommended setup.

Specifically, we do NOT use AWS NAT Gateways by default. They are far too expensive for behavior that should ultimately be available for free (and is in other cloud providers). For many organizations, NAT Gateway costs alone can produce 10-50% of total AWS spend.

Instead, we deploy a Panfactum-enhanced version of the fck-nat project. Using this pattern, our module launches self-hosted NAT nodes in EC2 autoscaling groups and reduces the costs of NAT by over 90%.

See the aws_vpc documentation for more details and to decide if it is right for you (it probably is).

Deploy the AWS VPC Module

We will now deploy the aws_vpc module via terragrunt:

1. Set up a new aws_vpc directory in the primary region for your environment (not global).

2. Add a terragrunt.hcl that looks like this. Replace the subnet configuration with your chosen settings.

3. Run pf-tf-init to enable the required providers.

4. Run terragrunt apply.

5. Ensure that your VPC looks similar to the following:

   

6. Ensure that your NAT nodes are running and healthy:

   

   Note that each node should have a Public IPv4 address which should match its Elastic IP. All traffic from your cluster will appear to originate from one of these IP addresses, and they will remain the same for the lifetime of your VPC.

Test Connectivity

Let's verify that networking connectivity in the new VPC is working as intended.

We provide a test script pf-vpc-network-test that ensures:

• inbound traffic is rejected from your NAT nodes.
• nodes running in your private subnets are able to connect to the internet through a NAT IP.

Run the test by calling pf-vpc-network-test <path-to-aws_vpc-module>. For example, if running the test from inside the aws_vpc folder, you would run the test as pf-vpc-network-test .. 4

If the test completes successfully, you are ready to proceed!

Next Steps

Now that networking is set up in AWS, we can deploy your Kubernetes cluster.