Part 2 · 3 chapters · ~25 min

Networking, Properly

A VPC piece by piece: CIDRs, subnets per zone, route tables, internet and NAT gateways; security groups versus NACLs, peering and transit, private endpoints and DNS; L4 and L7 load balancing and the CDN; and one packet from a phone in Lagos to a container in a private subnet and back.

VPCs, subnets, routes and gateways

a private network you design

The VPC is a private CIDR block in one region. Plan ranges company-wide so they never overlap.
Subnets: public, private-app and private-data in each zone. AWS subnets are zonal; GCP subnets are regional.
Route tables: every table has the local route, and the default route decides everything else.
Internet gateway: a public subnet routes 0.0.0.0/0 to it. Only load balancers and NAT gateways live there.
NAT gateway: gives private subnets outbound-only access. Run one per zone, and remember it bills per GB.
Data subnet: no internet route at all. Cloud APIs are reached through endpoints.

code

# terraform: the shape, not every argument
resource "aws_vpc" "main" { cidr_block = "10.0.0.0/16"  enable_dns_hostnames = true }

resource "aws_subnet" "public" {
  for_each          = { a = 0, b = 1, c = 2 }
  vpc_id            = aws_vpc.main.id
  availability_zone = "eu-west-1${each.key}"
  cidr_block        = cidrsubnet("10.0.0.0/16", 8, each.value)        # 10.0.0.0/24 …
}
resource "aws_subnet" "app" {
  for_each   = { a = 10, b = 11, c = 12 }
  cidr_block = cidrsubnet("10.0.0.0/16", 8, each.value)               # 10.0.10.0/24 …
  # …
}

resource "aws_route_table" "app" {
  for_each = aws_subnet.app
  vpc_id   = aws_vpc.main.id
  route { cidr_block = "0.0.0.0/0"  nat_gateway_id = aws_nat_gateway.per_az[each.key].id }
}

the CIDR mistake

Using 10.0.0.0/16 for every environment in every account is fine until the day you need to peer staging with a partner, or join an acquired company's network. Allocate ranges from one company-wide plan from day one. Renumbering a live VPC means rebuilding it.

A VPC, PIECE BY PIECE

a private network in a region, split into subnets per zone, with routes deciding which subnets can reach the internet and how

swipe the figure sideways, or tap expand for full screen

1/6

the VPC

The VPC: 10.0.0.0/16 gives 65,536 private addresses in one region. Pick ranges that will not collide with other VPCs, offices or partners you might ever connect to (peering and VPNs cannot join overlapping ranges); a common plan is a /16 per environment per region from a company-wide allocation.

Firewalls, connecting VPCs, private endpoints and DNS

who can talk to whom, and how

Security groups are stateful, allow-only and attached per interface. Reference other groups instead of IP ranges.
Network ACLs are stateless, ordered and attached per subnet. Use them only as a coarse second layer.
Peering is private routing between two VPCs. It is not transitive.
A transit hub takes one attachment per VPC and decides routing centrally.
Private endpoints reach cloud services and SaaS without NAT and without touching the internet.
Private DNS makes service names resolve to endpoint IPs and gives your own services internal names.

	security group (AWS)	network ACL (AWS)	firewall rule (GCP)
attached to	network interface	subnet	VPC network, targeting tags or service accounts
state	stateful	stateless	stateful
rules	allow only	allow and deny, numbered order	allow and deny, priority
best source reference	another security group	CIDR only	a service account
use it for	everything fine-grained	coarse subnet-level blocks	everything

the cost angle

An ECS service pulling images from ECR and writing to S3 through a NAT gateway pays NAT data processing on every byte. An S3 gateway endpoint is free, and ECR interface endpoints cost a few dollars a month. For image-heavy workloads this is often the biggest single saving on the bill.

SECURITY GROUPS, NACLS, PEERING AND PRIVATE ENDPOINTS

the two firewalls, the ways VPCs connect, and how to reach cloud services without touching the internet

swipe the figure sideways, or tap expand for full screen

1/6

security groups

Security groups: a stateful firewall per network interface. Rules only allow (no deny), and a response to an allowed request is automatically allowed back. The powerful pattern is referencing other security groups instead of IP ranges: "db-sg allows 5432 from app-sg" means any instance in the app group, wherever it moves, and nothing else.

Load balancing, CDN, and the packet's path

one request, every hop

DNS answers with the CDN's anycast address or the nearest region. Keep TTLs short for anything you might fail over.
The edge terminates TLS nearby, applies the WAF, caches what it can and forwards over the backbone.
An L7 load balancer routes by host and path to healthy targets in any zone.
Inside the VPC, security groups reference each other and endpoints keep cloud API calls private.
Outbound through NAT: the address your partners allowlist is the NAT's elastic IP.
The budget: the ocean and third parties dominate. Everything else is small unless it is misconfigured.

	L4 (NLB, TCP/UDP proxy)	L7 (ALB, HTTP(S) LB)
reads	IP and port	HTTP: host, path, headers, cookies
routing	by port to a target group	by host and path rules, weights, headers
TLS	passes through or terminates	terminates (certificates managed here)
latency	lowest; millions of connections	slightly higher; per-request features
static IP	yes (one per zone)	no on AWS (use Global Accelerator); yes on GCP global LB
use it for	non-HTTP, gRPC passthrough, extreme throughput, fixed IPs	web and API traffic, path routing, WAF, auth at the edge

the 502 and the 504

A 502 from the load balancer means the target answered badly: it closed the connection, returned garbage, or crashed mid-response. A 504 means the target did not answer within the LB's timeout. Both are logged at the LB with the target's address, so read the LB access log before the app log. The usual culprit is an app keep-alive timeout shorter than the LB idle timeout, so the app closes connections the LB still thinks are open.

ONE PACKET, END TO END

a tap on "Pay" in Lagos, followed through DNS, the edge, the load balancer, the VPC and back

swipe the figure sideways, or tap expand for full screen

1/6

DNS

DNS: pay.example.com resolves through the user's resolver to the authoritative DNS (Route 53, Cloud DNS, Cloudflare), which answers with the CDN's anycast address (or, with latency-based or geo routing, the nearest region's load balancer). TTLs decide how fast a failover reaches users: 60 seconds for anything you might move.