Part 5 · 2 chapters · ~18 min

Networking Down to Packets

An HTTP request wrapped in TCP, IP and Ethernet and unwrapped hop by hop, with UDP and QUIC beside it; then TCP in motion: the handshake, slow start, loss and congestion control, head-of-line blocking, socket buffers and flow control, and NIC queues with RSS and offloads.

10

Ethernet, IP, TCP and UDP

headers inside headers
  1. The application writes bytes to a socket.
  2. TCP adds ports, sequence numbers, acknowledgements and a window.
  3. IP adds addresses and a TTL, and routers forward hop by hop.
  4. The link layer wraps each hop in a new frame.
  5. UDP has ports and a checksum and nothing more. QUIC builds on it.
  6. Unwrapping: each layer checks and strips its own header.
code
# look at your own packets (macOS / Linux, needs sudo)
sudo tcpdump -i any -n 'tcp port 443 and host api.example.com' -c 20
# the three-way handshake: [S], [S.], [.]; then data with seq/ack numbers; [F.] to close
traceroute -n api.example.com          # every router hop and its round-trip time
A PACKET, LAYER BY LAYER
an HTTP request wrapped in TCP, wrapped in IP, wrapped in an Ethernet frame, then unwrapped hop by hop
swipe the figure sideways, or tap expand for full screen
1/6
application
Application: the browser writes "GET /api/balance HTTP/1.1…" (inside TLS, so it is ciphertext on the wire) to a socket. The application sees a reliable byte stream and nothing below it.
11

TCP in motion, sockets and NIC queues

reliability, speed control, and the plumbing
  1. The handshake costs one round trip before any data.
  2. Slow start begins at about 14 KB in flight and doubles each round trip.
  3. Loss halves the window. BBR models the path instead of reacting to loss.
  4. Head-of-line blocking comes from in-order delivery. QUIC removes it per stream.
  5. Socket buffers provide flow control.
  6. NIC queues, receive-side scaling and segmentation offloads.
the frontend link
On a fresh connection, roughly the first 14 KB of a response arrive in the first round trip. The rest waits for slow start. Inline critical CSS and keep the first screen's HTML small (the Browser course part 3), because each extra round trip from Lagos to Europe is about 100 ms. HTTP/3's per-stream recovery helps most on the lossy mobile links your users actually have.
TCP IN MOTION, SOCKETS AND NIC QUEUES
the handshake, the congestion window growing and halving, and how packets get from the wire into your process
swipe the figure sideways, or tap expand for full screen
1/6
handshake
The handshake: SYN, SYN-ACK, ACK: one round trip before data, so a new connection to a server 100 ms away costs 100 ms before the first byte (plus TLS: the Browser course part 1). Connection reuse and keep-alive exist to pay this once.