Sockets from Scratch

The socket API every client and server is built on — and the one bug that bites everyone the first time they leave the framework behind: TCP is a byte stream with no message boundaries, so one recv() is not one message.

A single PAIML course in Systems & Rust. You start from one fact — a socket is a file descriptor — walk the six calls (socket/bind/listen/accept/connect/send/recv), watch the connection lifecycle in ss, weigh TCP against UDP, and then build the framing layer that survives arbitrary partial reads. The capstone artifact is sockwire: a contract-governed Rust crate that is provably lossless and provably total.

Table of Contents

• What You'll Learn
• Course Outline
• Hands-On: sockwire
• Repository Layout
• Capstone
• Instructor
• License

What You'll Learn

Four weeks that climb from "a socket is a file descriptor" to a working framed TCP server, with the partial-read insight as the structural center.

• The socket as a file descriptor. A socket is a file descriptor you read, write, and close, and a TCP connection is named by its 5-tuple — protocol, source IP:port, destination IP:port — which is why two browser tabs to the same server are two different connections. You learn the address families (AF_INET, AF_INET6) and types (SOCK_STREAM, SOCK_DGRAM) and what each of socket, bind, listen, accept, connect, and close does.
• The connection lifecycle, in ss. The passive server path (bind → listen → accept) versus the active client path (connect), and why accept() returns a brand-new connected socket while the listener keeps listening. You read the TCP states — LISTEN, SYN-SENT, ESTAB, CLOSE-WAIT, TIME-WAIT — in ss, and learn why a just-restarted server hits "address already in use" and what SO_REUSEADDR does about it.
• TCP vs. UDP. SOCK_STREAM gives an ordered, reliable byte stream with no message boundaries; SOCK_DGRAM gives unordered, unreliable, message-preserving datagrams with no connection. You learn connected vs. unconnected UDP — a connected datagram socket has a fixed default peer, an unconnected one names the peer on every sendto.
• Reading and writing a stream — the key idea. send/recv over a connection that has no message boundaries: a single recv() may return half a message, two messages, or a message split across three calls. The fix is framing (length-prefix or newline-delimited) plus a buffer that survives partial reads, with a working understanding of blocking vs. non-blocking, read timeouts, and how select/poll/epoll let one thread watch many sockets.
• Build a server. A TcpListener accept() loop that hands each connection to a worker, a length-framed read loop driven by a bounded FrameBuffer, and graceful handling of a client that closes mid-frame — serving many clients without ever mistaking one recv() for one message.

Course Outline

Four modules, each shipping a short concept video, a runnable lab (ss/nc/tcpdump inspection, then the sockwire crate), and a reading set. The full lab commands live in labs.md; the module/lesson map is below (source of record: outline.md).

Module	Lesson	Core idea
1 · The Socket API	1.1 What a Socket Is	A socket is a file descriptor; the 5-tuple; socket/bind/listen/accept/connect/close
	1.2 The Connection Lifecycle	Passive vs. active; TCP states in ss; "address already in use" and SO_REUSEADDR
	1.3 TCP vs. UDP Sockets	SOCK_STREAM vs. SOCK_DGRAM; connected vs. unconnected UDP
2 · Reading and Writing a Stream	2.1 send, recv, and the Byte Stream	The key idea: a stream has no message boundaries; one recv() is not one message
	2.2 Framing Messages over a Stream	Length-prefix vs. newline-delimited framing; buffering partial reads
	2.3 Blocking, Non-Blocking, and Timeouts	Blocking vs. non-blocking; set_read_timeout; select/poll/epoll at the concept level
3 · Build a Server	3.1 A TCP Echo Server	The accept() loop; echo; orderly shutdown at end of stream
	3.2 A Frame Buffer for Partial Reads	A FrameBuffer that reassembles frames across any chunk boundary
	3.3 Serving Many Clients	Thread-per-connection vs. non-blocking/poll; many ESTAB sockets in ss
4 · Capstone Project	4.1 Capstone: Build sockwire	parse_host_port, encode_frame, FrameBuffer, and provable contracts

Hands-On: sockwire

The course artifact is sockwire — a small, dependency-free Rust crate that turns a stream of arbitrary recv() chunks into a clean sequence of whole messages. It is built contract-first under provable contracts (pv): the kernel contract contracts/sockwire-v1.yaml specifies the invariants before any code, contracts/sockwire-binding.yaml maps each equation to its implementing function, and make lint-contracts (pv validate) gates them. The crate exposes three things:

• parse_host_port(s: &str) -> Result<(String, u16), SockError> — splits a host:port string on the last : (so bracketed IPv6 like [::1]:80 keeps its host), total over any input: a missing colon, an empty host, or a non-numeric / out-of-range port is an Err, never a panic.
• encode_frame(payload: &[u8]) -> Result<Vec<u8>, SockError> — produces a length-prefixed frame: a u32 big-endian length, then the payload bytes. Output length is exactly LEN_PREFIX + payload.len().
• FrameBuffer::new(max_frame) with push(&mut self, &[u8]) and next_frame(&mut self) -> Result<Option<Vec<u8>>, SockError> — reassembles length-prefixed frames across arbitrary chunk boundaries: Ok(None) until a complete frame is buffered, then Ok(Some(payload)), retaining trailing bytes for the next call. A declared length over max_frame is Err(FrameTooLarge) — a bounded-buffer guard so a hostile peer cannot make the server allocate unbounded memory.

The invariant the whole course is about — one recv() is not one message. TCP delivers bytes, not messages, so a correct reader must impose its own framing and buffer across reads. FrameBuffer makes that a tested guarantee: feed the same byte stream all at once and one byte at a time, and both yield the identical sequence of frames — no byte lost, duplicated, or reordered. That property ships as a cargo test:

// Feed the same wire bytes all-at-once and one-byte-at-a-time;
// assert both reconstruct the exact same frames. (sockwire/src/lib.rs)
let mut wire = Vec::new();
for p in [b"hello".as_ref(), b"", b"a longer message", b"x"] {
    wire.extend_from_slice(&encode_frame(p).unwrap());
}

let mut fb = FrameBuffer::new(1024);
let mut got = Vec::new();
for byte in &wire {
    fb.push(std::slice::from_ref(byte));
    while let Some(frame) = fb.next_frame().unwrap() {
        got.push(frame); // same frames as the all-at-once run
    }
}
assert_eq!(fb.buffered(), 0);

Run the full Rust gate (format check, clippy with -D warnings, and the falsification tests):

make rust

The crate's tests are the contract's falsification tests: host:port parse totality and round-trip, chunking-independent reassembly, the max_frame DoS guard, and partial-frame totality (a prefix split across calls, then a payload split across calls, never panics).

Repository Layout

sockets-from-scratch/
├── README.md                  This file
├── outline.md                 Module/lesson outline (source of record)
├── labs.md                    Runnable labs: ss / nc / tcpdump + the sockwire crate
├── capstone.md                Capstone brief (deliverables, rubric, share-your-work)
├── coursera-assets/           Key-terms, reflections, role-plays, course-page, banners (SVG/MD)
├── slides/                    Per-lesson title-slide SVG animations
├── sockwire/                  The contract-governed Rust crate (parse_host_port, encode_frame, FrameBuffer)
├── contracts/                 sockwire-v1.yaml + binding (pv-valid)
├── assets/hero.svg            Course hero banner
├── Makefile                   lint / lint-contracts / rust / validate / check
├── .github/workflows/ci.yml   CI: markdown lint, structure validate, Rust gate, pv/pmat
└── LICENSE                    MIT

Rendered media (PNG/MP4) is regenerated from the committed SVG/Markdown/Lua sources and is never checked in.

Capstone

Implement and test sockwire — the framing layer — and use it in a tiny TCP echo/line server that parses host:port, frames messages, and survives partial reads. You ship the crate (read the reference implementation or pv scaffold your own against the contract), then wire it into a real accept()-loop server that is correct regardless of how the stream is chunked: the same logical messages emerge whether the client sends one frame per send, ten frames in one send, or one frame split across three. The full brief, evaluation tiers, and share-your-work prompt are in capstone.md.

Instructor

• Noah Gift — Founder, Pragmatic AI Labs · Duke University faculty

License

Course content © Pragmatic AI Labs. Code examples — including the sockwire crate — are released under the MIT License.