Why we compiled our own delivery edge

Delivery is the largest recurring cost in streaming and the one where paying more buys the least differentiation. That combination is what makes the tier worth owning. It is expensive, and moving a byte from a disk to a socket is not where anyone out-innovates anyone else.

What is worth engineering is the behaviour around that byte. This is a note on which behaviours turned out to matter, and which ones we assumed would and did not.

The request pattern is not the web's

A web cache is tuned for many small objects fetched whole. Video is a handful of enormous objects fetched in fragments, in an unpredictable order, by clients that give up halfway through more often than they finish.

The clearest case is a viewer dragging the scrubber to the ninety-minute mark. That is a range request landing in the middle of a large object, for bytes that may never have been requested before and may never be requested again.

A cache that cannot store part of an object treats every seek as a full origin fetch. At a few thousand concurrent viewers that is a capacity problem, not a cost one. The origin becomes the bottleneck for content the edge already holds.

Slices, not objects

The change that mattered most was the smallest to describe: cache in fixed slices, not whole objects. A range request maps onto the slices it overlaps. Those slices are fetched and stored independently, so a second viewer seeking near the same point is served from disk.

# What the origin sees, before and after, for one viewer who
# joins at 00:00, seeks to 01:30:00, then abandons.
#
# whole-object cache
#   GET /titles/abc/master.m3u8          304
#   GET /titles/abc/video-1080p.mp4      206  (4.1 GB fetched)
#
# slice cache, 2 MB slices
#   GET /titles/abc/master.m3u8          304
#   GET /titles/abc/video-1080p.mp4      206  (12 MB fetched)

The second number is not a saving on bandwidth alone. It is the difference between an origin that serves misses and an origin that is a second copy of the delivery tier.

HTTP/3 earns its keep where the network is worst

We expected this to be marginal and it was not. On a fixed line, HTTP/2 and HTTP/3 are hard to tell apart, and on a congested mobile network they are not. Head-of-line blocking in HTTP/2 means one lost packet stalls every stream on the connection, and a stalled segment fetch is a rebuffer.

The population that benefits is precisely the population most likely to abandon, which makes it a retention change wearing a transport-layer costume.

What we got wrong

Two things, both worth writing down.

• Geographic routing mattered less than peering. We spent effort on distance-aware node selection before establishing where traffic exits. One hop away over a local exchange beats being physically closer over transit, by a margin that made the first piece of work largely irrelevant.
• Cache size mattered less than cache policy. Adding disk to a node with a policy that evicts an archive's working set on every live surge buys almost nothing. Separating live and on-demand onto their own edges, on the same hardware budget, did more than doubling storage would have.

What it costs

Owning this tier converts a per-gigabyte charge into a bandwidth commitment and some hardware. That is a better cost shape for a subscription business, and it is not automatically a smaller number. It becomes one once delivered volume is predictable and large enough that the commitment is cheaper, which is a question about your traffic.

The full version of the recommendation is on the cost reduction page: read the invoices first, decide second.