Why we compiled our own delivery edge
A general-purpose cache treats a four-gigabyte film like a large web page. Video is not that — viewers seek, abandon and resume, and every one of those is a range request. This is what changed when we stopped renting the tier that answers them.
Delivery is the largest recurring cost in streaming and the one where paying more buys the least differentiation. That combination is unusual, and it is exactly what makes the tier worth owning: it is expensive, and nobody is going to out-innovate anybody else at moving a byte from a disk to a socket.
What is worth engineering is the behaviour around that byte. This is a note on which behaviours turned out to matter, and which ones we assumed would and did not.
The request pattern is not the web's
A web cache is tuned for many small objects fetched whole. Video is a handful of enormous objects fetched in fragments, in an order nobody can predict, by clients that give up halfway through more often than they finish.
The clearest case is a viewer dragging the scrubber to the ninety-minute mark. That is a range request landing in the middle of a large object, for bytes that may never have been requested before and may never be requested again.
| Behaviour | Generic cache | What video needs |
|---|---|---|
| Mid-object range | Fetch the whole object, or miss | Serve the slice, cache the slice |
| Partial object | Usually not cacheable | The normal case, not the exception |
| Abandonment | Wasted origin fetch | Nothing beyond what was watched |
| Object size | Kilobytes | Gigabytes |
A cache that cannot store part of an object treats every seek as a full origin fetch. At a few thousand concurrent viewers that is not a cost problem, it is a capacity problem — the origin becomes the bottleneck for content the edge is already holding.
Slices, not objects
The change that mattered most was the smallest to describe: cache in fixed slices rather than whole objects. A range request maps onto the slices it overlaps, those slices are fetched and stored independently, and a second viewer seeking near the same point is served from disk.
# What the origin sees, before and after, for one viewer who
# joins at 00:00, seeks to 01:30:00, then abandons.
#
# whole-object cache
# GET /titles/abc/master.m3u8 304
# GET /titles/abc/video-1080p.mp4 206 (4.1 GB fetched)
#
# slice cache, 2 MB slices
# GET /titles/abc/master.m3u8 304
# GET /titles/abc/video-1080p.mp4 206 (12 MB fetched)
The second number is not a saving on bandwidth alone. It is the difference between an origin that serves misses and an origin that is a second copy of the delivery tier.
HTTP/3 earns its keep where the network is worst
This one we expected to be marginal and it was not. On a fixed line, HTTP/2 and HTTP/3 are hard to tell apart. On a congested mobile network with real packet loss they are not: head-of-line blocking in HTTP/2 means one lost packet stalls every stream on the connection, and a stalled segment fetch during playback is a rebuffer.
The population that benefits is precisely the population most likely to abandon, which makes it a retention change wearing a transport-layer costume.
What we got wrong
Two things, both worth writing down.
- Geographic routing mattered less than peering. We spent effort on distance-aware node selection before establishing where traffic actually exits. Being one hop away over a local exchange beats being physically closer over transit, most of the time, by a margin that made the first piece of work largely irrelevant.
- Cache size mattered less than cache policy. Adding disk to a node with a policy that evicts an archive's working set on every live surge buys almost nothing. Separating live and on-demand onto their own edges — same hardware budget, different failure domains — did more than doubling storage would have.
What it costs
Owning this tier converts a per-gigabyte meter into a bandwidth commitment and some hardware. That is a better cost shape for a subscription business, and it is not automatically a smaller number. It becomes one at the point where delivered volume is predictable and large enough that the commitment is cheaper than the meter — which is a question about your traffic, not about the architecture.
The honest version of the recommendation is on the cost reduction page: read the bill first, decide second.
Is this the bill you are paying?
If the pattern in this write-up describes your setup, the next step is reading your actual numbers against it. Send twelve months of billing and your concurrency curve.