Skip to content

Delivery2 min read

The mid-film seek is the whole cache design

Everything interesting about a video cache follows from one interaction: a viewer dragging the scrubber into the middle of an object nobody has requested before.

  • caching
  • range-requests
  • origin

Ask what a cache should do and you get an answer about hit ratios. Ask what a video cache should do and the honest answer is: survive somebody skipping the first ninety minutes.

Why that one interaction dominates

A seek produces a range request into the middle of a large object. Three things follow immediately.

  1. The bytes requested may never have been fetched before, so this is a miss on an object the cache is otherwise holding.
  2. The bytes before them are irrelevant, so fetching the object from the start is wasted work at exactly the moment latency is visible.
  3. The viewer is waiting. A seek is the one moment where a stall is attributed to the service rather than to the network.

Slice-based storage

Objects are stored as fixed slices, addressed by content hash. A range maps onto the slices it overlaps; only those are fetched.

# Slice arithmetic for a 2 MB slice size.
#   seek to 01:30:00 in a 4.1 GB rendition
#   byte offset ≈ 2,415,919,104
#   slice index  = offset / 2 MB = 1152
#   fetch slices 1152..1157 (six), serve, cache
#
# The next viewer who seeks anywhere in that ten-second
# neighbourhood is served from disk.

Slice size is the one tunable that matters and it is a trade between two bad extremes. Too small and the index dominates; too large and a seek fetches material nobody watches — which is the whole-object problem again with extra steps.

Content classes behave differently

A blended hit ratio hides the case that hurts. Splitting by class is what makes the number actionable.

ClassAccess shapeWhat the cache should do
LiveEveryone at the same offset, onceHold the trailing window, evict fast
New releaseConcentrated, front-loadedHold whole, evict slowly
Long tailSparse, scattered offsetsHold slices, never whole objects

Sharing one policy across all three means the live surge evicts the long tail's working set every time there is an event, and the archive's hit ratio collapses in a way the overall average never shows.

Which is an argument for separate edges

Not separate policies on one machine — separate machines. A policy boundary does not stop a surge consuming the page cache and the disk queue that the other class needs.

Is this the bill you are paying?

If the pattern in this write-up describes your setup, the next step is reading your actual numbers against it. Send twelve months of billing and your concurrency curve.