Skip to content

Observability2 min read

The servers were fine and playback was not

Infrastructure monitoring will tell you the edge is healthy and the origin is responding. It cannot tell you that viewers on one mobile network are rebuffering every ninety seconds, because from the server's side every one of those requests succeeded.

  • telemetry
  • qoe
  • monitoring

There is a gap between "the system is up" and "it is watchable", and almost everything you can measure on a server sits on the wrong side of it.

Why the server cannot see it

A rebuffer is a client-side event. The player ran out of buffered media and stopped. From the edge's perspective the segment requests around that moment all returned 200 in reasonable time — because they did. The failure is in the relationship between arrival rate and playback rate, and only one end of that relationship is on your infrastructure.

The four numbers

Every player — web, phone, television — reports per session:

  • Time to first frame. The one users describe as "slow", and the one most affected by decisions in the first two seconds.
  • Rebuffer ratio. Time stalled over time watched. The number that correlates with abandonment.
  • Delivered bitrate distribution. Not the average — the distribution. An average hides a population pinned to the bottom rung.
  • Errors by class. Distinguishing "manifest 404" from "decoder failure" is the difference between a five-minute fix and a week.

Sliceable by network, device, region and title, because the finding is almost never global. It is one carrier, or one device generation, or one title with a bad ladder.

Self-hosted, and why that is not only about cost

Per-session licensing prices exactly the thing you are trying to grow. But the stronger argument is that the data is your viewers' behaviour, held in your own estate, with retention you set — and there is no third-party processor to declare in a privacy assessment.

Alert on it, do not report on it

Rebuffer ratio crossing a threshold for a network or a region is an alert. If the same fact reaches you as a line in a monthly report, you found out weeks after the viewers did.

The uncomfortable first month

Turning this on will surface problems that already existed. That is the point, and it is still unpleasant: the graph does not distinguish between "we made it worse" and "we can finally see it". Worth deciding in advance who owns the number, before the first bad week makes it a debate.

Is this the bill you are paying?

If the pattern in this write-up describes your setup, the next step is reading your actual numbers against it. Send twelve months of billing and your concurrency curve.