There is a gap between "the system is up" and "it is watchable", and almost
everything you can measure on a server sits on the wrong side of it.
Why the server cannot see it
A rebuffer is a client-side event. The player ran out of buffered media and
stopped. From the edge's perspective the segment requests around that moment all
returned 200 in reasonable time — because they did. The failure is in the
relationship between arrival rate and playback rate, and only one end of that
relationship is on your infrastructure.
The four numbers
Every player — web, phone, television — reports per session:
- Time to first frame. The one users describe as "slow", and the one most
affected by decisions in the first two seconds.
- Rebuffer ratio. Time stalled over time watched. The number that correlates
with abandonment.
- Delivered bitrate distribution. Not the average — the distribution. An
average hides a population pinned to the bottom rung.
- Errors by class. Distinguishing "manifest 404" from "decoder failure" is
the difference between a five-minute fix and a week.
Sliceable by network, device, region and title, because the finding is almost
never global. It is one carrier, or one device generation, or one title with a
bad ladder.
Self-hosted, and why that is not only about cost
Per-session licensing prices exactly the thing you are trying to grow. But the
stronger argument is that the data is your viewers' behaviour, held in your own
estate, with retention you set — and there is no third-party processor to
declare in a privacy assessment.
Alert on it, do not report on it
Rebuffer ratio crossing a threshold for a network or a region is an alert. If
the same fact reaches you as a line in a monthly report, you found out weeks
after the viewers did.
The uncomfortable first month
Turning this on will surface problems that already existed. That is the point,
and it is still unpleasant: the graph does not distinguish between "we made it
worse" and "we can finally see it". Worth deciding in advance who owns the
number, before the first bad week makes it a debate.