The servers were fine and playback was not

There is a gap between "the system is up" and "it is watchable". Almost everything you can measure on a server sits on the wrong side of it.

Why the server cannot see it

A rebuffer is a client-side event: the player ran out of buffered media and stopped. From the edge's perspective the segment requests around that moment all returned 200 in reasonable time, because they did. The failure is in the relationship between arrival rate and playback rate, and only one end of it is on your infrastructure.

The four numbers

Every player, on web, phone and television, reports per session:

• Time to first frame. The one users describe as "slow", and the one most affected by decisions in the first two seconds.
• Rebuffer ratio. Time stalled over time watched. The number that correlates with abandonment.
• Delivered bitrate distribution, which is the distribution and not the average. An average hides a population pinned to the bottom rung.
• Errors by class. Distinguishing "manifest 404" from "decoder failure" is the difference between a five-minute fix and a week.

Sliceable by network, device, region and title, because the finding is almost never global. It is one carrier, or one device generation, or one title with badly chosen quality levels.

Self-hosted, and why that is not only about cost

Per-session licensing prices the very thing you are trying to grow. The stronger argument is that the data is your viewers' behaviour, held on your own infrastructure, with retention you set. There is no third-party processor to declare in a privacy assessment.

The uncomfortable first month

Turning this on will surface problems that already existed. That is the aim, and it is still unpleasant: the graph does not distinguish between "we made it worse" and "we can finally see it". Decide in advance who owns the number, before the first bad week makes it a debate.