The application tier
Catalogue, search, authentication, entitlements, billing and the managed database. Small payloads, spiky load, and a bill that stays in the noise.
Rent the elastic tier. Own the expensive one.
Application on AWS, delivery and origin on your own metal. The tier that spikes gets cloud elasticity; the tier that bills by the gigabyte never enters a metered path. This is the pattern that moves a streaming bill the most, and it is the one most services should be running.
The boundary is drawn where the money is
You want cloud elasticity for the parts that spike and you refuse to pay metered egress for the parts that do not need to be rented.
Control traffic — catalogue lookups, sign-in, billing calls — is small, bursty and latency-sensitive. That is the exact workload public cloud prices well and operates well. Media traffic is the opposite: enormous, steady and utterly undifferentiated. Renting it is the single most expensive decision a streaming service makes, and it is the one most services make by default because it came bundled with everything else.
So the boundary goes between them. AWS carries the application tier, where the bill stays small and elasticity genuinely earns its keep. Your own hardware carries encoding, origin storage and the delivery edge — the three line items that grow with every viewer you succeed in attracting.
Two estates would normally mean two operating models, and that is the reason most teams never try this. It is only acceptable because both sides are declared in the same Terraform and shipped by the same pipeline. One merge request, two providers, one workflow.
How it fits together
Blue is yours. Grey is rented. Every media byte leaves from the blue side.
Untrusted — the public internet
- Web player
- Mobile apps
- TV apps
- DNS
No media byte is ever served from here
- Load balancer
- Platform APIs
- session-authz
- Managed database
- Object storage
- CDN
Every media byte leaves from here
- traefik-edge
- CrowdSec
- Live path
- VOD path
The bytes never enter a metered path
- live-transcode
- live-origin
- vod-transcode + GPU
- vod-origin-cache
- MinIO
- key-delivery
- OpenBao
- ClickHouse
The same pipeline provisions AWS and the metal
- Proxmox VE
- Terraform
- Ansible
- Cost controls
- Prometheus
- Grafana
What sits on which side
The split is not a compromise between two options. Each tier is placed where its cost and failure behaviour belong.
Encoding, origin and delivery
GPU transcoding, an S3-compatible MinIO origin, and a compiled edge serving live and VOD. Capital you have already spent, not a meter that runs.
One control plane
The AWS and Proxmox providers sit in the same Terraform. A change to either estate is the same merge request, reviewed the same way, applied by the same pipeline.
The partial-failure case
If AWS becomes unreachable, sign-in and catalogue degrade while playback of already-authorised sessions continues from your edge. That split is deliberate, and it is tested rather than assumed.
Placement and billing shape
Read the right-hand column first. It is the argument.
| Tier | Where it runs | Why there | Billing shape |
|---|---|---|---|
| Catalogue & search | AWS | Spiky, tiny payloads | Per request, negligible |
| Auth & entitlements | AWS | Must be globally reachable | Per request, negligible |
| Billing | AWS | Availability matters more than cost | Per request, negligible |
| Encoding | Your metal | GPU already owned; no per-minute meter | Capital, already spent |
| Origin storage | Your metal | Storage plus egress is the worst rented pair | Capital, already spent |
| Delivery / CDN | Your edge | Egress is the largest recurring cost in streaming | Bandwidth commit only |
| Control plane | Git | One workflow across both estates | None |
The honest trade-offs
What you gain
AWS handles the spiky, low-volume tier and its bill stays small. The high-volume tier — egress, transcode, storage — runs on capacity you have already paid for.
What you take on
Two estates to operate. That is only acceptable because both are declared in the same Terraform and shipped by the same pipeline, so it is one workflow, not two.
The failure mode to design for
If AWS is unreachable, sign-in and catalogue degrade but playback of already-authorised sessions continues from your edge. That split is deliberate and it is tested.
The things people ask first
Isn't running two estates twice the operational load?
It would be, if they were operated separately. They are not. Both are declared in one Terraform configuration and shipped by one pipeline, so the day-to-day is a single workflow. The second estate adds hardware to look after, not a second way of working.
What if we already have a CDN contract?
Then we start by measuring what it actually costs per delivered gigabyte, including the origin egress feeding it. Sometimes the answer is to keep it for overflow and serve the baseline yourself. That is a legitimate outcome and we will say so.
Does the AWS side lock us in?
It is deliberately the tier with the least gravity — stateless services and a standard PostgreSQL. Nothing in the application tier depends on a proprietary service that has no equivalent elsewhere. Moving it is a provider change in Terraform.
How do you keep the AWS bill from drifting?
Budgets and anomaly alerts per service, right-sizing reviewed against measured concurrency, and commitments bought against a baseline that has actually been observed rather than forecast. Cost is treated as an ongoing engineering discipline, not a quarterly surprise.
Bring your AWS bill
The fastest way to know whether this pattern is worth it for you is to look at where your current bill actually goes. Send it, and you get an architecture and a number back.