Skip to content

Building block · storage

An S3-compatible origin you control

Object storage is the one place where a rented decision quietly compounds: you pay to keep the bytes, and then you pay again every time anyone reads them. S3-compatibility is what makes that decision reversible.

Storage plus egress is the worst rented pair

Holding a catalogue is cheap. Serving it is not, and in a rented object store those two are billed separately — which means a popular title costs more to own than an unpopular one, forever. Moving origin onto capacity you control removes the second half of that bill entirely.

The reason it can be moved at all is the S3 API. Every service that reads or writes media speaks it, so the origin is a target rather than a dependency. That is a deliberate design constraint, and it is what makes the hybrid pattern possible.

Detail

How the origin is built

Compatibility

S3 API, deliberately

Everything speaks S3. The origin can be MinIO on your metal or a cloud bucket, and swapping one for the other is a configuration change.

  • MinIO on owned hardware
  • Cloud buckets as an alternative target
  • No proprietary client anywhere

Layout

Sharded, content-addressed

Objects are laid out by content hash across a sharded prefix, so a catalogue of hundreds of thousands of segments stays fast to address.

  • Content-hash addressing
  • Sharded prefixes
  • No unlistable directories

Durability

Erasure coded

Drive failure is an operational event, not an incident. Capacity is added by adding drives and then nodes.

  • Erasure-coded sets
  • Drive failure tolerated
  • Grows by drive, then by node

Lifecycle

Retention in code

Expiry and tiering rules live in Terraform, so a retention policy is enforced by the system rather than by somebody remembering.

  • Lifecycle rules in Terraform
  • Access-matched tiering
  • Auditable deletion

Specification

Origin characteristics

ConcernRented object storeOwned origin
Capacity costPer gigabyte, per monthDrives, amortised
Read costPer request and per gigabyte outNone
Popular titlesCost more the more they are watchedCost the same
APIS3S3
LifecycleConsole or policy fileTerraform, reviewed
PortabilityEgress fee to leaveCopy at line rate

Questions

The things people ask first

Is MinIO production-ready for a real catalogue?

It runs the origin for a live consumer service on this platform today. The important properties are the S3 API and erasure coding, and both are well proven.

What about off-site backup?

A cloud bucket makes an excellent second copy — cheap to write, rarely read. Owning the origin does not mean owning every copy.

How do we migrate an existing catalogue in?

A bulk copy over the S3 API, verified by content hash. It is bounded by bandwidth and by the egress fee your current provider charges to let the data leave.

What is your storage actually costing?

Capacity is the small half. Ask what the reads cost, and whether a lifecycle policy has ever been applied.