Shebaka Reliability & recovery
What happens when things fail.
Machines fail. This page says exactly what that means for your service and your data — what recovers on its own, what we restore by hand, and the numbers behind both. No fine print: if a limit exists, it is written here.
Two shapes of service, two recovery stories
Everything below follows from one question: does your service carry data? A service is stateless when it keeps nothing on disk — the app itself. It is stateful the moment it has a persistent volume or an attached managed database. Code can run in two places at once; your data lives in exactly one, so the two recover differently, and the console tells you which story applies in the service's DR tab.
Stateless: a warm standby in a second region
Turn on DR from the service drawer and we run a second, identical copy of your app on a different cluster — a different provider in a different data centre. Every deploy updates both. The standby is metered as its own line on your statement, at the same published rates, so the cost of the insurance is a number you can read, not a bundle.
- Failover
- Automatic at the edge: when the primary stops answering, traffic serves from the standby. Manual failover and failback buttons in the console give you the same control deliberately, for planned work.
- Data loss (RPO)
- None — a stateless service has no data to lose. Requests in flight during the cutover can fail and retry.
- Downtime (RTO)
- The edge health-checks the primary continuously; cutover is seconds to a few minutes, not hours.
- Drift
- The standby's running version is recorded and compared on every deploy — a standby that silently fell behind is flagged in the console and the daily digest, never assumed healthy.
Stateful: backups, and an honest restore path
A service with a volume or an attached database cannot run a warm standby today: its data lives on one cluster, and pretending otherwise would be selling you a failover that loses your writes. The console refuses the toggle and says so. What protects you instead is a layered backup posture — and, for a service with a managed database, a cutover you can run yourself: the database rebuilt on another cluster from the off-node archive, verified by exact row count before anything is pointed at it, the service redeployed there, traffic routed.
- Managed databases
- A nightly dump inside the cluster covers the everyday accident — a bad migration, a mistaken delete. Off-node protection is a daily base backup shipped to independent object storage under a different provider, so a lost node or a lost provider does not take the backup with it. The database drawer's Backups panel lists the actual backup objects — what exists, not what a policy promises.
- Volumes
- Snapshotted nightly and shipped off-node, 14-day retention — the same node-loss posture.
- Data loss (RPO)
- For a managed database with off-node backups, the write-ahead log (Postgres) or binlog (MySQL) is shipped continuously, every 60 seconds at most. A planned cutover takes a fresh backup and flushes the log first: no loss expected. An unplanned one — the cluster is gone — loses what was committed after the last shipped segment: seconds to a minute when shipping is healthy, back to the last base backup (up to 24 hours) when it is not. The console shows the actual window before you confirm, and never fails a stateful service over on its own.
- Downtime (RTO)
- Minutes, not seconds: the database is restored, not switched. We rehearse the whole path — two services, MySQL and Postgres, the cluster deleted outright — and publish the measured figures with their caveats; restore time grows with data size, so ask us for a drill on your database rather than trusting a number measured on ours. status.shebaka.com is where you watch an incident happen.
- What you can do now
- Keep the stateful surface small: attach the database to the API that needs it and keep everything else stateless with DR on. Ask us (salaam@shebaka.com) for a restore drill on your database — we would rather rehearse it with you than meet it fresh during an incident.
Where stateful is going
The managed Postgres under your database is CloudNativePG and the managed MySQL is Percona XtraDB Cluster — both already know how to do more than we currently expose. These are queued, in roughly this order — each lands when a customer's need pulls it, and this page changes when they do.
Queued Highly-available databases
Postgres replicas with automatic in-cluster failover — a node loss becomes a seconds-long promotion instead of a restore. The instance count becomes a knob on the database, priced per replica at published rates.
Queued Import an existing database
Bring a production database in by pointing us at a dump (or a connection
string to stream from): the migration path onto Shebaka without hand-carried
pg_restore.
Queued Cross-region database standby
A streaming replica in a second region — the stateful answer to the warm standby, tightening RPO from hours toward seconds.
Queued Postgres upgrades from the console
Minor versions as rolling updates; major versions as a staged upgrade you trigger — no ticket, no SSH.
Queued App replicas
Run more than one copy of a service behind its URL — capacity and node-level resilience for the app itself.