Production data is copied every night, and every copy is proven by restoring it into a scratch database and checking the row counts. The copy goes to a private, encrypted cloud folder through a key that can only add files. Control Tower shows when the last good backup happened, and turns red if a day goes by without one.
Every night
each copy is restored and checked
3 days
kept, then the storage deletes them itself
Status: designed and approved, being built. The Tourney backup is specified, and the shared version for a second app follows it. Nothing on this page is running in production yet, and I will change this line when it is.
01 Where it runs
The backup helper is a scheduled job on the production server, outside the app containers, so a slow backup can never stop a site from starting. Control Tower runs on a desktop PC, so it reads the result over SSH instead of being called by the server.
02 What is backed up
Tourney
The MySQL database: bracket entries, picks and user accounts, plus settings and the daily commentary. Team, player and score data can be re-downloaded from ESPN, and it is in the copy anyway.
GetSeen
Its SQLite files, once it runs in production next to Tourney. The helper takes a consistent snapshot while the app is writing.
Runway
Nothing. It has no server-side database; each user's plans stay in their own browser.
Dev and SIT
Nothing. They hold test data that can be rebuilt.
03 What happens each night
Copy. The database is dumped in one consistent snapshot, without locking the site, and compressed. The dump includes the stored procedures the app needs to rebuild itself.
Test. The copy is restored into a throwaway database, never the real one, and the row counts for entries, picks and users are compared with the live database.
Store. The copy is uploaded to a private, encrypted bucket under a timestamped name.
Expire. The bucket's own lifecycle rule deletes anything older than three days. The backup job has no permission to delete.
Report. The result and time are written to a small status file that Control Tower reads.
04 How the copies are protected
A backup of a users table is a second copy of personal data, so the design assumes the upload key will one day leak.
The upload key can add files to one bucket and do nothing else: it cannot read, list or delete. A leaked key does not expose earlier backups.
The bucket is encrypted by default and blocks all public access.
Retention is enforced by the storage, not by code that could be changed or fail.
Keys live in the server's environment file, never in a repository. The secret scanner covers the script and its docs.
05 How I know it worked
Control Tower shows two things next to each app's deploy state: when the last backup finished and when it last passed the restore test. If no new backup appears for more than a day, the badge turns red. A backup that has never been restored is a guess, which is why the restore test runs every night and not once.
06 Decisions
Nightly test
The restore test runs on every backup. An earlier plan tested once at build time; a nightly test catches a silently broken backup within a day and gives the badge a real timestamp.
One storage
Backups live only in Amazon S3, with no second copy at another provider. That leaves one risk open: someone who took over the AWS account could reach the backups as well.
Built once
Each app declares what it stores in a short settings file, and one helper handles the copy, the test and the report. A new app is a new file, not a new script.