Sometimes I pirate something that is so terrible that I still feel I’m owed compensation for the time I wasted watching it and the bandwidth I used to download it.

undefined> It included a helpful family tree leaflet.

You’d think in some instances the family tree would look more like a wreath though.

Possible yes. Cost effective / valid business case probably not. Every extra 9 is diminishing returns: it’ll cost you exponentially more than the previous 9 and money saved from potential downtime is reduced. Like you said 32 seconds of downtime, how much money is that for the business?

You’re pretty much looking at multiple geographically diverse T4 datacenters with N+2 or even N+3 redundancy all the way up and down the stack, while also implementing diversity wherever possible so no single vendor of anything can cause you to not be operational.

Even with all that though, you’ll eventually get wrecked by DNS somewhere somehow, because it’s always DNS.

I run linux for everything, the nice thing is everything is a file so I use rsync to backup all my configs for physical servers. I can do a clean install, run my setup script, then rsync over the config files, reboot and everyone’s happy.

For the actual data I also rsync from my main server to others. Each server has a schedule for when they get rsynced to so I have a history of about 3 weeks.

For virtual servers I just use the proxmox built in backup system which works great.

Very important files get encrypted and sent to the cloud as well, but out of dozens of TB this only accounts for a few gigs.

I’ve also never thrown out a disk or USB stick in my life and use them for archiving, even if the drive is half dead as long as it’ll accept data I shove a copy of something on it, label and document it. There’s so many copies of everything that it can all be rebuild if needed even if half these drives end up not working. I keep most of these off-site. At some point I’ll have to physically destroy the oldest ones like the few 13 GB IDE disks that just make no sense to bother with.