All of my services run on LXC containers. Some files and configs are backed up to NAS and offsite. The containers are snapshotted in their entirety before I do any work on them. A snapshot takes 5 seconds to make and causes no downtime. If I regret a change or mess it up, I can restore the snapshot in under a minute at the cost of some seconds of downtime.
My only non-container machines are my desktop (doesn't count), my NAS and the Hypervisor. The Hypervisor is very clean and wouldn't be much fuss to reinstall and the NAS is literally just Debian with NFS. All of these have a regular rsync which runs to backup the important files.
The container is reproducible. Container configuration is in version control. That leaves you with the volumes mounted into the container, which you back up like any other disk.
btrfs + snapper takes care of most level-1 situations, and I take a snapshot of every /root change, plus one nightly /home snapshot. but it's pretty demanding on disk space, and doesn't handle drive failure; so I also do
restic + USB drive, which I can cram way more snapshots onto, so I keep a couple of weeks of daily snapshots, one monthly snapshot for a year, and one snapshot per year, going back several years. I currently have snapshots from my past 3 computers on one giant drive. However, these drives can also fail, and won't protect me from burglary or house fire, so I also do
restic + BackBlaze. I just take a nightly snapshot for every computer and VM I manage. My monthly B2 bill is around $10. The VMs don't change much, and I only snapshot data and config directories (only stuff I can't spin up fairly quickly in a container, or via a simple install command), so most of the charge comes from a couple of decades of amateur digital photography, and an archive of all our digital music (because I'll be damned if I'm going to spend weeks re-digitizing all those CDs).
The only "restore entire system b/c of screwing up the OS" is #1. I could - and probably should, make a whole disk snapshot to a backup drive via #2, but I'm waiting until bcachefs is more mature, then I'll migrate to that, for the interesting replication options it allows which would make real-time disk replication to slow USB drives practical; I'd only need to snapshot /efi after kernel upgrades, and if I had that set up and a spare NVME on hand, I could probably be back up and running within a half hour.