How to keep checks on a 14 TB external?
How to keep checks on a 14 TB external?
Hey folks!
I have a WD easystore 14 TB External HDD connected to my Plex server (running on windows 11).
I am using about 4 TB of it, but not for anything truly important. It’s storing plex media mostly.
I’d like to use it for storing memories. But how do I trust it?
What are good tools for me to keep a check on the drive so that I can hopefully get enough warning when it starts losing sectors?
I have some tool installed based on recommendations online and I started a “surface test” of the disk and it said it’ll take a measly 300 hours. Not ideal.
There are tools for this, but my recommendation is that if the data is important, don't trust it. Follow 3-2-1 and keep some real backups of the important stuff. So use that drive, but also keep another copy elsewhere on different media.
Another copy of the data is the only good option
3 is 2
2 is 1
1 is none3-2-1 is usually a good practice OP. It does significantly increase costs though.
Would love to understand what 3-2-1 means please.
Simply put.
3 copies of anything critical.
2 types of media
This is more difficult for non business as non HDD storage is expensive but one cloud backup can work instead.
1 copy stored off site.
Having three copies at home is not much help if your house burns down etc.
I'm a sysadmin and daily, I use servers with many more disks with much more access, so obviously I'm a bit jaded about these things but still:
You can never ever trust a single disk. If you bought it recently it should be good for a while but one day it's going to fail on you and that data is either gone or you're paying $10k to have a professional pull the data off the platters if that data is worth that much to you.
If you're at that level, there are many offerings from NAS companies that would love your business to sell you a simple 2-disk raid NAS, or you could build your own and learn how that works!
I just raw dog RAID0 everything
i physically cringed
Everything fails except old PATA disks. For some reason they tend to last forever.
Get a second one at least. You can't really trust it otherwise. If the truly important data is relatively small you can even just put it on a cloud service as an extra backup. Or use something like Backblaze B2 and keep it connected to your computer and backed up that way.
Backblaze seems truly interesting. Would they really backup my entire 14 TB and more if/when I’m able to fill it up? And yes, I do have my personal data on two separate cloud services. But can’t trust anyone these days.
Yes. I do the exact same thing as you, a Windows Plex server with a huge external hard drive. And Backblaze backs it all up for $7 per month.
Edit: I'm also backing up all of my other devices on the network to that same external drive, by the way. My primary PC mounts that external as a network drive and uses it for a file history backup. And my android phone automatically dumps all my photos to that external using SyncThing. So Backblaze is literally doing everything for $7 per month, it's a great deal.
Backblaze is also cloud service btw. I use kopia for daily backup to 2nd HDD and also to backblaze b2. 1TB cost 6$/mo so I backup only important files, so no movies etc. Backups are encrypted, and you can also encrypt backups on gdrive for example. That way they cant do shit with your files so trust is less important. Also, test your backups every now and then
WRT 3-2-1 I'm only part way there for my personal stuff, I haven't been willing to invest in anything offsite yet but gained some peace of mind anyway. Offsite backup for the most important stuff is in my future for sure. FWIW here's what I have, and what I did:
In order of size: 1000+ movies, a couple dozen TV series I may want to revisit, something like 10000 songs (mostly in albums), and a slew of scanned family photos and genealogy documents as well as offline ancestry databases.
In order of importance: family photos and genealogy stuff, music, movies, TV shows.
I ended up with a primary NAS, Synology 4 disk with one disk redundancy, and a second older Synology set up the same way that serves as backup. Each with battery powered UPS. The older NAS has degraded twice over about 12 years, and each time I popped another disk in and a couple of days later all was good. They do scheduled data scrubbing and I subscribe to an inexpensive SMS service that texts me when that gets done or if anything else happens.
I don't use any backup or replication software. Anything on there started on either my desktop or my wife's laptop, and we just copy it twice. Except the TV shows and the one-and-done Netflix movies, they just go to the primary and I assess later if they are in need of backup.
Hope that's useful in some way.
If you do not mind cloud solutions, almost every backup software can now copy data to S3 compatible storage. It takes a while but for incremental backups it works fine.
Backups. You want backups of your important data. You want multiple copies, and for one to be offline and offsite if it's critical.
That's about it. Tools won't guarantee that you'll catch a failing disk (or have time to migrate if you do) before it fails.
So, related question for the other people in the comments section here, what's the best method for copying all your data to other drives? Surely it's not just copy, paste, then wait?
Robocopy on Windows. RSync on Linux.
These are quick and dirty but work and can pickup where you left off on failure.
others have answered but another option is SyncThing which has a benefit of working across multiple devices including mobile
Storage Spaces for Windows OSes.
Seems risky, I had really bad experiences with windows software RAID so I'm not sure I would trust such a solution.
It's quite complex but I use git-annex for this purpose.
In addition to the other suggestions I'd recommend Stablebit Scanner. It monitors the drives in your system and will alert you when a problem is detected.