How do you manage mega torrents?
How do you manage mega torrents?
When you have some torrent where there is a huge collection of files only some of which you want right now, but maybe you will come back to it later to get something else.
Example: This is a listing for torrents of audiobooks from The Eye. (Alphabetically by author, one torrent per letter.)
So I don't want to download every audiobook ever. I selectively choose which to download. Then the torrent is "completed" when those are done. But I want to keep them around because maybe later I want something else. I just leave them in the queue?
In the torrent apps I've used, they seem to get confused by these. If you move the downloaded file to a proper location in your filesystem, then it is having a "missing files" error, unable to seed, and the torrent is in error state. But if you leave the obtained files, it's still in the "not yet downloaded" directory forever.
Wondering if there is some smart way of managing this, or what?
I just select the files I want from the bigger torrents, and then proceed to not touch it ever again, unless I want to add more stuff to the downloaded files.
I also don't move things around - I'm on Linux so all the torrents live in one place with symlinks pointing to where I need/want the data to be as I figured out yeeeears ago that trying to manage a couple thousand active torrents while having the data spread everywhere is a quick trip to migrane town.
Am also on linux. I have a bigger question about how to manage the downloaded files overall but I decided to start with a more specific post and see how that went. ;)
Previously I would move everything after done and a bit of seeding. Lately I've just been letting it accrue where it lands and seed indefinately. However that's not really viable as I move past VLC as media player. Media management software (kodi, jellyfin etc) has file structure requirements, it wants to write nfo files and so on. Also I don't have infinite HDD space so some stuff I just delete after watching.
Is there some kind of tool you use to manage the symlinks? Or you create them all manually in the cli or gui/cli/tui file manager? What's the workflow?
Another benefit of leaving everything in the torrent client is remembering what I already have without doing a file system search all the time. I do really like that aspect.
If you're using the *arr apps (Sonarr/Radarr/etc) for managing the sourcing and downloading of your torrents, they natively support using hard-links for "moving" the torrent files to their required location in the media server directory structure. It's a hardlink instead of a symlink as well which also means the copies don't rely on each other. They can each be moved/renamed/deleted without breaking the other file. Trash guides is a really helpful guide for setting up the *arr apps properly which includes a section on hard-links. https://trash-guides.info/
Radarr and Sonarr both have features to sym/hardlink files to new places after the download client tells them it's finished.
Filebot also gets mentioned a lot for this task, though I haven't used it.
Looks like others have provided MOST of the answers.
Radarr/sonarr do the heavy lifting making symlinks where symlinks are required, but there's still the occasional bit of manual downloading.
I also have a script that'll check for broken symlinks like once a week and notify me of them and I'll go through and clean them up occasionally, but that's not super common and only happens if I'm manually removing content I made manual symlinks for, since I'll just let radarr/sonarr deal with it otherwise.
(The full stack is jellyseerr -> radarr/sonarr -> qbittorrent/sabnzb -> links for jellyfin)
I usually just let the files be where they are and
cp -lto a directory where I can manage them however I want. This creates hardlinks, so you don't use up any extra space.Huh til the
cpcommand has a hard link option. I always uselnand then have to look up what the arguments are every time.hmmm I just tried this out and using
cp -lron a directory. I appears to have recreated the whole directory structure, hardlinking the files; because directories can't be hardlinked. Is that correct? If so wow.Yes, it just does what
cpdoes. Only difference is that it makes hardlinks instead of copies. Unless you're using a COW filesystem, this might be better for most situations.
I just let my torrent client become an unholy mess and do my organization with my file system. I have a 'new stuff' folder everything gets dumped to by default and/or I redirect the location to wherever it'll eventually be organized
I'm pretty sure most torrent applications have a "set location" button on the torrents, which automatically moves the torrent to the location you choose.
I know that qbittorrent also allows you to rename files and folders within the torrent.
The only thing that I have not seen possible is to change the directory structure within a torrent - i.e. move files one directory level up to "flatten" it. If a file is within three nested folders, then it is always like that, but you are able to change both the file and folder names, just have to do it from within the torrent client so that it can pick that change up.
But what you can always do is use filesystem links instead of moving the actual files around, keeping the actual torrent files in some invisible data directory and only "operating" on the links. Works more or less well depending on your OS :D
You can change directories with deluge. When renaming a file, just enter a / between each directory. The ui will fix the hierarchy once you accept the change.
I usually keep a copy of the torrent in a folder just for those kind of torrents. then I make a copy in my qbittorent autoadd folder and tell qbit to not download anything except just the stuff i want before i start the torrent.
part of that is cuz i have my client set to auto-allocate space and don't want some 50+GB thing eating all of my hdd space for temp files i might not even be interested in.
Move the stuff out. If one day I need something else in it, i set the file I have to "do not download" and run "force recheck". Haven't had any issues.