150TB, 1 Million Videos, YouTube Collection
150TB, 1 Million Videos, YouTube Collection
https://i.imgur.com/FbVi4YK.png
With everything going on over at YouTube, what is everyone using as their backup solution?
I'm using TubeArchivist to maintain this VERY large collection.
Can I just use this as a downloader and then move the collection to Jellyfin? Or is there a way to link this to Jellyfin?
Technically impressive! That’s a HUGE collection!
I am curious, though, as to your reason for doing this?.. Storing and maintaining 150 TB of small video files is going to be costly and time consuming…. While I do understand that it may be a good idea to archive some specific content that you may fear will otherwise be lost over time, in this case the volume is huge that you can never even watch all of it — let alone know if the content is something worth keeping?
No offense, everyone is free to spend their own time and money on whatever. But I do get curious what the plan is behind all this?
I've been pretty selective, mostly YouTube channels that do 'Tech Documentary Stuff' cause I enjoy re-watching that. LGR would be a prime example.
However, I integrate all of these in my Kodi infrastructure, so only large channels have metadata online in the TVDB or the like.
For those without, I'd love some tips on something to generate Kodi compatible series metadata directly from YouTube.
Quasi-related - what other tech channels are you following? I like LGR quite a bit. Just found The Serial Port - loving that one, building a dial-up ISP.
Techmoan is great. As is Technology Connections.
With TubeArchivist, all the metadata is stored in an ES database and can be queried via API. There's already a Plex and Jellyfin plugin folks are using.
Can you link me on the Jellyfin fork??
I'm using TubeArchivist as well. My collection is slightly larger than the OP's at this point. I collect everything from personally-enjoyable and re-enjoyable content, random stuff at risk during policy changes, to stuff from my early internet years that seems to slowly rot away on the internet. I also like the new themes
Are you willing to share your collection?
If you have a specific request that i have or want to send me 164TB of drives, sure
Before you ask - No, I don't have a full export of what I have in inventory yet. I'm not too savvy with ElasticSearch (yet) and as far as I'm aware, no one else has built a tool for export yet
What is going on at YT?
Removing inactive accounts is one thing, but they’re also cracking down on ad blocking. Could be a first sign that they’re going to be working on stopping YT downloads
I believe Google is removing some inactive accounts by December of this year.
No offense, but I don’t get it.
One could safeguard thousands of movies, song, art of any kind, that might disappear because some of it becomes lost media. Instead some prefer to backup content made for an ad platform.
I genuinely want to know what these 1m are about
I think independent media has the biggest potential of becoming “lost media”. With potential no other copies of this media accessible online, I find it increasingly important to have this backed up.
You’ll always be able to find a copy of Forrest Gump online, but go through your liked videos on YT and see how many are lost forever.
This is a big part of it IMO. Even 'streaming exclusive' stuff that see's no physical release gets pirated enmass so many copies exist. Few people copy YouTube content however, since it's already online, free, and easily accessible. ...Until the day it's not accessible of course.
One could safeguard thousands of movies, song, art of any kind, that might disappear because some of it becomes lost media. Instead some prefer to backup content made for an ad platform.
Let's be clear here, movies, songs, and most art are also vehicles to serve you ads or generate admission/purchase revenue off the consumer. Like you criticize YouTube for being an 'Ad Platform' but elevate 'Songs' as if you've never heard of a radio station?
Funny enough, most films, music and other content are much better protected from being lost. They are mass produced, mass released, and have many many many pirated copies not to mention retail physical copies. YouTube on the other hand? People take for granted that it's there, online, easy to access from anything, and much of it doesn't easily have redundant copies on the internet so it vanishes in a flash when removed from YouTube.
Because some of it is really good?
I've been using a python script I wrote years ago and a config file with channel links / folder directories. it looks like I need to check out TubeArchivist after reading this thread.
My immediate reaction is “why?!?”.
Followed almost instantly by “well, someone should probably be doing it”. Then “I’m really glad it isn’t me”. So I guess thanks for your service.
Whats going on at Youtube?