Is there a collection of all human knowledge ever created ?
Is there a collection of all human knowledge ever created ?
Recently I was wandering if there is someone or some group preserving , collecting , organizing and publishing all the knowledge of mankind ever created throughout its existence so that if ever mankind faces the 6th mass extinction we don't have to reinvent the wheel and can have a kick start to our new post apocalyptic civilization .
Wikipedia is a great start. You can download its entirety, roughly 100gb. Most of the basic and advanced human knowledge.
Check out kiwix to get it offline
Seconded Wikipedia. The amount of knowledge that can be gleaned in mere minutes from Wikipedia is insane. It contains enough information to do most stuff, aside from blatantly illegal things.
Luckily it isnt that easy to burn down every server as burning carpets in south america or books in other places
You can do all of Project Gutenberg too. It's only about 75gb, surprisingly.
ASCII text is hella lightweight.
Wikipedia is basically useless without the sources, though. It's just a TL;DR summary of things.
You son of a bitch
Such an insightful commentary on the importance of the social contract and the irreplacibility of the individual. The only way forward is to share our personal experiences and strive for understanding. Once we know each other's value, we will never surrender our common bonds, disappoint one another, go behind each other's backs, nor do each other harm.
I feel like you're stretching the definition of "knowledge" and definitely "human knowledge" a bit there.
Maybe so — I think that's kind of the fun of it though 🙂
There isn’t. Yes Wikipedia has a lot of info but think of all the information that is in the hands of governments and corporations that are closely held secrets. Or that’s only in the minds of a few experts on the planet.
Like sure Wikipedia can tell you what a CPU is. But to build one from scratch, from the silica to building the machines and factories, that information is spread across multiple companies and never shared with the public. And only a few experts truly know how to do every step in the process, they have vital knowledge of that process that they feel is common sense and is not written down, which they pass on to the people they mentor. If those few people die at the same time in a catastrophe the knowledge that isn’t written down dies with them.
We already lost a lot of information of old tech from not that long ago because the companies went bankrupt or the people involved all died. Like we don’t even have all the knowledge to rebuild the Saturn V rockets, because the people involved, who hold vital knowledge, are dead and the supporting infrastructure, like the sub contractors (who also had vital knowledge), is gone as well.
Perhaps https://archive.org/ is the closest you could get? With nearly a trillion web pages in its archive, I don't think I've ever come across a database of knowledge that comes close to it's collection. What's interesting is that archive.org preserves not only web pages, but several pieces of binary content such as music, movies, art and even software applications and entire operating systems. Not sure if it would be enough to rebuild our society, but it would be a great starting point for most of our essentials.
Specifically their OpenLibrary division. They had a mission to make as many books as possible digitally available and free for everyone to borrow but unfortunately they keep getting hit with lawsuits and slowly take down more and more of their collection.
It’s “its”, not “it’s”, unless you mean “it is”, in which case it is “it’s “.
your bein pedantic. its okieish. Kaithx for grammly lesson. your the bestest.
yes archive.org is pretty good in finding some very obscure and rare movie from 30s or 40s or some old books or niche software or music but most of the things on it are webpages and 100s of copies of the same webpage in different times (not very useful in a post apocalyptic world tho as most of the things on the archived websites you can't even click because its only a snapshot )
You're using it right now.
That is pretty much exactly the goal of the Wikimedia Foundation which runs Wikipedia and its sister projects.
But by now we figured out what wikis can do well and what not. Wikis are suitable for crowdsourcing objective facts about the world (all it takes is one person to add any given fact), they are not a universal remedy for everything, especially not contentious issues or useful instructional materials.
I have made more than 100000 edits to their projects. I don't participate there anymore. The time when they were a force for good in the world is long past.
That is pretty much exactly the goal of the Wikimedia Foundation
Their goal isn't to collect all human knowledge, only notable human knowledge.
See https://en.wikipedia.org/wiki/Wikipedia:Notability and https://en.wikipedia.org/wiki/Wikipedia:What_Wikipedia_is_not#Wikipedia_is_not_an_indiscriminate_collection_of_information .
Why aren't they a force for good anymore?
Many reasons most of which you'll only understand if you pay some attention to what's going on behind their scenes.
There are reasons why nowadays pretty much everywhere else on the Internet more content is created all the time than on the Wikimedia projects.
The Wikipedias' "neutral point of view" policy used to mean "we try to treat all sides fairly", now it means "we are writing an unconditional propaganda organ for the status quo". The mainstream media that is accepted as "reliable" as Wikipedia sources just isn't that credible anymore.
Also, when I started editing there, the individual projects were mostly left alone by the WMF. Nowadays the WMF issues intransparent sanctions, up to lifetime bans from all projects, left and right.
I wish someone started an organization with the same goals as the WMF with an actually working system where people could actually enjoy participating.
All of Wikipedia is <256 gb.
All of Wikipedia in English <64 gb.
Then archive.org for multimedia, ~10 peta bytes. Yipes.
Archive.org has a ton of multimedia
Well archive.org has much more videos
A Library. Or if digital, Wikipedia and Archive.org.
most libraries lack 90% of collective human knowledge and most libraries today (in asia particularly) are pretty shit in number and quality of books .. Wikipedia yes but still it can be manipulated by rich people or government for their own interests
Not just can be. Wikipedia has been proven to be manipulated by political interests.
I don't personally know of any but in a similar vein there are some stone monuments intended to convey information after an apocalypse like the Georgia Guidestones or the nuclear waste site warning stones. GitHub put a snapshot of all active code repositories from 2020 in arctic permafrost, and there is the arctic seed vault for preserving plant species.
"On July 6, 2022, an explosive device was detonated at the site, destroying the Swahili/Hindi language slab and causing significant damage to the capstone. Nearby residents reportedly heard and felt explosions at around 4:00 a.m" the rocks got destroyed by a mere explosive and they thought it could survive a nuclear war lol
Well, if a nuke actually hits anything built to withstand nuclear war, it will break. There is nothing really that can withstand direct exposure to powerful explosives.
I assume the plan was Georgia since there's nothing worth nukeing there.
Check out this book: https://en.m.wikipedia.org/wiki/The_Knowledge:_How_to_Rebuild_Our_World_from_Scratch. It analyses that precise question in the first chapter. The author argues that even though Wikipedia is probably the closest thing there is, there is a clear lack of practical knowledge that will be essential in the situation that you are describing. Science progress heavily relies on industrial progress, and even if you know how to build something that doesn't mean that you can do it, as there are other things that are required first.
Damn that sounds like interesting book to read. Got to get a copy
Well, technically, Library of Babel though that probably isn't really what you're looking for.
I think the internet as a whole is going to be the closest we'll ever come. Capitalism will make sure it's never even close to complete so it always has something to monetize.
I wouldn't say "complete" can even be sufficiently defined in this case. Every functional definition I can think of has a limiting factor.
Let's try to define knowledge. What kind of information qualifies? We can usually think of important, useful info like physics and medicine. But what about other data, like sports game stats, atmospheric sensor readings, or even something more esoteric, like the location data of every object on earth.
And even if we could have the information of every single thing at any particular time, what about when things change in the next second? And the one afterwards?
Essentially, nothing will ever be "complete". Thanks for listening to my rant on semantics.
That was a lovely rant on semantics. I thoroughly enjoyed reading it!
you can read pretty much (except the lost media like those lost in library burnings , film destruction and wars) read any book written by humans since 2500 bce (example Rig Veda the first ved of Hinduism was written even before 2500 and is today said to be 98% at its original state thanks to Indian gurus and saints who passed it on orally and was made into a book only after 8th century) , watch any movie ever released , hear any music ever made after recording was invented .
ofcourse there is a catch that these medias are not freely and publicly available and most you have to pirate in order to consume it thus we need to have a centralised database of these things safely kept somewhere so that we don't have to reinvent the wheel in case of a catastrophic event .
Besides what other commenters already said, archive.org does a great job.
I'm surprised no one mentioned projects like libgen and scihub. They are much better than Wikipedia imo.
imo zlib is much better but they keep changing their domain ... also sci hub is only for research papers which most people can understand
Though I doubt digital things will survive an apocalypse.
z-lib is still up and the best option in my opinion ... you have to just get the right domain not the fake ones
It's never going to be all knowledge, since a lot of stuff is just lost or never recorded. A ton of stuff (like this thread) are probably low on the priority list for recording as well. But the closest you'd probably get to a full catalog of human knowledge (at last text based) are the huge data sets of nearly all text data on the internet used for training LLMs. I wouldn't be surprised if there are ones soon that include video and pictures as well, since newer AI models are starting to be able to interpret those too.
I believe this is one of those data sets: https://github.com/yaodongC/awesome-instruction-dataset
Edit: here's a big data set used for a lot of gpt3 https://commoncrawl.org/
IIRC the LOC has a copy of every book published in the US... I need to re-verify that, but I believe that was one of the info bits I gathered from my last visit. I don't retain knowledge very well when I glean it whilst gaping at architecture.
Related monument / installation:
https://www.historyingranite.org/
While its not all of Human knowledge it's a very important part of it, our history.
Ultimately if the goal is for it to survive societal collapse to the point where knowledge gets lost then books are a better bet than any digital storage. As such most large libraries hold the knowledge needed to take us up to the industrial revolution at least. Semiconductors is hard to teach in only text, by that point you really need the ability to produce the machines needed etc. But the underlying theory will be in a library. Also a lot gets proprietary and trade secrets when we get within the last 30 years and that knowledge won't ever get written down in a central repository until it's aged out of being protected.
Knowledge is not what make people distroy this planet though, or what helps against it. We know enough already
read the query again
I read it, what I meant is: is knowledge the thing that makes our society nice? Or would it lead to the same shit we have right now, if its not accompanied by something else?