Online Content Is Disappearing

Poorly thought-out Facebook posts are forever; coverage of city council malfeasance from two years ago, not so much.

This isn’t helped by most websites reinventing themselves every couple of years so the old links 404 even though the content still exists.

If anyone is considering how to avoid this on their own site: https://indieweb.org/URL_design

I think much of Geocities remained accessible until 2013/2014 before going completely (apart from Japan 2019 or so).

I wonder how much of this stuff may still be around on harddrives somewhere. Random blogs probably not because they were using shared hosting that would overwrite and reuse the space when the blog went down, and typically destroy the drives when the servers got decomissioned. But maybe large platforms like Geocities might still be archived somewhere.
- Internet archive actively tried to gather as much as possible before it was decommissioned:
  
  https://archive.org/web/geocities.php
- I wonder how much of this stuff may still be around on harddrives somewhere.
  
  Probably quite a lot!
  
  Just as an example, I'm a part of an art and writing focused community that's been around off-and-on since the late 90s. Typically each member has/had their own website. So a few years ago when we went from an "off" phase back to "on" again, a major project became reconstructing the stuff that used to be on Geocities, the various smaller platforms of the 90s and 00s, and ISP-provided webhosting. And obviuously it's hard to judge how much stuff we don't remember and therefore don't know we're missing, but well over half of what we have reconstructed has come from "I found my external hard drive from 2006 and it had X, Y and Z on it!" I personally had ~3000 files sitting on my NAS, which I had moved off my own hard drive at some point, but had been unwilling to delete, so I just dumped it into long-term storage. Four years into the reconstruction project, we still occasionally find files we thought were lost forever, usually when someone's found an old hard drive in a box in their attic/basement. The found content often was created by someone else, but downloaded and archived by the hard drive owner.
  
  Although this is representative of just one community, given how apparently common it was for people to download offline copies of websites they liked, it could well be that large swathes of the old internet are sitting on people's hard drives, waiting to be rediscovered.
- Probably a good bit. I have a backup from my personal website from 2006 to about 2013. Along with a lot of media going back to my teen years. I'm really lucky I've been a good digital steward of my own data and haven't lost almost any of my personal digital history.

If the information was important wouldn't it already be passed around and expanded upon? The Internet is probably 99% junk, at least the posts I've made. Only the good stuff like goatse survives.

Problem is, people rarely realize the importance until they're lost. Plenty of posts from 90s and 2000s containing valuable insights are probably lost forever. Remember that not everything online is in English, either.
- Finding sources about Bush and Cheney fuckery from 2000-2008 is getting increasingly difficult. Their crimes are getting memory-holed.
  
  EDIT: Specifically, does anyone else remember the specific act that Bush wanted to hit Quakers with terrorism charges over? I remember it being a bunch of Quakers in kayaks doing a blockade of a naval ship, preventing it from leaving port to go to Iraq. I can't find a fucking word on it anymore, and I can barely even find sources on Bush wanting to hit Quakers with terrorism charges other than some broken links at the ACLU. Quakers, as a reminder, are the only religious group in the USA that are default conscientious objectors because violence is 100% antithetical to their religion. These are the kind of people they wanted to use "terrorism" charges against.
From a historical or intellectual archaeological perspective, no one in 2000 BC Babylon thought their pottery would be of historical significance, but 4000 years later, it is. These websites, particularly ones independently created and maintained by hobbyists, are snapshots of the ideas of the time and people that created them. These websites may not have been intensely popular, but they were in many ways a foundational part of the inchoate tapestry of the internet that would eventually become the "modern web."
- On the flip side, nobody can be expected to keep their website up for 4000 years. Hosting costs money and time, and at some point, the thing you're hosting will fall out of relevance enough to no longer be worth the cost.
  
  This is why archiving is important. Hopefully most of the content that was lost was archived at some point. Getting a good chunk of that content onto long term storage would do future generations a favor (even if it's just a bunch of tape storage locked away in a warehouse or something).

This is even more concerning (or funny, depending on how dark your humour is) when you realise that it will be replaced by AI-generated webpages. Humanity's presence on the internet is disappearing before our eyes.

Dead Internet Theory is one of the few "conspiracy" theories I sort of buy, in the sense that it's probably not descriptive of the nature of the current internet, but rather predictive of what it's becoming: https://en.wikipedia.org/wiki/Dead_Internet_theory

And also with less of the whole "they're doing this to manipulate people into believing...things" and more "people want quick and superficial information so that's all that's being produced and since it's easier for machines to produce it than humans, humans will automatically get outcompeted and eventually that's all the internet will be." The internet is becoming a dead mall, filled with the corpses of long abandoned Hot Topics.
- I think the Wikipedia article needs to be updated to be honest. Continuing to describe it as a "conspiracy theory" is quite misleading given the phenomenon is already underway and only picking up pace.

And now with Google regurgitating a summary of the content they've crawled there will be no incentive to publish because no one will click through to get ad payments.

We need to revive the days people write blog posts to help others instead of pushing ads to make money. The content was far better.
- I started a home server on an RPI running some services with a dinky HTML page on it, I need to start actually posting to it. I'm already not on fb or Twitter or anything, if you want to know what I'm doing go to my homepage loser! 🤣

54% of Wikipedia pages contain at least one link in their “References” section that points to a page that no longer exists.

My impression was/is that over the last years/decade Wikipedia made efforts to/switch to not linking directly but extending direct links with (dated) Web Archive links or using Web Archive links directly (dated as "sourced from this in this state; which protects against upstream edits too).

Have not contributed enough to Wikipedia! I'd like to implement this on links I come across.

How does one get an Archive link for a specific date and website? I tried snooping around but am not familiar
- Wikipedia has guidance for it as Citing sources. Regarding web links specifically section Handling links and Preventing and repairing dead links.
  
  The Web Archive "Wayback Machine" is available at web.archive.org. It has a "Save Page Now" action too.
  
  https://web.archive.org/web/*/beehaw.org gives you a history of archived versions of that URL.
  
  The Web Archive "Wayback Machine" is a project from archive.org, which does much more in archiving and accessibility efforts. An alternative service for websites is https://archive.ph/.

Yeah. Might need to start backing up stuff myself. I noticed a lot of old YouTube videos go missing as well.

I think it's much more impressive that stuff that was added in 2018 and 2019 has a much higher probability of being deleted today than if it was added 2017...

Wonder if that has anything to do with covid and maybe new businesses models opened 2 years before failing and therefore websites of this companies disappeared.

Also I think it would be nice to see a graph of new websites being opened other the same time span.

I mean, that kind of makes sense. A lot of small websites are probably for temporary projects, or may even be experiments. When the project ends, it usually makes financial sense to quit paying for hosting and domains.

Whole lotta small projects during COVID.

This isn't inherently bad.

Some web pages are extraneous, fedundant, or only relevant for a limited period of time. A sign up page for a concert doesn't need to exist permanently. Consolidating a large website down to fewer pages that are accessible for everyone is a good thing.

Archiving services that retain web pages that deserve saving are how we should retain that history of the web, but the actual creators don't necessarily need to indefinitely maintain a web page that becomes obsolete.

Yes, a lot is lost that could have just continued to exist and archiving is good, but getting rid of clutter is not a bad thing.

But how do you know if it's the clutter that is being removed? One of the indicator listed in the article is broken links from wikipedia. These links are very likely to point to informational resources or news articles, but are also being lost at a high rate.
Yeah but Amish Rakefight GFY (a dedicated page you sent someone a link to which then toñd you to Go Fuck Yourself -along with a counter to twll you how many people were told to fuck off) has been gone for a few years now....and damn i miss it.
- For sure, things that were not intended to be temporary or were replaced with a better version are sad to lose.
  
  At least we still have https://penisland.net/
Consolidating a large website down to fewer pages that are accessible for everyone is a good thing.

For consolidation, the clean thing is to introduce redirection to the new location.

@funn Online content is, sadly, more vulnerable than we think. All it takes is one server to go down, and the entire website/thing goes bust.

Dead Internet.

This is like the disinvention of the printing press, at least from an archeological perspective.

Well, websites can still be made, so not quite the same, but I get what you mean:-).

And similarly, a ton of written material was lost e.g. when a library was burned - often unique or rare material subsequently lost in other ways, so very much the same process.

The memory hole is real.