%%excerpt%% Reddit has commenced its assault on search engines, blocking those that don’t have a commercial relationship with the company, like Google.
Instead of investing money in stop crawlers why do not make the data they are trying to crawl available to everyone for free so we can have a better world all together?
I work for a different sort of company that hosts some publicly available user generated content. And honestly the crawlers can be a serious engineering cost for us, and supporting them is simply not part of our product offering.
I can see how reddit users might have different expectations. But I just wanted to offer a perspective. (I'm not saying it's the right or best path.)
Reddit responded: "Only google pays us". The content is not yours. You built this of naive user base that just wanted to share now these fuckers are taking it as their entitlement. As early an reddit user - fuck that place, I'm still angry.
Google just enshittifying even harder. Reddit results in Google searches are often old and anemic these days.
I used to want Reddit threads to show up in search results. Now I avoid them because they are so often a waste of time. More reason to use Duck Duck Go.
I wish Lemmy were searchable better. The search function actually works decently well, but it's not on the same level of actual search engines, it doesn't seem to look for related/similar terms and also relevancy doesn't seem right.
I've posted this elsewhere, but it bears repeating:
Just use ddg bangs if you use Duckduckgo and you can search reddit directly.
!reddit search term
or:
!r search term
It still picks up latest posts related to reddit, it just searches reddit directly instead of searching Bing's results. It's that simple.
You can even use a redirect extension like Libredirect in conjunction with this Duckduckgo feature to redirect your search to a privacy respecting frontend like redlib.
I can't imagine this being a great long-term deal for Google. There's minimal good new content being created on Reddit. Searching for useful information mostly brings up old posts, while new posts are heavily spam generated or designed to support AI learning.
I imagine buying access to historic reddit content from creation to ~2020 would be valuable. While paying for ongoing access to new content is going to be far less valuable and turn into AI devolution as we get to where AI is learning from other AI and spiraling into progressively worse outputs.