Technology @lemmy.world catculation @lemmy.zip 3 mo. ago

Reddit changes have blocked all search engines except Google amid AI 'misuse' [U]

9to5google.com Reddit changes have blocked all search engines except Google amid AI 'misuse' [U]

Following a deal with Google and "misuse" of scraping its site, Reddit is now blocking all search engines from using its content.

38 comments

Lemmy welcomes you, new ex-redditors.
- Thank you, kind stranger!
  
  tiptiptiptip
- So that means Lemmy is more accessible to search engines, right?
  
  And AI scrapers and bots.
  
  I wonder if all the political shilling is in hopes of future AI learning from it and being biased in the intended way.
- Mbin*
  
  I'm confused by context. Why is everyone downvoting him for suggesting the smaller platform? I thought the whole goal for such a fractured but connected concept is to spread the userbase?
  
  Mbin > Lemmy, and it isn't particularly close.
I'm sure they've convinced the board and the shareholders that this is some kind of big win. But I don't think it's going to be impressive for very long.

There's only so much value an AI can learn from reddit bullshit like "1. break off all contact 2. hit the gym 3. profit" and "the narwhal bacons at midnight" and endless boring pun threads.
- Short term profit is all they care about until this platform crashes down completely
- It sounds a lot like this quote from Andrej Karpathy :
  
  Turns out that LLMs learn a lot better and faster from educational content as well. This is partly because the average Common Crawl article (internet pages) is not of very high value and distracts the training, packing in too much irrelevant information. The average webpage on the internet is so random and terrible it's not even clear how prior LLMs learn anything at all.
  
  So it will end in a downward spiral because it starts learning from AI articles, from which articles are being written, from which the AI learns, from which articles are being written ...
- Well it learned to put glue on pizza, eat rocks, and smoke while pregnant.
  
  You forgot jumping off the Golden Gate bridge
How to kill even more traffic to Reddit
Something something net neutrality?
Just use ddg bangs if you use Duckduckgo and you can search reddit directly.

!reddit search term

It still picks up latest posts related to reddit, it just searches reddit directly instead of searching Bing's results. It's that simple.
“Hey, so it’s me, the guys who left all those comments. Yeah, so we decided that since we wrote them, and the American system says that means we hold the copyright, we don’t really want you selling them without (a) securing our permission first, and (b) giving us a cut of the action. Were thinking maybe like a 30% royalty. It’s not like exorbitant; it probably won’t work out to much more than a few cents per user. But it’s more about the principle, you know?”

“Anyway, what do you think?”

WHAT DO I THINK

I THINK IT’S ALL MINE

DO YOU HEAR ME

MINE

NOW PAY ME FOR THE USE OF MY API YOU FILTHY PEASANT

PAY ME NOW

IT’S ALL MINE, PAY ME

600K A YEAR IS NOT ENOUGH

PAY ME MORE PAY ME PAY ME PAY ME
- no worries. Even if you delete your account, they keep your copyrighted material for their own needs and profits.
- I hate to break to you, but when you accepted the TOS you gave away everything including your soul. Check out Tosdr, look for Reddit and click on “you wave your moral rights”
And I switched engine, so that it's not polluted with Reddit garbage anymore, especially since there's so much AI spam there now.
Spez adores Musk and wants to follow his footsteps
Antimonopoly enforcement when?

_never, _of _course
- It's happening with Amazon now.
  
  Wheels are in motion on anti-monopoly, but it's a major societal shift and that takes time. Time and not electing billionaires to public office. A democracy isn't going to build up momentum to do anything about the unchecked power of billionaires if around half the population is voting for billionaires.
Why do I still see Reddit results on DDG? Is that just old stuff and new stuff won't be indexed?
- 404 notes that Bing, DuckDuckGo, Mojeek, and Qwant are all affected, with results either not showing anything recent, or not showing the full site result. Kagi, a paid search engine, is apparently still showing data, but only because it buys some of its search index from Google, which continues to have access to Reddit data through the aforementioned deal.
- Anything after 2nd of July is not accessible, entries before that date are fine.
Asking for a friend…

What would it take to create a domain that just acts as a proxy to Reddit but serves up its own robots.txt that allows all bots?
- Probably a LOT of proxy IPs to act as different "Users" so you can overcome the rate limit that I expect they would be using to enforce such a deal

You've viewed 38 comments.