Skip Navigation
Technology @lemmy.zip BrikoX @lemmy.zip

The text file that runs the internet

www.theverge.com The rise and fall of robots.txt

As unscrupulous AI companies crawl for more and more data, the basic social contract of the web is falling apart.

The rise and fall of robots.txt
0
Technology @midwest.social alex [they, il] @jlai.lu
The rise and fall of robots.txt
Technology @beehaw.org alyaza [they/she] @beehaw.org
The rise and fall of robots.txt: As unscrupulous AI companies seek out more and more data, the basic social contract of the web is falling apart.
Technology @lemmy.world Andy Reid @lemmy.world
AI companies are violating a basic social contract of the web and and ignoring robots.txt
0 comments