@ensanguin
Reddit Tries to Block Bots to Stop Unlicensed AI Data Scraping
Reddit is updating its Robots Exclusion Protocol, or robots.txt file, to try to block bots and web crawlers from swiping data and content from its site.
Reddit says "good faith actors" like the Internet Archive will continue to have access to its platform, however, and adds that most Reddit users won't be affected by or notice the change. Reddit will also continue its practice of rate-limiting, which may help prevent third-party scraping.
This isn't an ironclad solution; as Google notes, there are loopholes to evade robots.txt rules.
0 reply
0 recast
0 reaction