728 x 90

Patreon stops asking AI bots not to scrape and starts blocking them | TechCrunch

Patreon stops asking AI bots not to scrape and starts blocking them | TechCrunch

Patreon, the membership platform for creators, is cracking down on AI that mines its content for training purposes. On Thursday, the company shared that it is working with internet infrastructure provider Cloudflare to directly block access to AI robots designed to train their AI models on creators’ work without permission. The strengthened measures were necessary

Patreon, the membership platform for creators, is cracking down on AI that mines its content for training purposes. On Thursday, the company shared that it is working with internet infrastructure provider Cloudflare to directly block access to AI robots designed to train their AI models on creators’ work without permission.

The strengthened measures were necessary because AI scraping has become more sophisticated since measures to deter AI trackers were first implemented in 2023, the company says. Additionally, Patreon’s paywall has long locked much of creators’ content out of the reach of trackers. But more recently, the company introduced new discovery tools, like a redesigned Home Feed and its tweet-like Quips, that could expose more content to trackers.

The changes come as more editors and online content creators realize how AI is absorbing their work in order to make their AI models smarter. To combat this, Cloudflare now offers tools that allow website publishers to restrict AI bots, including a marketplace that allows websites to charge AI bots for scraping, called Pay Per Crawl. Earlier this month, it changed its policies so that “mixed-use” crawlers — those that index and train a website’s content — are blocked by default on any page that hosts ads.

Patreon says it is expanding its current work with Cloudflare to use the company’s AI Crawl Control technology to update its AI policies and enforcement tools. The difference here is that instead of simply asking AI crawlers not to scrape content using robots.txt files (a standard way of providing robots with instructions on how they can use your site), Patreon is now actively blocking AI training robots.

“Consent should not depend on whether a scraper chooses to behave,” a Patreon blog post explains, referencing the stricter measures.

When testing the features, individual AI training trackers’ weekly attempts to access Patreon went from “thousands of attempts to zero,” the post notes. That indicates that the AI ​​scrapers were ignoring Patreon’s robots.txt file and scraping the site anyway, despite their requests.

However, the company said it will allow bots to index pages and organize information that can be used to send users back to Patreon.

“As AI agents become increasingly powerful and popular, creators deserve a meaningful voice in how AI companies use their work,” Patreon head of product Drew Rowny said in the announcement. “Across most of the internet, creators have to accept AI training in their work just to reach and grow an audience. Patreon has a different vision: creators should be able to grow their audience and control how their work is used.”

When you buy through links in our articles, we may earn a small commission. This does not affect our editorial independence.

Keep following us for the latest insights.

Posts Carousel

Latest Posts

Top Authors

Most Commented

Featured Videos