Meta removes language that may have allowed it to scrape news sites
Meta removes language that may have allowed it to scrape news sites
Meta updated its commercial terms, removing wording interpreted as allowing scraping of content including news sites for AI training, which Cornell digital and information law professor James Grimmelmann and others said could override robots.txt blocking directives. Meta spokesperson Daniel Roberts responded that the scraping clause applies only to ads and commercial products, the company's use of web crawlers has not changed, and content owners can still express preferences via robots.txt.
Language that experts say allowed the scraping of news sites to train its AI has been removed from Meta’s terms.
Language that experts say allowed the scraping of news sites to train its AI has been removed from Meta’s terms.
by Terrence O'Brien
Oct 17, 2026, 3:04 PM UTC
Illustration by Nick Barclay / The Verge
Terrence O'Brien
is the Verge’s weekend editor. He’s covered the tech industry for over 18 years and knows a thing or two about synths.
After backlash, Meta is updating its terms again to remove language that suggested it could scrape content from sites, including news sites, to train its AI. Specifically, the following language no longer appears on the commercial terms page:
… Meta may use any content from such domains, web pages, or URLs to provide, develop, improve, research, train, and maintain the integrity of, the Meta Company Products, services, and features provided under these Commercial Terms and any other applicable supplemental terms.
Many, including Neiman Lab and James Grimmelmann, a professor of digital and information law at Cornell, interpreted it as an attempt to give Meta the right to use content scraped from sites to train its AI products for free and supersede blocking instructions in a site’s robots.txt file.
Andy Stone, Meta’s vice president of communications, and spokesperson Daniel Roberts pushed back, saying that “There is no change to how Meta uses web crawlers; any suggestion to the contrary is false.” Today’s update to its terms and a new statement go further, removing the controversial language entirely. Roberts said in a follow-up statement to The Verge this morning:
“We’ve heard you and want to clarify: the crawling terms we consolidated only applied to our ads and business products, and we’ve clarified that in the terms themselves. It does not change how Meta uses web crawlers. As always, content owners can use robots.txt to signal their preferences. This is not about scraping news sites, and these updates don’t give us new rights that supersede those preferences.”
Follow topics and authors from this story to see more like this in your personalized homepage feed and to receive email updates.
- Terrence O'Brien
Source: The Verge · AI · theverge.com