Some political issues don’t become mainstream talking points, but that doesn’t make them any less urgent.
They’re more like “hygiene [and] vegetables,” quipped Danielle Coffey, president and CEO of the News/Media Alliance, as in, a little unglamorous, but crucial for a healthy body.
Or, in this case, a healthy content marketplace.
In late July, three US representatives – Valerie Foushee (D-NC), Laurel Lee (R-FL) and Gus Bilirakis (R-FL) – introduced the Stealth Bot Prohibition Act in the House. The bill is a proposed piece of bipartisan legislation authored by the News/Media Alliance, a trade association that represents over 2,000 media brands. If passed, it would require AI stealth crawlers to disclose their identity and purpose to the host of a website. (A version of the act passed in New York state earlier this year.)
These so-called “bad bots” are “creating a reseller market,” Coffey said, by crawling and scraping publisher sites and then serving or selling the content elsewhere without permission.
The bill would allow enforcement agencies like the Federal Trade Commission and state attorneys general to bring cases involving undisclosed crawlers to court. Each violation would incur a $53,000 fine. The hope, Coffey said, is that steep penalties combined with active monitoring by website owners and enforcers will help deter malicious scraping.
Any bot that’s trying to hide its identity is probably not above board, said Amelia Binder, SVP of global government affairs at media conglomerate Axel Springer, a member of the News/Media Alliance.
Most of the stealth bots on Axel Springer’s sites have been scraping content against its terms of service and using it for their own financial gain, Binder said, either by passing it off as their own or reselling it to third parties. This has led to decreased revenue for the publishers and higher odds of inaccuracies as the content gets regurgitated out of context.
The new bill is “an first important step” toward transparency and licensing deals, said Binder. “We need this first line of defense.”
Identify yourself!
There is currently no legislation in place mandating that bots disclose their identity and purpose, even though that sounds like the bare minimum.
One reason is because AI has evolved so rapidly that it’s “outpaced existing regulations,” said Lark-Marie Anton, chief communications and brand officer at USA TODAY Co., also a member of the alliance.
To understand why that gap is such a problem for publishers, it helps to understand how exactly the bots work.
Historically, web crawling has essentially run on an honor system via public robots.txt files. These files are hosted by publishers and indicate to web crawlers which parts of a site they can and can’t access. The problem is, there’s no real enforcement; the files are simply recommended guidance that “good” bots choose to follow.
Recently, bots have begun to ignore and even manipulate these files by masquerading as human viewers so that sites categorize them as human traffic rather than bot traffic. They then ingest a site’s content and either pass it off as their own, resell it to third parties or provide it as training data for AI models and the APIs built on top of them.
Several dozen companies – including Perplexity – have been profiting off of sales from illicitly-scraped data, per Digiday.
The practice also opens the door to a whole host of problems. For example, even though the underlying content is public, there are national security implications. Many of the bots originate from Russia and China, said Coffey, and when they harvest news and information at scale, they can mine it for patterns, train their own AI systems and undercut the business models that support independent journalism in the first place.
Intellectual property “needs to be respected outside of our borders,” said Conan Gallaty, CEO and chairman of another News/Media Alliance member, the Tampa Bay Times. When these bots scrape content, he added, they do “whatever they wish” with it, further threatening the content marketplace.
Every security threat has a silver lining
However, stealth both have also created some unexpected side effects that could work in favor of publishers.
For one, the flood of AI-generated content has increased the demand for “trusted, high-quality journalism,” said USA TODAY’s Anton. That dynamic could ultimately lead to the development of licensing and attribution frameworks that actually pay publishers for their work.
Meanwhile, the concerns over national security might be the push the bill needs to pass. Those worries, together with the broader marketplace imbalance that content creators are facing, has led to “a lot of interest” in the bill from both sides of the political aisle, said Gallaty.
The current administration has generally been opposed to stricter AI regulations in the name of staying competitive in the global AI arms race, though it is currently finalizing an AI oversight framework due to increasing concerns about electricity costs and the potential for cyberattacks. (Still, President Trump has been steadfast that the US not “come in second to China” in the AI arms race.)
Even with that momentum behind it, there’s no guarantee the bill will pass the House, let alone the Senate. (After all, it’s not easy to become a law!)
But the hope, Coffey said, is that the bill’s bipartisan appeal, along with “the fact that it’s so reasonable and on its face unobjectionable, will give it the best chance that we could get.”
Tit for tat
And publishers are in dire need of that “best chance” as they struggle with the downstream effects of AI bot traffic.
Some publishers, for example, say that visitors have had trouble accessing articles because unauthorized bot activity is weighing down their networks. “It taxes our ability to serve customers,” said Gallaty.
Axel Springer, meanwhile, which owns numerous publications including Politico and Business Insider, has also been hit hard. Twenty-five percent of Politico’s hosting costs now go toward bot management, Binder said, which is something the company “didn’t budget for.”
The sheer volume [of stealth bots] is crazy,” she added.
And the damage isn’t limited to publishers; consumers are also likely getting less reliable information.
When unmoderated chatbots steal content, Binder said, “spitting out ‘journalism’ that may or may not be accurate,” there’s an increased likelihood of bias and inaccuracy.
A “healthy media ecosystem” has plurality, she said, and numerous viable options for accessing news and analysis. But this is only feasible, she said, when publishers have “willing partners” that license their content and ensure it’s being used fairly and accurately.
“Quality journalism requires significant investment,” said USA TODAY’s Anton, as well as “expertise and accountability.”
For AI to have a healthy place in the information ecosystem, she said, it has to support and sustain the human work that gives it value in the first place.
