Bots get a bad rap, but they’re not all bad. Without web crawlers, for example, search engines couldn’t index new content on the internet.
But what is bad is when verification providers, including HUMAN Security, Integral Ad Science and DoubleVerify, serve ads to these garden-variety bots, calling into question whether they’re letting malicious bot traffic slip through, too.
An Adalytics report released Friday details numerous instances of brands serving ads to known bots that appear on the IAB Tech Lab’s International Spiders and Bots list and TAG’s Data Center IP List. These bots operate out of data centers whose IP addresses are known for bot activity, and they typically declare themselves as bot user agents.
In other words, bots that aren’t even trying to hide that they’re bots.
AdExchanger reached out to the tech companies mentioned in the Adalytics report and will update this story with any comments.
The brands impacted include Disney, Hershey, Unilever, Procter & Gamble, Microsoft, IBM, JPMorgan Chase, Visa and thousands of others, plus state governments and federal agencies like the FBI, the US Army and Navy and healthcare.gov.
The fact that federal agencies are mentioned in the report has drawn attention from Congress. Sen. Mark Warner, D-VA, on Friday published a letter to the DOJ requesting that ad verification providers be investigated for failing to deliver services paid for by the government, and a separate letter to the FTC asking it to investigate verification providers for false advertising.
AdExchanger discussed the Adalytics report with more than a dozen industry sources – including media buyers, bot mitigation experts, publishers and former employees at ad verification firms. All of these sources agreed that the bot traffic examined by Adalytics should have been easily identified by analyzing the user agent and IP address associated with the bid requests.
However, it’s possible that DSPs are not always passing the user agent data to verification platforms, which would make using that signal to identify bots in real time impossible.
Multiple sources said that, based on the report, verification platforms appear to be blocking domains based on historical, post-bid assessments of the site’s bot traffic, rather than doing real-time, pre-bid bot blocking on an impression-by-impression basis.
If that’s the case, the sources added, then the pre-bid bot filtration solutions offered by verification platforms are essentially overpriced domain blocklists that advertisers should not be paying a percentage of their CPMs to use. Whereas buyers might pay a couple of thousand dollars for a domain blocklist, some pay millions of dollars a year in small CPM fees for pre-bid bot filtration, multiple buyers confirmed.
The majority of our sources requested anonymity in order to speak freely.
Widespread waste
Specifically, the 240-page Adalytics report found that IAS and DoubleVerify did not always prevent ads from being served to known bots – even in instances where it appeared advertisers were paying these companies for their pre-bid bot filtration services.
Adalytics also observed multiple major DSPs and SSPs serving ads to these bots, despite claiming to use bot detection tech, including from HUMAN Security, to scan 100% of impressions.
The scale of potential waste reflected in the report is staggering. Adalytics analyzed source code for millions of ad impressions served to bots between 2019 and 2025.
Although it isn’t clear whether affected advertisers were actually billed for these bot impressions, if so, it’s possible they spent millions on ads that weren’t served to humans.
But whether or not platforms eventually flagged these bot impressions and offered the buyers make-goods, three buy-side sources pointed out that buyers pay IAS and DV a portion of their CPMs for pre-bid bot filtrations services, so buyers are being billed by verification vendors regardless. And these services don’t appear to work as advertised, they said.
“I don’t have faith in any brand safety or verification platforms any further,” one brand media executive who read the report told AdExchanger. “As far as I can tell, none of them work, so we’re just buying vaporware to make ourselves feel better as an industry.”
Or, as a person who works in the bot detection industry put it to AdExchanger, these companies are “missing the low-hanging fruit,” which casts doubt on how effectively platforms are filtering out more sophisticated bots.
IAS provided the following statement in a blog post: “IAS takes these claims and the flawed assessment techniques upon which they’re based very seriously. We are also continuously evaluating and innovating our offerings to respond to today’s rapidly changing digital landscape.”
DV likewise offered the following statement in a blog post: “In every example shared with us prior to publication […] DV had correctly identified the bot traffic. When that occurs, the impressions are removed from billable counts reported to DV’s advertiser customers, as per industry standards.”
But a buy-side source said DV is “couching themselves in the defense that assumes all clients use pre- and post-bid verification. Post-bid will give them user agent, albeit I’m not sure how accurate their billing clawback is.” They added, “If pre-bid doesn’t work, then all clients should stop paying for it.”
Unsophisticated invalid traffic
Adalytics examined ads served to three types of non-malicious bots: those associated with HTTP Archive, which crawls sites for data on how web pages are constructed; bots associated with URLScan.io, which catalogs potentially malicious sites; and traffic from an unnamed third bot vendor.
HTTP Archive crawlers typically self-identify as bot user agents when visiting a site and use IP addresses tied to data centers, rather than home addresses, according to multiple sources. URLScan.io bots, in contrast, try to appear as valid human traffic.
Self-declared bot activity falls under the Media Rating Council (MRC) classification for “general” invalid traffic. This means it doesn’t rise to the level of so-called “sophisticated” invalid traffic, which purposely tries to obscure its bot status.
Whenever a web browser accesses a site, it sends a request to the site’s web server that includes the user agent, which tells the browser how to render a page (such as which language to use, whether the page should have a desktop or mobile layout, etc.). The request also includes the user’s IP address.
Similarly, when an SSP sends a bid request to a DSP, it can also include the user agent and IP address for the ad impression. However, including the user agent in the bid request is recommended but not necessarily required under OpenRTB standards.
Verification vendors can get pre-bid or post-bid access to bid requests to filter out invalid traffic before or after an impression is bid on. Log files for ad impressions can also include the user agent and, often, the IP address sent in the bid request as well.
Missed signals
In short, HTTP Archive crawlers are transmitting clear signals throughout the bidstream that they’re bots, not humans.
Sander Kouwenhoven, CTO at Oxford BioChronometrics, a firm that specializes in online fraud prevention and user authentication, said he was “flabbergasted” that these declared bots weren’t being caught.
“It’s IP address and user agent – there’s no easier way to flag a bot,” said Kouwenhoven, a frequent collaborator with Adalytics who contributed to this report.
Several sources told AdExchanger they want to see more accountability throughout the supply chain for avoiding bot traffic, but they’re not holding their breath.
Advertisers are paying their verification partners to absolve them of risk, said Jay Friedman, CEO of ad agency Goodway Group, and agencies can easily hide behind their verification vendors and blame them whenever a report like this comes out.
At the same time, he said, brand procurement teams typically don’t hold marketing accountable for buying wasted impressions.
Verification breakdown
Meanwhile, three of the largest verification vendors – and a who’s-who of the top ad tech platforms – are implicated in the report.
Adalytics was able to provide the most granular data on IAS because of IAS’s Publisher Optimization tool. This tool transmits a client-side signal that makes it easy to see which entities IAS labels as a bot or not.
According to Adalytics, IAS’s publisher pixel tagged known bots as human traffic 16% of the time. IAS identified URLScan.io bots, which attempt to obscure their bot status, as human traffic 77% of the time.
What explains those results? Sampling could be at least partly responsible. An ex-IAS employee told AdExchanger they personally observed IAS running bot detection code on only 50% of impressions, rather than 100%.
Adalytics didn’t offer similar percentage breakdowns for DoubleVerify, but a publisher source who requested anonymity shared their company’s own analysis of DV’s publisher-side bot mitigation tools with AdExchanger.
DV labeled bot activity on this publisher’s sites from URLScan.io as human traffic 21% of the time. The publisher also shared that their company pays DV hundreds of thousands of dollars per year for publisher-side bot mitigation.
In addition, Adalytics noted that source code for bot impressions contained code associated with IAS’s and DV’s respective pre-bid bot-filtering solutions. According to Adalytics, this code indicates that advertisers paid either IAS or DV for audience segments that were supposed to be bot-free – and yet they still served ads to known bots.
Some individual advertisers spend millions of dollars annually for these bot-free segments via small CPM fees, according to three buy-side sources who spoke with AdExchanger.
However, it could be the case that IAS and DV are not receiving the user agent data from the DSP. For example, a buy-side source said The Trade Desk’s (TTD) API does not contain functionality for excluding or including an ad impression in real time based on the user agent.
TTD provided this statement: “Ad verification is an area where we use a combination of internal tools and integrated partner technologies. We will continue to evaluate and work closely with our partners to review performance and maintain our leadership in this area.”
DSP and SSP impact
But verification companies aren’t the only ones with some explaining to do.
Adalytics examined how effectively DSPs prevented ads from being served to bots by calculating the percentage of successful SSP bid requests that resulted in, well, ads being served to bots.
For example, according to Adalytics, 15% of bot-associated bid requests won by The Trade Desk came from Microsoft Advertising, 15% came from Index Exchange, 9% from Sovrn, 8% from Yieldmo and 7% from Sharethrough – all SSPs that have publicly announced partnerships with HUMAN.
Yieldmo was the only of these companies to respond prior to publication of this story, and declined to comment until it was able to review the Adalytics report.
In addition, Adalytics observed that The Trade Desk’s direct-to-publisher supply path OpenPath saw a higher proportion of ads served to bots than any third-party SSP. According to Adalytics, 17% of impressions that were served to the declared bots by buyers using The Trade Desk were purchased via OpenPath.
Meanwhile, Adalytics also observed Google’s DV360 serving ads to known bots, including via YouTube’s TrueView offering. Notably, Adalytics claims to have found hundreds of thousands of instances of Google serving ads for healthcare.gov to bots using data center IP addresses.
Some of these bots even appear to operate out of Google Cloud data centers. For example, in October 2024, Adalytics analyzed a subset of ads that were served to bots tied to Google Cloud data centers. Advertisers bought 90% of these bot impressions via DV360, 5.6% from The Trade Desk and 3.2% from Amazon’s DSP.
Google provided the following statement: “Google has sophisticated systems in place to protect advertisers from invalid traffic, and Adalytics’ report reflects a fundamental misunderstanding of how our IVT defenses work. It’s important to note that just because an ad serves on invalid traffic or to a bot, does not mean that the buyer was charged. It’s quite likely that our systems detected and marked the corresponding traffic as invalid prior to the advertiser receiving an invoice.”
Google added, “There are a number of reasons why we may intentionally allow an ad to serve and leverage post-serve filtering for invalid or bot traffic, particularly when dealing with an undeclared bot. For example, we may do this to avoid prematurely alerting bad actors that we’ve detected their traffic. It’s also possible that our systems need to collect additional signals to further assess traffic, ensure we accurately classify it as invalid and avoid false positives.”
Adalytics also found evidence suggesting curated private marketplaces sold through several SSPs served ad impressions to bots, including Index Exchange, Microsoft Advertising, Yieldmo, JWP Connatix, GumGum, Sharethrough and Kargo.
JWP Connatix offered the following statement: “We partner with Media Rating Council-accredited industry leaders to ensure a secure and fraud-free experience for our partners. As a client of HUMAN, we expect comprehensive IVT protection, as we send 100% of our inventory through their MediaGuard product, which uses pre-bid filtering to prevent ads from being delivered on IVT traffic.”
Sharethrough offered the following statement: “We work closely with HUMAN Security, an MRC-accredited partner, to help filter and block invalid traffic before it reaches our buyers. As an MRC-accredited vendor and the only one approved by The Trade Desk, HUMAN ensures these tasks are carried out in line with industry standards. If the claims in the Adalytics report are accurate, we are committed to collaborating closely with HUMAN and other partners to investigate and address the issue promptly.”
HUMAN appears to be the common denominator among all of these ad tech platforms. For example, both TTD and Google have publicly touted their bot mitigation partnerships with the company. And the Adalytics report cites numerous examples of other companies promoting their work with HUMAN.
But multiple sources told AdExchanger that it’s unclear from the report where there was a breakdown in the chain: whether HUMAN’s tech did not properly identify bots or if the platforms either overruled HUMAN’s signals identifying bots or were not properly configured to react to them.
Still, whether HUMAN is at fault or not, several sources who spoke with AdExchanger predicted that ad tech platforms will likely lay the blame at HUMAN’s feet.
Meanwhile, brands and agencies will likely blame their verification providers, DV and IAS, buy-side sources said.
Not that attributing blame to just one tech vendor makes sense when the entire ecosystem is widely implicated, multiple sources said. Several other ad tech platforms not mentioned in this story were also implicated in the report, albeit to lesser degrees.
And it’s also important to zoom out.
Underlying the debate about who’s responsible for detecting such obvious bot activity is the role that industry groups like the MRC and the Trustworthy Accountability Group (TAG) play in accrediting anti-bot solutions. All of the verification vendors mentioned in the Adalytics report are accredited by the MRC and/or TAG.
TAG CEO Mike Zaneis provided this statement: “TAG follows the requirements for invalid traffic as laid out by the Media Ratings Council, which allows a vendor to conduct pre-bid OR post-bid filtration of data center IP addresses. DV, Human and IAS have all been independently audited by the MRC/EY to ensure they are able to meet this requirement. TAG’s Data Center IP list is made available for companies to assist with IVT detection, and it is intended to complement other threat detection and removal services and tools.”
But buyers still feel unprotected in the programmatic wilds.
“The MRC is supposed to help defend me,” a brand media executive told AdExchanger, but “as far as I can tell, all [the MRC] does is make sure that I only have two options,” they continued, referring to the two main ad verification companies that work with buyers, DV and IAS.
The fact is, accreditation groups need reform, said Goodway Group’s Friedman, including more extensive oversight from third-party auditors: “Who is supposed to police the police?” he asked.
Correction 3/28/25: An earlier version of this story said that both HTTP Archive and URLScan.io crawlers self-declare as bot user agents. While HTTP Archive crawlers do self-declare as bots, URLScan.io crawlers do not. The story has been updated to reflect this.
Update 3/28/25: This story was updated with comments from TAG, Sharethrough, Google, DoubleVerify and IAS.
