Bots Are Hot, But Publishers And Advertisers Are Cold To Combating The Situation

botnetImpression fraud has been a major industry problem for a while, but it’s only lately that buyers, sellers and vendors have sought to quantify and arrest it.

Even at a roundtable discussion of more than two dozen online ad executives held by Integral Ad Science on Thursday morning, participants seemed resigned to the idea that bots will remain one step ahead.

Despite recent efforts by a number of ad verification companies — including Integral Ad Science, DoubleVerify,, WhiteOps and Solve Media — to expand their respective responses to combat bots that leave phony impressions, their attempts are hindered by a lack of regulatory action and complacency by some marketers and publishers concerned about the appearance of losing scale.

In general, these brand managers are not well-versed in the technical complexity involved with explaining the problem to higher-ups. And as a recent Integral study showed, creating phantom impressions is lucrative and there’s little risk that offenders, many of whom operate in parts of Asia and the former Soviet republics, will be penalized.

“Nothing will change until there’s a public hanging,” said Gartner analyst Andrew Frank, echoing the reaction of roundtable attendees. Many of the two dozen or so individuals said they would at least settle for some sort of legal action against bot generators.

Big dollars, bigger indifference: In Integral’s report, titled “Digital Fraud: What’s The Fuss About?”, the company estimates that 15% of all Internet impressions are fraudulent, driven by bots. This roughly amounts to a $6 billion slice of the total $42 billion online marketplace. Yet, to some companies, it’s just the price of doing business on the Web.

Scott Knoll, Integral’s CEO, related a story of one advertiser’s indifference about curtailing fraud. “Someone who is in-house managing makegoods for fraud told me, ‘If we do this, it’s going to hurt my scale,'” Knoll said. “He doesn’t realize that because of fraud, they’re paying for less scale. But it’s too difficult for him to explain the situation to his bosses.”

Bots in-banner: There have been a number of widely ranging estimates of how big the bot problem is. Even Integral conceded that some of its own percentages (based on the 80 billion impressions a month) of fraud activity are conservative and vary from day to day.

  • In-banner video, typically the province of lower-cost direct-response ads, fields the most fake impressions (40%).
  • After in-banner, the other hardest-hit categories by bots include bid requests (30%), exchanges (25%) and networks (20%).
  • Preroll and publisher direct appear to be the most secure from bot activity, with only 3% and 2% of impressions, respectively, compromised.

Integral is not the only ad verification company that has examined this problem. Earlier this year, figured that while botnet traffic to websites cost advertisers an average cost per mille (CPM) of only $0.69, the accumulated monthly figure was $6.2 billion.

Mobile is the next battlefield in the bot war. Kiril Tsemekhman, Integral’s chief data officer, noted an uptick in fraudulent impressions on the mobile Web (Integral doesn’t have visibility into the mobile app environment), particularly on Google’s Android system. The attacks on Android could also have to do with its growing market share: Research firm IDC found that smartphones running the Google-owned operating system made up 81% of all the devices during Q3 of this year.

Toughening situation: Whether on mobile or desktop, all roundtable attendees agreed that the bot situation is getting worse. According to a Solve Media report, Q4 spending will show an increase in impression fraud, with bots wasting as much as $9.5 billion this year alone.

The reason, said Solve Media CEO Ari Jacoby, is because the ad ecosystem has become more complex and “effectiveness rates” are decreasing dramatically. Click-throughs, for instance, are “now worse than .001%,” and there is little click-to-play (as opposed to autoplay) video inventory available.

“Getting consumers to pay attention has become increasingly harder with the rise of ad-blocking technologies,” Jacoby said. “As such, there are incentives to cheat the system. More importantly, a lack of security in the ad-tech industry is being exploited by some of the smartest criminals around. It is much easier, less risky and profitable for underemployed Ph.D.-level scientists to steal ad budget than it is to sell drugs or weapons.”

Look before buying: Integral’s SVP of product management, David Hahn, discussed his company’s approach to dealing with bots by blocking the serving of such impressions before the bid is made. (It should be noted that rival company DoubleVerify has a somewhat similar approach.)

There appeared to be a consensus that blacklisting whole domains infected with bot activity is not the way to go. As Hahn told the attendees, it throws out good traffic along with bad. Andrew Pancer, Dstillery’s COO, agreed.

“Combating fraud is not about blacklisting sites,” he said. “Bots reside on the computer, so you need to understand what to look at to identify when the ‘consumer’ is actually nonhuman. Many times, bots are running in the background during a normal user session. This is where the fraud-detection vendors are best positioned to help. Anyone bidding in an exchange environment should either develop their own tools to detect this behavior or enlist the help of someone who can.”

Apart from Integral’s prebid blocking, Pancer offered some practical ways for buy-side players to reduce exposure. The easiest is to avoid sites that are generally unfamiliar, yet appear to have more traffic than a comScore 100 site. Another obvious red flag is if a site buys a large percentage of its traffic.

Publishers’ Prerogative: Defeating botnets is primarily in the publishers’ court. As Pancer noted, it’s very tempting to buy very cheap traffic that a seller is easily able to arbitrage. But cheap traffic is cheap for a reason. In the short term, publishers benefit from the sale, but the long term is a different story, said Solve Media’s Jacoby.

“The onus is on publishers to defend against bots, as their livelihoods are at risk,” Jacoby said. “Agencies are being tasked by their clients to purchase human audience and not bot traffic.”

Enjoying this content?

Sign up to be an AdExchanger Member today and get unlimited access to articles like this, plus proprietary data and research, conference discounts, on-demand access to event content, and more!

Join Today!


  1. Marcus Pratt

    While I agree that blocking entire sites is not the solution, this can certainly be part of an approach to combat fraud. For many domains, there is really no good traffic to throw out with the bad – they exist only for the fraudulent traffic, or, at best, violate copyright law in hopes of collecting some organic search traffic in addition to the bots.

    If a site consists of scraped content and has no bylines, about us or contact pages, I’m not too worried about losing out on good traffic

  2. A number of things help from the advertisers’ perspective. Three key ones are:

    1. Pay the media buyer for their time, rather than a % of the media budget they spend. This deals with scale issue. If there is spare budget, great, put it into something else.

    2. Buy media on a CPM rather than taking perceived shelter from risk in CPC and CPA metrics. CPC and CPA buying allows fraudsters to arbitrage fake impressions and multiply their margins massively.

    3. Do proper analytics and attribution on the effect of the media. Bot traffic may be capable of looking like it generates clicks or conversions but as yet I don’t think it can game uplift.

    Do one or more of the above and the incentive to buy inhuman traffic is vastly reduced. Advertisers ultimately want value for media, not to get through budgets. If you are working for a boss who doesn’t understand this mentality and it frustrates you to be wasting money change company. Ultimately if you’re generating value from the media budget you’ll build long-term value in your company and your career.

  3. Steve Balducci

    John Were, you say “CPC and CPA buying allows fraudsters to arbitrage fake impressions and multiply their margins massively.”

    Can you explain how this happens? Thanks.

    • No problem Steve. Here are three example buys of the same fraudulent media on different metrics. The media is recorded as correlating with 2 clicks and one conversion.

      1. Advertiser ‘A’ buys 1,000 impressions at a $1 CPM. The seller receives $1.

      2. Advertiser ‘B’ buys 2 clicks at $1 CPC. The seller recieves $2.

      3. Advertiser ‘C’ buys 1 conversion at a $10 CPA. The seller recieves $10.

      These are simple examples which conceal many variables but given the highest CTR we’ve seen on a campaign in the last 7 days is over 1% (whereas my second example uses 0.2%) you can see how the supplier margins increase rapidly with arbitrage.