Dirty Tricks, Red Flags And Pitfalls You Should Know About ROI

jean-baptiste-rudelle“Data-Driven Thinking” is written by members of the media community and contains fresh ideas on the digital revolution in media.

Today’s column is written by Jean-Baptiste Rudelle, CEO and co-founder at Criteo.

Return on investment (ROI) is the only thing that matters for an advertiser: How much return am I really getting for my money?

To calculate ROI, typically you will define a key metric against which you’ll measure the impact of display ads. With a retail or travel website, for example, you would measure direct sales generated from your display ads or, more specifically, visits generated by those ads and the conversion into sales. For other verticals, such as automotive or CPG, conversions can be a test drive, an application submission or other industry-specific business metric. Once the metric is chosen, you need to decide when and how to attribute a conversion to this specific ad campaign.

This is the point where things become slightly more complicated.

There are fundamentally two scenarios that bring a user that has been exposed to a display advertisement to your website and ultimately generate a conversion. In the first scenario, the user arrived directly following a click on a display ad banner. This is called a click-through or post-click conversion. The second scenario is indirect, as the user decides on their own to visit the website. This is called a view-through or post-view conversion.

Click-through conversions have no issues. Getting a user willing to interrupt their browsing to click on a banner ad is incredibly hard. As a result, it’s a very strong marketing signal. Furthermore, it’s very easy to control quality of click-through conversions. You just have to monitor conversions of those incoming post-click visits during a predefined attribution window, which is typically 30 days. This has been done at scale for years on search campaigns. As a result, advertisers are usually very quick to spot the difference between high-quality clicks generating strong post-click conversions and poor clicks that need to be eliminated from the marketing mix.

The implicit assumption of a cost-per-click (CPC) pricing model, which guarantees clients always pay the right price, is that only click-through conversions should be taken into account for the calculation of the ROI. This is why the CPC model is so popular among advertisers, as it makes both ROI calculation and quality control so easy.

The challenge is that it can be difficult to show a decent ROI with pure click-through attribution. Very few service providers are capable of offering technology that performs at scale under this demanding attribution model.

As a result, it’s very tempting to add to the mix some view-through conversions on top of the click-through conversions. Suddenly, the ROI looks so much better. View-through conversions are based on the idea that a banner ad impression has “influenced” the user causing him or her to visit the website at a later time. View-through attribution gives ad banner campaigns some credit for those conversions.

This is where things get messy.

View-Through Dirty Game

How do you know if a particular user was truly influenced by a banner ad or just visited the site for another reason?

In practice, there is no way to make this distinction. View-through advocates try to circumvent this issue by asking for a very conservative attribution window. For instance, while click-through conversions typically have a 30-day window, they will measure view-through during a 24-hour window or even just a few hours. This tight attribution window looks very reasonable at first glance, but in reality, even a small view-through window can completely distort the ROI calculation.

To explain why, we need to go back to a little secret of the display industry: A very large fraction of the ad banner inventory is below the fold – more than 70% of impressions on certain networks. Those impressions are very cheap to buy, precisely because almost nobody sees them.

To artificially boost view-through conversions, you just need to buy large amounts of those very cheap below-the-fold impressions. Users won’t see them, but who cares? The cost is minimal and the point is to simply drop a tracking cookie for each of those impressions and then claim credit for any conversion later. This is what experts call “cookie stuffing.” Unfortunately, there are still some advertisers that are unaware of this dirty trick – and even when you are, it’s very hard to prevent – which makes this technique even more tempting.

Furthermore, if retargeting is a significant part of your mix, giving credit to view-through conversions starts to make things really unpleasant. By definition, users who leave your website are the ones most likely to come back spontaneously. So if all your own traffic gets tagged with cheap below-the-fold cookies, you will end up with a massive number of illegitimate conversions wrongly attributed to your retargeting campaign. It looks like great ROI, but in reality, it’s a complete waste of money.

Scientific Measure

Ultimately, the only way to measure the precise uplift of display ads is to run a statistically significant and valid A/B test where you expose only one group to your display ads and you then measure the difference in results. To avoid any sample biases, A/B tests require a very tight protocol, as rigorous as implemented for drug trials.

Based on hundreds of A/B tests, experience has shown that actual total revenue uplift is, on average, around 20% higher than when only click-through conversions are taken into account. In other words, view-through conversions typically generate around 20% of the overall value for display ads.

So if your ROI calculation requires a contribution of view-through revenues significantly higher than 20%, it should immediately raise a red flag. A share of view-through conversions above 50% is very suspicious, and most probably grossly overestimates the overall value of your display ads.

Furthermore, to guarantee that interests are truly aligned, you want to incentivize your service provider for click-through conversions only. Focusing on click-through conversions is also the only way to rigorously compare different solution providers. Beautiful view-through conversions are just, at best, icing on the cake.

Don’t be fooled. Your money is too precious and you should demand real ROI.

Follow Jean-Baptiste Rudelle (@jbrudelle), Criteo (@criteo) and AdExchanger (@adexchanger) on Twitter.

Enjoying this content?

Sign up to be an AdExchanger Member today and get unlimited access to articles like this, plus proprietary data and research, conference discounts, on-demand access to event content, and more!

Join Today!


  1. Thanks for highlighting the measurement challenge and I completely agree that an A/B test is one solid way to measure the incremental benefit of a given marketing campaign. Of course, many conversions happen that are NOT covered by pixels firing (i.e. cross device, offline, cookie expunged, et cetera) – so any valid A/B test needs to account for these use cases too.

    My concern is that someone reading the article might be tempted to rely on a 20% average for some sort of ROI calculation. Averages can be a dangerous thing! The actual contribution of a view-through impression varies dramatically from campaign to campaign, based upon the audience targeting, advertiser, creative, media placement et cetera. It is just plain wrong to plug in 20% as an ‘average’ into an ROI calculation. How do I know? We have been doing carefully controlled split run testing for our clients over the past couple of years. We systematically measure the incremental impact of digital campaigns on both online and offline sales by doing a 1:1 CRM match with actual client sales data.

    I can tell you that the results vary dramatically campaign to campaign. We have measured results for a given campaign at 43% incremental impact and as low as a 0% impact… all for the same client.

    To be clear – I don’t think you are advocating marketers should assume a 20% average view-through conversion credit, but my concern is that some might read your thoughtful piece and be tempted to take that short cut.

    Marketers should demand a real ROI, but plugging in an average value to an attribution equation will not get anyone the real ROI that we are all seeking. That kind of thinking will always lead to “average” results and data-driven thinking should be about de-averaging to find the exceptional.


  2. There is another major “pitfall” or “trick” that is missing here: believing that a clic on a retargeting ad followed by a post-clic sale equals to incremental sales.
    As rightly suggested in the article, implementing an A/B test is the best way to quantify the true business impact. This applies to clic-based retargeting as well. Very often, A/B tests of retargeting show that the true business is limited. It turns out that many clics on retargeting banners ads are actually navigational and do not reflect a modified user behavior. Just like brand terms in search.

  3. In theory of course you are right, but clicks can just as easily be gamed as impressions can, with click bots being capable of generating fake clicks. In truth neither impressions or clicks are a particularly good metric for online display ads. Your point on using an exposed vs non exposed sample is a good one and this is one of the ways to measure influence. Another way is to look at the impact on cumulative sales when activity is run, but this is difficult to do as the effect needs to be isolated from other activities such as PR and external influences accounted for. After that we are back to traditional advertising methods, trying to measure the psychological effect of advertising on people.

    Digital advertising has significantly advanced the quantitative measurement of campaigns, but qualitative measures have hardly moved on at all. In truth for influencing activities such as online display qualitative measurement is required, but the costs are still prohibitive and many digital advertisers don’t believe in their validity. It’s a case of chicken and egg!