Everything You Know About Frequency Is Wrong

By
  • Facebook
  • Google Plus
  • Twitter
  • LinkedIn

andrewshebbeareupdated"Data Driven Thinking" is written by members of the media community and contains fresh ideas on the digital revolution in media.

Today’s column is written by Andrew Shebbeare, founding partner and global innovation director at Essence.

It’s a slightly sensationalist headline, I admit. It’s been a while since I contributed to this series and I figured I might need to grab some attention.

This isn’t complete link bait though. A year ago I wrote that frequency control was “RTB’s unsung hero.” I still believe this is true. One of the greatest sources of waste in media is ads directed at people who see them more often than needed, or too few times to notice. Programmatic has the power to eliminate that waste.

Unfortunately, this is more complicated than it might sound. I wanted to share some of the things I’ve learned and some practical suggestions that should help.

Long Tail, Stubby Torso?

It is a dirty digital secret that frequency tends to get concentrated. The averages you hear mask the fact that frequency distributions usually have a very long tail, with small percentages of users seeing a big share of impressions. This effect can stay hidden in standard reports which cut off at around 15 exposures, but some log analysis will unearth the truth: I have seen some cookies – likely not all human – subjected to thousands of impressions a month.

With all those impressions soaked up, somebody has to be underexposed. The flip side of the long tail is a modal frequency sitting at around one or two ads – large swaths of your audience probably aren’t noticing your ads at all.

We know that frequency drives breakthrough, but too many impressions cause wear-out. Imagine if everyone saw your message exactly the right number of times. You’d see a big ROI jump, those overexposed users would thank you and maybe some of the bots could take the afternoon off.

OK, But Why Is This Hard?

Somewhere between “too much” and “not enough” must be a creamy middle – the much vaunted “optimal frequency.” But how to find it, exactly?

Let’s say people exposed to n impressions respond better than any other group. You see increasing marginal returns below n, and decreasing returns at n+1 and beyond. You might conclude that n is your optimum. In fact, is it just as likely that users with a predisposition toward your brand tend to see six impressions, because that’s how much they use the sites on your media plan. You really can’t tell cause and correlation. Experiments pretty much always show a positive relationship between frequency and conversion rate – more ads exposures usually equal more sales. When we repeat the same analysis with a placebo creative, the relationship remains – albeit weaker. Clearly there are effects other than ad exposure at play.

Undeterred, you might setup a control group, and measure the difference between the control and exposed audience at each level of frequency. A noble thought, but unfortunately even that won’t give you the answer. I have seen hard evidence proving that people who come to see n ads are different to people who see n+1. The difference in control and exposed lift could be as much to do with their demographic, psychographic or media consumption profile as the impact of your ads – you still have no clear measure of what’s optimal in terms of per-user frequency.

If you really want to understand the marginal impact of your ad impression, you need to target an audience with n impressions and compare them to a control that saw n-1 but would have seen n impressions if you hadn’t prevented it. This kind of stratified experiment is possible with programmatic or dynamic ad replacement but is fiddly to set up and requires big slices of control inventory.

If you do bravely embark on this journey, the next problem you will run into is significance. Since you are trying to measure the difference between quite similar treatments, you will need a good degree of precision, so quite large samples at each frequency level. This is hard for direct-response marketers, and even tougher for brand marketers contending with survey response rates.

A brand testing three frequency levels and seeing exposed awareness rise from 10% to 15%,16% and 17%, respectively, would need about 12,000 respondents in each frequency group to say with 90% confidence and power which is the best by a margin of 1% or more. More levels will need yet more. These numbers are going to be a challenge for anyone.

Worst of all, no matter how rigorous your experiments, it unlikely that they will spit out a single magic number – different audiences will have different reactions to ads, pre-existing attitudes and intent. We’ve not addressed the time interval over which frequency is measured, the interplay with cookie deletion or cross-device measurement. I’ve ignored the importance of pacing and recency, which are effects even harder to understand because they carry so much variability. Basically, we’re just scratching the surface.

What You Can Do About It

You might feel a bit deflated by now. The good news is that you don’t have to fix all these problems to get performance gains in your campaigns. Do the best analysis you can, mix in some common sense, segment your users and test iteratively. Use A/B audience splits and make sure you can break out campaign performance by segment – you can still unlock a bunch of value.

You don’t need to know precisely what frequency is optimal to be able to feel your way to improvements. For example, I recently worked on an experiment spanning four different frequency management strategies against randomly selected audience segments. It turns out the best treatment had up to 9% better ROI overnight. It is still impossible to say exactly what frequency is best, but layering three capping windows and reinvestment at low frequencies was better than a simple frequency cap. These represent millions of dollars in efficiency.

I recommend testing layered strategies. Most ads work better when they are delivered evenly rather than in bursts. A two-per-24 cap guarantees that no individual day gets too crazy, but still adds up to more than 60 potential exposures per month. On the other hand, you can burn through a 25-per-month cap in a few minutes if you aren’t careful. Apply both caps together, and you have a better chance of achieving steady, measured campaign delivery.

Depending on your DSP, you might need to get quite creative. Most platforms only support one cap at each level in the campaign, and it can be hard to manage the aggregate. You should team caps with pacing settings if your chosen platform has this helpful feature. Alternatively, you can use a DMP to group users by exposure history and target each segment individually – it’s more work but most flexible.

Finally, you might consider bidding a little more for impressions that aren’t your first, with the aim of building frequency to effective levels before you burn your entire budget on reach.

For The Product Managers Out There

I hope we will eventually see awesome frequency targeting algorithms that will help model and manage these effects better. In the meantime, these things would help:

  • Pacing control: I know of a couple DSPs that support pacing control, but it’s not the norm. This feature feels important, if tricky to execute well.
  • Vector-based bidding: Frequency caps are a blunt instrument. In reality, ads don’t suddenly stop adding value after your optimum is reached; it would be better to progressively taper bids based on the expected marginal value of each extra impression.
  • Supply modeling: Some users are harder to reach and will give you fewer chances to build frequency. If you expect to have plenty more opportunities, you might bid more conservatively. This game of Texas Hold ’em requires a lot of data – you’ll need to be a large-scale advertiser and/or have access to lost bid data to predict scarcity.
  • Refresh latency: It is still hard to manage frequency really tightly. Managing ad collision through near-concurrent bids is tricky, and you can still technically exceed a frequency target “by accident.”
  • Native viewability: I talk about viewability a lot, but frequency only matters for ads that are seen. The easier it is to filter signal from noise, the better we can optimize. The link needs to be at the impression level.

This geekery has massive potential. It’s the stuff that will make programmatic media kick traditional buying’s butt, especially when it comes to branding.

Follow Essence (@essencedigital) and AdExchanger (@adexchanger) on Twitter.

  • Facebook
  • Google Plus
  • Twitter
  • LinkedIn

Email This Post Email This Post

By on at

8 Responses to “Everything You Know About Frequency Is Wrong”


  1. Kristina says:

    Interesting things to test, however, I'm quite concerned about refresh latency. AMs from various DSPs told me that exact frequency cap is impossible to keep due to latency, which is from my understanding crucial for suggested test (particularly in case the tested caps are quite close to each other). I would be interested how big is the latency issue (what error margin is there usually) before designing the test.

  2. John Were says:

    Good to see you back with another insightful piece Andrew. Another important consideration is the size of the target audience versus the media budget. If the target audience is large compared to the budget then the budget could easily be wasted with too low a frequency across too high a reach. Layering frequency caps is a great idea to combat this.

    If a minimum number of impressions are needed per user to maximise impact, say 40, and the average CPM is $2 then you can say that for a $10,000 budget the maximum number of users to be targeted should be 10000/2*1000/40=125,000. New user prospecting line items can then aim to serve one impression each to 125,000 relevant users who have not seen an ad or visited the site. Other line items then retarget users who have seen an ad up to a maximum of 40 ads per user, ideally with a per day cap to even out delivery if you've got enough frequency cap layers to play with in your DSP.

  3. Hi Kristina,

    That is absolutely true, and pretty much impossible to avoid with today's technology- indeed I mention it in the post. However just because you can't control frequency with perfect precision needn't stop you from managing exposure much better. You may not be able to tell what "perfect" is, and even if you did you probably won't be able to hit it - but that shouldn't get in the way of good!

    You can measure impact of latency by comparing your campaign setting to your delivery logs across unique cookies. I can't say that I've seen any robust experiments comparing different platforms' performance in this regard for myself.

  4. Jay Friedman says:

    This is easily the single best piece on frequency I've ever seen and the most informative adexchanger article in months. While most of us have run similar internal tests and studies, it's great to see it laid out in plain english like this. Thank you for taking the time to do this.

  5. Ben says:

    Fantastic article, Andrew - as an Ops / Product guy, this is stuff I love to see on AdExchanger. Very well thought out article on how to tactically use frequency caps in exchange environments. You even covered the statistics side for good measure, which I think is often the missing piece of a lot of performance reports.

    Personally I'd like to see bidding and ad serving technology do more to integrate confidence level & interval measurements into their technologies, since the math is straightforward and most people aren't that comfortable with the topic.

    Keep writing!

  6. Siyun F. says:

    With all due respect, and some valid points, I still think the optimal frequency is measurable and important. Especially we are now adding mobile.tablet and whatever connected media to come in the picture. Knowing how many Total impressions are effiecient is a way to differentiate.

    For frequency analysis, we have very robust testing. We used propensity, or forensic if you prefer, to create "virtual control group. In this case, we are ensuring the test and control group users are similar and comparable. And we also analyze frequency efficiency and impact using historical data. In this way, we can identify pattern - for clients in specific vertical and with specific campaign type, say DR, there is, a magic number.

  7. Tom says:

    Great article. The hockey stick moment for RTB will be when the big brand advertisers genuinely start to leverage target audience reach and frequency plus variable bidding using all the data available to them... and see their business take off versus competitors. And hopefully not a click in sight, just products moving off shelves.

  8. Thanks everyone for all the smart (and kind!) comments.

    Siyun - I'm intrigued to hear more - shoot me a note if you care to share. I am sure a modeled control helps, but I also have my doubts about the accuracy of these methods - I've seen many such studies carry demonstrable bias.

Leave a Reply