The Cookie Is Crumbling: What’s Next?

andrewshebbeare“Data Driven Thinking” is written by members of the media community and contains fresh ideas on the digital revolution in media.

Today’s column is by Andrew Shebbeare, Founding Partner and Global Chief Strategist at Essence.

Don’t worry; I’m not planning to reprise the old debate over whether data is good or evil here. I assume this audience sides mostly with the argument that using data to make ads better for informed, consenting audiences is A Good Thing. Instead, I want to talk about how we might try to fix the rapid erosion of our common currency, the humble cookie.

Our industry is among the most innovative on the planet. The speed with which the face of advertising has changed is bewildering – a testament to the power of free enterprise and accelerated competition with good information. On the other hand, we have a pretty poor track record in standardization or creation for the common good. Throughout the 13 years I’ve been lucky enough to be in the business, it has been awash with complaints about standards – including ad specs, viewability, GRPs, Do Not Track and so many more.

The one thing on which we’d been mostly able to agree was the basic building block of our data ecosystem – the cookie. Yet we can take no credit for inventing this standard; the cookie was never designed with advertising in mind. It was an accidental gift to marketers, one with which we made hay until “cookie” became a dirty word.

Now that gift is being taken away. Unlike Do Not Track, where the ad industry can simply snicker and dismiss IE 10’s efforts, we can’t unblock a rejected cookie. Our hand is being forced, and quickly. At Essence, we see around 9% third-party cookie rejection in the US. As Firefox updates, we’re expecting that figure to rise to 20% by late August. With two major players now taking this stance, it’s easy to imagine Microsoft following suit. That could easily get us to 40%, or two in five users for whom we’d find ourselves unable to measure reach, to frequency cap, to sequence ads, to hold out control audiences, to measure conversions – some of the key selling points of the digital medium.

The Internet has the potential to change the face of advertising for the better, but to get there we need to make ad experiences valuable to users and to brands. That value rests on relevance and accountability.

So what’s the alternative? Can the industry develop and police a mechanism that also looks after the consumer’s interest? Personally, I’m not sure any of the options available today qualify as silver bullets.

First-party ad serving? Serving all your ads as rather than might help if someone has already visited you, but if your ad is placed on someone else’s site, you’re still a third party. So that change wouldn’t help any further up the funnel than a click, and it doesn’t help grow new audiences. Big brands with huge first-party coverage would see some extra mileage here, but a B&B owner in Vermont would struggle.

Fingerprinting? Very clever – and a brilliant hack – but not the long-term answer, in my opinion. Much like the cookie, fingerprinting uses a feature of HTTP for something far from its intended purpose. But the fingerprinting arms race and low barriers to entry are creating fragmentation and inconsistency, making it difficult to manage privacy well. Some of the innovators in this space are already reinventing themselves as media companies to monetize their inventions, exacerbating this silo effect. Fingerprinting is probably helpful in the near-term, but it all feels very fragile. I could easily see browser makers altering their security models to obscure the data used for fingerprinting from third parties altogether.

Panels? Great for all sorts of things, but I can’t see them becoming the single currency of digital marketing. They’re inherently proprietary, biased, limited in reach (especially globally), and they suffer from sampling issues. Finally, they only really help with measurement anyway; they aren’t going to form the basis for a data economy.

Logins? Facebook, Google and Apple are rapidly heading for domination of the data market, thanks to logged-in user bases on their respective platforms. Tracking and targeting users across mediums and devices will help them monetize in completely new ways, which will create amazing opportunities for marketers. Yet relying only on logins would reduce the accessibility of our ecosystem. That would, in turn, diminish innovation in our business because so much of our innovation comes from the smaller players.

For the record, I do think all of the above are helpful. In fact, we use or recommend them all to our clients today. I’m just trying to take a punt at a long-term solution. We’re in a classic collective-action problem;  it’s in everyone’s interest to find an answer, but no individual interested player can get us over the hump. My sense is that a sustainable answer has to be designed specifically to fix this problem, has to be understood by consumers and has to be “open,” or not owned by anyone.

So what might this magic something look like? I’ve got a rough sketch of a potential solution, and I’d love to know what you, the AdExchanger readers, think. I’ve kicked this idea around with colleagues, but I’m sure we’ve missed plenty of angles. I started thinking about the two roles that the advertising cookie serves today, then about how we might solve for each one if we were designing from scratch.

1) Measurement: We need to know if our ads work. At a basic level, this means comparing the behavior of people who see our ads to those who don’t. I don’t think this needs to happen at the user level.

Here’s an idea: When a browser is installed, we could have it generate a persistent, anonymous, random and non-unique group identifier. A number from 0 to 100, for example. Let’s call it the Audience Group for now, or AG for short. The AG would be passed to the server with every HTTP request the browser makes, so that advertisers could use it to persistently segment audiences into up to 100 groups. It could be reset by the user, much like the Apple IDFA. For most advertisers, there will be thousands or millions of individuals in any AG.

Marketers would use the AG to compare relative ad effectiveness between groups. Instead of looking at clicks, CTRs and so on, we’d all be engaged in the much sounder practice of comparing the real overall effect of ads on audiences. Show Ad A to Group 1, Ad B to Group 2, no ad at all to Group 3, then compare their behavior. It would be easy to build proper Group-based creative tests rather than make spurious comparisons across rotating ads at uneven frequency or recency levels. We might finally get everyone aligned on conversions that are genuinely incremental rather than post-click or post-impression.

I suspect people will be reading this and silently complaining that they can’t build an attribution model without data-describing sequence, frequency and recency. That’s true, but it’s the price of real anonymity. On the bright side, since every publisher and ad server would see the same AG value, all media would have equal access to the segmentation, making it easy to create pretty amazing experiments to measure the contribution of different channels (such as, for example, avoiding showing half your audience generic search terms and subsequently comparing the number of brand searches they make and/or their likelihood of watching your video to completion).

2) Targeting: This is where user-level data comes in. To measure reach, to frequency cap, to retarget, to run behavioral campaigns, to trade in data, to build an attribution model, we need to be able to identify individuals in a consistent way.

If we’re getting personal, we should ask for permission. So here’s another idea: When a browser is installed or updated, we could prompt users to tell us if they’re prepared to share anonymous data about their web browsing, recorded against a Tailored Advertising Flag – or TAF – unique to them. We’d explain that sharing this data would help support the sites they enjoy, help advertisers to deliver ads that are relevant to them and reduce the amount of repetitive ads they see. I see no need to default to opt-in or opt-out; just ask people to make a choice. Should the user opt in, we’d generate a random, unique TAF that would be passed into every HTTP request, just like the AG. That key could be used just like a cookie is today, except that the value would be the same across all requests. This would make it easier for users to manage their privacy while also making it easier for marketers to use different platforms and for ad people to trade data. All the systems we have today would basically still work; we’d just change the key and accept the fact that only a part of the audience would be covered.

Advertisers would probably gravitate toward the TAF opt-in audience, and we’d have to find ways of understanding and managing ads against those who opt out. For example, if we can’t frequency cap, we have to expect worse results for users who opted out. This might create a bit of a two-stream digital-ad economy, but at least the choice would be made — and understood by — consumers. You don’t have to look too far ahead to imagine asking users to pay a small fee to opt out, or to reward them for opting in, with money, access to premium content or fewer ads. This seems only fair, given the revenue differential for publishers.

This route has the significant advantage that only the browser makers would really need to change anything significant. Sure, those browser makers are behemoths with less-than-aligned interests, but they’re getting better and better at this standards malarkey.

If we can align behind something like this, the reward feels worth some  effort. We’d finally have a solution consistent with the principles of the Internet – owned by no one , helpful to everyone. We’d have a level playing field for innovation,  better tools for marketers and clearer choices for consumers.

Follow Essence (@essencedigital) and AdExchanger (@adexchanger) on Twitter.


Enjoying this content?

Sign up to be an AdExchanger Member today and get unlimited access to articles like this, plus proprietary data and research, conference discounts, on-demand access to event content, and more!

Join Today!


  1. Lizzie

    Wouldn’t it be more efficient use to use the “TAF” for measurement as well? Or is there a reason to have 2 models if one can potentially do both things?
    Also, I’m wondering if there’s a possibility to leverage this across mobile as well. For example, when you get a phone you opt in or opt out up front and a “TAF” number is assigned. This actually expands what we can currently use cookies for today.

  2. Hey Lizzie,

    Thanks for your comment.

    Yes – if you get a TAF opt-in from a user, it would definitely serve both purposes. However it will never cover opted-out users, so it is only ever going to give directional reads on performance. The AG gives an absolute overall measure of lift, for the whole population, with anonymity.

    And completely agree – this is a big opportunity for mobile clients, and cookie-less environments in general (why not embed AG / TAF into set-top boxes, consoles, e-readers..)

  3. Joseph

    I find the premise of using TAFs intriguing, but I don’t anticipate users opting-in unless a compelling value proposition is made to them. Whether that is access to exclusive content on the publisher side or discounts/free stuff on the marketer side I don’t know, but I feel like most users would need some form of compensation to cough up such valuable data. As they should.

    • Jae H. Lee

      I think initially many users will opt out. But that is where the economics of a capitalistic market would prevail. Over time marketers and content providers will begin to bifurcate their offerings naturally based on the user base – opted in with rich data vs. opted out with generic data. Over time consumers will realize the difference and the value they will get will outweigh the concerns they have and the pool of opted in users will increase.

      At the end of the day it will get us to the same ending as you described – compensation for opting in – but in a way that is more organic than what I envision compensation programs to be. This of course is depends completely on marketers and in turn publishers finding enough value in the initial opted in category to target those users specifically with higher value assets.

  4. Andrew Casale

    The browser makers wield great power – the power to disrupt ad tech by taking away our precious cookie – and the power to give us something back. Your thoughts are really forward thinking which I like, my only question is even if we all collectively rally around an alternative, can we convince the browser makers to put something on their roadmap for us? I feel like a lot of their latest moves have been strictly motivated by creating more differentiation in the eyes of the consumer – a “safer” or “more privacy friendly” or “less creepy” browsing experience – merely to win marketshare.

    • Andrew – Totally.

      I’m up for some concerted lobbying if you are 🙂

      Seriously though, I can’t see how else we get out of this corner without a concerted effort as one voice – but it seems everyone wins if we do.

    • I think the pitch has to come from the publishers – if the NYT starts putting something on their site saying they prefer you to view their site with Chrome because they are seeing lower revenue from Firefox… Mozilla is going to look for a way to get rid of that. Mozilla isn’t going to listen to a bunch of adtech people in the same way they would listen to the big publishers.

  5. Alejandro Correa

    for whatever it’s worth, I think #1 is brilliant. I don’t think the browser makers would necessarily have to be essential to this approach…in theory, publishers could manage these groups based on their own cookies (or ISPs based on IP ranges, or mobile providers by cell tower), but as you point out, the browser would be the cleanest and easiest to implement solution.

    As for targeting, it has always seemed to me that the immediate context of the impression is more important than any insight into “who the user is.” In fact, the “behavioral profile” of the user is just an aggregate of any context through which a person is reached. Rather than come up with a better cookie, I would focus more on improving the “of the moment” data. Higher levels of granularity regarding the user’s physical location (anything from what the weather is like, to what they’re likely to be doing there, etc) to better semantic analysis on content, to better understanding of the device on which the media is displayed. As audiences fragments and attention spans decrease, I think the immediate context of messaging will be more indicative of propensity to take an action than anything that could be inferred from the past.

    Love the ideas here!

  6. Siyun

    First off, I support the AG and TAF solutions whole heatedly.

    One question though, how this can solve the problem of cross-platform users tracking.

    More specifically, how you can consistently identify the same user across app/ mobile web /desktop? TAF might be able to link mobile web and desktop user but I cannot figure out how you can sync with app users. Currently this can be done on a log-in based scenario – the same Android ID, for example, will follow your phone wherever you go, either on apps, on web or desktop (assuming you use Gmail on your phone; sync gmail account with Andorid ID).

    • Hi Siyun,

      First off, happy to hear it!

      Second, very good shout. Obviously any login relationship that might be used to “join” TAF identities is confidential data shared with the first party (ie Google / Facebook / Amazon / eBay) and should be governed by their respective privacy policies. Those companies and only those companies would be able to make the connection – for themselves or for sale. One tricky area if that data were to be sold is that it is easy to port/steal. Once I know 2 TAFs belong to the same person, I know it forever. type solutions would also continue to work probabilistically using 3rd party data, IP, geolocation etc. just as they do using cookies . But you still run into user choice – what if I don’t want to be matched?

      The alternative is asking users to join their own TAFs together or log in to all their devices with a single TAF. That feels like a tall order at the moment – a fair amount of effort with that much obvious payoff. In the future though, functionality like Chrome’s browser login could easily sync TAFs across devices in the same way they sync bookmarks; if you are willing to opt into more relevant advertising, you’d presumably be willing to have that relevance everywhere. As browser / ecosystem choice becomes a multi-device decision, this could catch on.

      • Siyun

        Got it. So lots consumer educations and industry wide cooperation ahead 🙂

        I know Drawbridge and their algo. While I do think it is a good intermediate solution, it is not going to fix the problem we all facing today. Especially there is mobile cookie involved – we all know mobile cookie is unreliable and buggy :/

        Anyway, kudos to thinking ahead of the curve 🙂

  7. Adit Abhyankar

    Great article. If TAF is also going to essentially be a sample of your population because some don’t opt in, do you end up with more coverage than you will just staying with cookies? Also, if 3rd party cookies get treated as 1st party cookies by the browser upon a click it will dilute the effect of Mozilla’s cookie blocking over time.

    The core challenge here, however, is not in cookie technology. The challenge is unaligned interests. For that we don’t need new tracking technology. We just need to make sure interests of all parties are aligned via compensation schemes. If a user was paid/offered free stuff to share data and the browser makers were somehow compensated for enabling the ad-tech economy we could happily live off cookies forever because it would be in no-one’s interest to kill them. It is far from perfect technology and we do need a new solution that will work cross-device, but the immediate focus should be on aligning everyones interests and any technology that will enable that.

    • Hi Adit,

      I like your thinking.

      I had always assumed that Mozilla realise that treating a redirect served cookie as a first party makes their rejection of 3rd party cookies somewhat meaningless, but that’s conjecture on my part. I see as I write this that they’ve decided to delay anyway.. maybe we can take the credit 🙂

      As to why we need something other than the cookie – this started with the “AG” idea, which I really like. I don’t think we could deliver that with cookies. The AG requires non-uniqueness, anonymity, consistency, persistence. The non-uniqueness is enforceable because the AG is set by the browser; the value is read-only as far as an advertiser or website is concerned. I think this type of approach could create a new measurement paradigm – simple but truthful.

      If you’re doing that anyway, it seems a natural choice to create an equivalent parameter that is unique but pro-consumer and transparent from the outset by requiring user opt-in. Cookie is unfortunately a dirty word among legislators and consumers. The benefit of a single ad preference setting and a common currency for data helps, of course.

      But I agree. In theory, the 3rd party cookie problem is fixable and if the incentives were right cookies could give us much of the upside of the TAF. In practice, I wonder if it’s just too far gone.

      Hope to catch up soon.


  8. Fraser Steen

    TAF sounds suspiciously like DNT. The only difference being the unique ID which would not be beneficial for the consumer only ad providers. I see no reason why any consumer focused company would implement something so blatantly public. Since the ID would identify your browser individually it would not be far from broadcasting your email address in your http requests.

    The fact remains that cookies/local storage are the only methods that allow marketers to store sensible date whilst at the same time giving users control of whom they allow to store data and what data they store.

  9. We are working an a couple of things to address that at our lab.

    Image analytics – an understanding of content consumed and submit to social media will do behavioral analytics one better. Think about the visual content you consume from the browser both desk top and mobile and what that reveals about your wants. Image analytics of social media, camera rolls, and mobile apps will provide highly detailed data about the persona of a target consumer. Lots of privacy issues but everyday people consent to location and personal information sharing with Apps. Just imagine what your Flickr content would say about you!

    ADs as APPs- when the cookie crumbles it will leave crumbs. if you understand a little about ad-hoc networks you can invision digital ads as little apps that leave a trail (of crumbs.. get it) of their consumption and actual transactional results.

    necessity is the mother of invention.

  10. I like the idea of a TAF, even if it is substantially similar to DNT. I like to think about the problem from a slightly different angle. Where most discussions to date have been centered around the problem cookie deprecation will have on ad networks/exchanges/DSPs, etc. It might be worthwhile to we move our focus a bit more upstream to the publishers (who will be negatively impacted from a revenue standpoint).

    Here’s my logic. Users want “free” access to great content, much of that content is monetized by advertising and much of that monetization is driven by 3rd party sources. If the monetization falls away, then publishers/content producers will be substantially impacted. Thus offering the user a choice along the lines of:

    A. Subscribe
    B. Turn on TAF (or other mechanism)
    C. Access substantially “crippled” content

    In essence, make the pain felt universally in ways that are related to the desires of each player in the ecosystem. If a user is willing to subscribe (pay) for content, fine since they are doing so in an exchange of value that would indicate that they won’t be responding to advertising anyway. If they don’t pay, they don’t get the “goods” and if they open themselves back up to the TAF, then we are back to the status quo.


  11. Adam Shahbaz

    I don’t think your line of thinking is bad, but perhaps a bit idealistic. Specifically, I question your logic around not using logins. It may be true that login information is currently concentrated in a few hands, but in a post-desktop world, almost every app has access to that information (or a persistent hash) if they want it.

    Even if that weren’t the case, stifling innovation amongst smaller players probably isn’t going to convince large players like Google and Microsoft to move away from using login data. If anything, they are encouraged to discourage cookies in their browsers and move more towards logins so they maintain more control.

    From a user perspective a login can be better, too. Today I got an airbnb ad that was clearly based on my history while browsing on a computer I had never touched before. Of course, I was logged in to Google.

    From a privacy perspective, I think it makes sense for the publisher to say “hey we’re going to show you adds via a hashed match to your login credentials.” And logging in is what differentiates the full experience from a “crippled” one.

    This concept sounds like it favors the Googles and Microsofts of the world, but maybe this is the kind of pressure we need as an industry to move beyond our old habits, and let new forms of media consumption and native advertising flourish.

  12. For AG Rather than rely on the browser for the HTTP request, could it be possible that the ISP provide this information? If that is the case, then the user could potentially be rewarded for providing this data, for example 10% off their broadband bill. In return the industry would pay the providers for this information.