“Data-Driven Thinking” is written by members of the media community and contains fresh ideas on the digital revolution in media.
Today’s column is written by Christophe Collet, CEO at S4M.
When was the last time you heard A/B testing and IDFA in the same sentence? The former is traditional ad campaign testing methodology going back decades. The latter is Apple’s mobile app tracking mechanism that will be deprecated in early 2021 with changes to iOS 14.
Mobile app tracking will eventually require consumer opt-in consent, and our industry will lose one of its primary targeting mechanisms. In no way can we expect the SKAdNetwork to be useful beyond billing and fraud detection on iOS.
The good news is that there is actually a way forward to mitigate the impact of this major disruption. I believe that today’s souped-up version of A/B testing will play a critical role in how our industry pivots toward a less deterministic targeting model and away from the IDFA and the cookie.
A return to basics will elevate the often-obscured importance of data science. To navigate the inevitable drop-off in the scale of deterministic consumer data signals that IDFA deprecation will cause, we would be well-advised to adopt a flexible and creative approach to statistical modeling to maintain effective audience targeting. There will be asymmetrical data sets within different populations and segments that will require more sophisticated extrapolation of consumer signals on a smaller scale.
The new era of A/B testing
Apple’s postponement of the IDFA changes is a great opportunity for marketers to start A/B testing with new statistical models that emphasize signals such as device and geography, compared to the traditional cookie- and IDFA-centric targeting.
While convenient, relying almost solely on IDFAs to extrapolate conversions has always been an incomplete exercise. This is particularly true if the holy grail is true omnichannel attribution that links online signals to drive offline and online results.
I think a more balanced approach would be to apply whatever deterministic attribution is available to statistical models that can precisely measure a more comprehensive cohort of a brand’s target audience. This may actually reveal that less addressability might not necessarily mean less return on advertising spend. I predict that incremental reach based on online and offline causation will receive greater attention as a key transactional metric in future advertiser attribution models.
Also, a greater emphasis on A/B testing will help marketers minimize test costs and balance exploration vs. exploitation in the pursuit of conversions.
To optimize A/B testing, data scientists should use the multiarmed bandits technique. The term originated in casinos; multiple slot machines – referred to as one-armed bandits – have different win probabilities, unbeknownst to the average customer. It truly is a game of chance, unlike blackjack or poker, which are truer tests of player acumen. But in the realm of multiarmed bandit A/B testing, acumen is necessary. The varying creative units, with their different win probabilities, are probably A/B testing’s most familiar one-armed bandits.
In a typical A/B test, I can dedicate the first 5% of my campaign budget for the “exploration” phase, where I employ a wide range of different creative units to determine the one with best performance. The goal here is to reach the “exploitation” phase with the smallest number of trials possible. In other words, if the “exploration” phase budget winds up as just 1-2%, that is more efficient than starting at 5%. At any rate, I then solely bid on inventory using my star creative unit hereafter.
Furthermore, A/B testing with the multiarmed-bandit framework can also be applied beyond testing creative units. It can be used to test media channels (premium vs. non-premium), dayparts, location/geography and non-IDFA-built audience segments in order to optimize conversion.
No easy path forward
I won’t sugarcoat it: Marketers are going to have to elevate their data science resources and sacrifice some level of efficiency without the benefit of the IDFA, which made A/B testing quite easy in terms of plug-and-play. But smarter modeling is the template to create user cohorts from a patchwork of consumer signals. Sure, it is going to be more work-intensive, but the end result may actually be as valuable as third-party deterministic tracking – if not more – to truly engage audiences going forward.
Change is always hard and a little scary, but I sincerely believe that a post-cookie and IDFA world might actually be a healthier and more balanced one for our industry. Innovation and reinvention are always good, and in this instance will be necessary if we want to transition successfully into the next chapter of digital advertising.
Follow S4M (@S4Mobile) and AdExchanger (@adexchanger) on Twitter.
