Ocean Spray is using a version of the vigorous measurement approach that the Food and Drug Administration uses to test medicines and vaccines, including the COVID-19 vaccine.
The method, called randomized control trial (RCT) measurement, isn’t a new concept.
(Fun fact: Apparently, the first randomized control trial took place in Nuremberg, Germany in 1835, as part of an effort to measure the effects of a salt-based homeopathic medicine.)
Since then, social scientists, political scientists and medical researchers have embraced the approach, in which individuals are randomly assigned to a control group or exposed group in an experiment to reduce the potential for selection bias.
Now, marketing scientists are beginning to experiment with RCT.
Last year, the Advertising Research Foundation kicked off a collaborative research project to come up with standardized methods for applying RCT to cross-platform advertising analysis.
One approach devised by analytics consultancy Marketing Attribution is to combine intent-to-treat RCT with multi-touch attribution (MTA) to more accurately measure ad effectiveness for small, targeted digital campaigns inside walled gardens like Facebook and Google.
RCT measurements require such an involved process because the platforms don’t enable user-level data to leave their own systems – which means the data must be modeled – and because multiple platforms will often self-attribute a conversion for the same user. (After all, the walled gardens can’t see over the walls of their fellow gardens, either.)
Ross Link, CEO of Marketing Attribution, which is running MTA/RCT beta testing for Ocean Spray, compared the approach to a “laboratory” of sorts, because the marketer creates exposed and holdout groups before involving the ad platform.
“Both MMM [marketing mix modeling] and MTA are time series analysis; you have to go across time and collect the data. And that can take a while,” Link said, speaking at the ARF’s Attribution & Analytics Accelerator event earlier this week. “We don’t have to wait to collect the causal data, because we create our own – we’re ready to do analysis as soon as we get the sales data.”
For Ocean Spray, Marketing Attribution created a panel of 8.5 million randomly selected households, with plans to increase it to 10 million soon. KWG, Ocean Spray’s agency, regularly shares its media plans with Marketing Attribution, which also gets daily loyalty sales data by household from Catalina.
LiveRamp does all the onboarding, although it’s more like “reverse onboarding,” Link said. Rather than onboarding impressions, Marketing Attribution uses LiveRamp to distribute suppression lists to social and demand-side platforms.
“It’s intent to treat [analysis], so we don’t know who got exposed, we just know for sure who didn’t,” he said.
Marketing Attribution selects 10% of households to be suppressed for every campaign. LiveRamp translates those households to device IDs and authenticated audiences and suppresses them before sending segments to the buying platforms.
Both the control and exposed groups in an experiment like this are massive compared to the typical size of groups in most scientific studies.
With 10 million people in a panel, there’s 9 million people in the test group and 1 million people in the holdout. The COVID-19 vaccine trials included tens of thousands of people on each side.
Eight weeks into Ocean Spray’s pilot test of MTA/RCT, it’s learned that new buyers identified during peak COVID – like those acquired during the halo period around Ocean’s Spray’s viral TikTok moment in 2020 – are responsible for more sales than infrequent Ocean Spray buyers.
Last October, TikTok user Nathan Apodaca, aka Doggface208, posted a video of himself riding a longboard while drinking Ocean Spray juice and lip-synching to Fleetwood Mac’s “Dreams.” The post went supernova viral, reintroducing the 90-year-old Ocean Spray brand to – and making buyers out of – TikTok’s young audience.
Although it’s still early, Ocean Spray has also found evidence through MTA/RCT that site visitors retargeted using first-party data is the most effective target audience, said Jeff Doud, Ocean Spray’s director of advanced analytics and reporting.
Ocean Spray has already been able to use some of these findings to tweak its media plans, and the brand intends to continue running tests.
“This corroborated some of the earlier theories we had, as well as prior research,” Doud said. “The beauty of RCT is you can get this low-level, digital deep dive, and the speed you get [means] you can swap out and make changes to your different target groups or to where you want to put your investment.”
