If you’re playing buzzword bingo, your card’s about to fill up, because this article covers automated incrementality testing, causal measurement and full-funnel media-mix modeling (MMM).
On Tuesday, media measurement startup Measured released a souped-up version of its platform that fully automates the process of incrementality planning and optimization, including geo-based testing.
The platform update also combines an MMM methodology with causal (aka incremental) measurement to help marketers better determine the true impact of their marketing spend on campaign performance without user-level tracking.
“The best modern measurement is all cohort based,” said Measured CEO Trevor Testwuide in a not-so-subtle dig at multitouch attribution, which is becoming untenable as a result of signal loss and privacy changes.
“By leading with experimentation,” he said, “our clients have pulled us in the direction of adding MMM to our platform.”
By all measures
MMM and incrementality measurement each have their pros and cons, which is why bringing them together creates “a more complete, elegant measurement solution,” Testwuide said.
Incrementality measurement uses randomized test and control groups to help answer the question of whether an ad exposure led to a conversion or whether that person would have taken an action regardless.
But the tests themselves are time-consuming and difficult to set up.
“It takes a lot of humans and hands on keyboards to design the experiment and deploy the campaigns in the holdout conditions,” Testwuide said. “And then they have to interpret the math, which is labor intensive, and there’s a high degree of risk [of miscalculations].”
It’s also not always easy to isolate the impact of a specific media channel. Who’s to say, for example, whether people in a test group in a certain geo really converted because they were exposed to a particular podcast ad or influencer campaign?
An MMM approach, meanwhile, considers the statistical relationship between different factors that can influence a sale, such as market trends, seasonality and promotions. This can help marketers identify their most effective channels. But MMM models typically take a long time to gather enough data to do analysis, which can be expensive.
The update to Measured’s platform simplifies and speeds up the process of incrementality testing. The system automatically selects the smallest possible – but still statistically significant – holdout group, which reduces the risk of sampling bias. Carving out a smaller holdout group is also less disruptive to a brand’s other marketing efforts.
The results of these incrementality tests can then help improve the accuracy of Measured’s MMM model, particularly for channels that can be difficult to measure, such as social influencers, streaming audio and national linear TV.
“An incrementality test gets you as close as possible to the ground truth of media contribution,” Testwuide said. “But you have to feed incrementality into an MMM model for it to be truly reliable.”
ROAS isn’t reality
The ability to measure incremental marginal return is highly appealing to Measured’s long-time client Faherty Brand, a family-run apparel company founded in 2013 by Alex Faherty and his twin brother, Mike.
Alex, the company’s CEO, has a background in finance, so measuring return on ad spend – which most brands prioritize as a matter of course – never made sense to him, he said.
“I want to know the incremental profit per dollar we’re making on our ad spend on an incrementally adjusted basis,” Faherty said. “It’s not that ROAS and clickthrough aren’t relevant, but that’s not what we care about on a daily basis.”
Faherty Brand started working with Measured in 2018, roughly a year after the measurement startup first launched. At the time, it spent a lot of money on its catalog as a prospecting channel, but incrementality testing proved catalogs weren’t efficient, so it pulled back.
That money can now go toward more brand awareness-building channels at the top and in the middle of the funnel, such as CTV, podcast advertising and influencer marketing.
Next up, Faherty Brand is planning to run tests to understand the causal impact of media spending in different markets on in-store sales versus online.
“Overall, we spend a lot less time thinking about the bottom of the funnel, all those high ROAS last-click channels,” Faherty said. “It’s just a more realistic way to approach spending money.”
But the relationship with Measured is also helpful because this stuff is technical. And even though Faherty Brand drank the “incrementality Kool-Aid,” Faherty said, it’s an apparel company. There are no data scientists on the team.
“We can see all our channels in one integrated dashboard against the incrementality, and that makes the data very clear,” he said. “It gives us comfort to spend money in areas that are harder to measure.”
Not that any measurement approach is perfect, though. Perfect measurement is impossible, Testwuide acknowledged. But perfect isn’t the point.
“You know that phrase, ‘All models are wrong, some models are useful’?” he said. “Testing to line up to causality is the closest you’re going to get to ground truth contribution – certainly not perfect, but we can take those reads and feed them into a correlation-based model like MMM.”
