This is the year of testing and innovation in digital advertising.
While that may sound exciting, it actually means 2024 will be a difficult – and expensive – time for marketing departments.
ASICS, the shoe and apparel brand, is an early tester of new ad targeting and analytics methods. It started by partnering with data clean room operator Habu as an “innovation and experimentation” vendor in 2020, having already built its data set on Google’s Ads Data Hub clean room.
Its work with Habu and across data clean rooms and walled garden media platforms is a good example of how marketers can effectively test and learn using new-age programmatic solutions.
But the brand’s testing regime also shows how difficult a multiyear process of experimentation is for marketers.
Growth by increments
ASICS added Habu as a testing vendor primarily to understand the incremental customer acquisitions generated by different online channels (Google and Meta, namely), said Devin McGuire, the brand’s manager of global performance marketing.
Many marketers are trying to “get beyond traditional media metrics,” she said.
But although the platforms have all responded to this need by developing incrementality products, each is, unsurprisingly, grading the value of its own media and has its own way of determining so-called “incrementality.”
Some “incrementality” measurements, which claim to be the acquisition of net-new customers to a brand, are just based on whether the campaigns added new profiles to the CRM or if users weren’t recognized by the walled garden’s platform.
“User-based” incrementality measurement, meanwhile – such as when you split a list of known customers to test half and half or attribute incrementality based on CRM additions – comes with its own biases, said Habu’s head of customer success Avanti Gade.
It might only work for specific prospecting campaigns or to target narrowly within one platform, she said.
That’s why ASICS and many large brands use geo-testing, Gade said. Geo-testing doesn’t evaluate at the user level, but instead compares entire markets as control and testing groups.
Although this approach isn’t deterministic, it can be scaled to include retail store sales.
The geo setup
For ASICS, choosing geo-testing for its own incrementality measurement was the easy part. Setting up the experiments, though, is a beast.
ASICS must work with its retailers and distributors in different markets to see where and when there are sales or new product launches that could disturb the equilibrium of testing. It must also carefully balance the scale requirements to conduct statistically significant tests without interfering with the company’s sources of daily revenue.
New York , say, or Los Angeles, both major population centers with lots of active runners, are simply removed from the testing pool, McGuire said. Those markets are too important to sales for marketing to mess with, bearing in mind that geo-based incrementality testing requires audience suppression).
Once ASICS has chosen geos with comparative size and demographics, online and ecommerce sales rates, local promotional calendars and cross-channel media consumption, it must then isolate the channel it wants to test, such as incremental customer acquisition from Google Search.
“Setting up these incrementality experiments is very time consuming,” McGuire said.
The other costs
Aside from the obvious cost of adding a vendor, brands must also think carefully about how testing affects the overall sales funnel and the indirect costs of audience suppression (you’re missing those customers, after all) and experimentation.
With cloud-based platforms like Habu or Ads Data Hub, which is a Google Cloud Platform product, the costs are tied to compute power.
Partly as a cost management measure, ASICS uses Habu as an experimentation vendor for post-campaign analysis, feeding in data about specific campaigns in specific markets. If ASICS had its entire data warehouse on Habu and queried that full data set constantly for analytics and campaigns, it would consume too much bandwidth.
With walled garden platforms, they just want more: Your live server data, your CRM, every single creative you’ve ever made. “Feed me,” says the walled garden.
When it comes to cloud analytics, however, anything that consumes data must be contained, or it will run up the cloud bill.
ASICS does plan to expand its testing program with Habu, though, McGuire said. Incorporating data from wholesale partners – like Dick’s Sporting Goods or Footlocker – in a data clean room collaboration would improve the incrementality tests.
She added that privacy regulations and data protection policies have complicated the job of performance marketers – or any marketer, really.
Innovation and experimentation come with a cost.
“But we need to make sure we’re showcasing to our stakeholders the true value of our marketing efforts,” McGuire said, “especially in the age of budget cuts for marketing departments worldwide.”
