“Data-Driven Thinking” is written by members of the media community and contains fresh ideas on the digital revolution in media.
Today’s column is written by Alice K. Sylvester, Partner at Sequent Partners.
Marketing is too complex today to be decoded by the naked eye. Machine learning and statistical models have evolved to provide an extremely sophisticated understanding of the contribution of each marketing investment and the interactions between them.
Those models are, in fact, black boxes. They can’t be easily judged or questioned, leaving brand managers and CMOs to either blindly follow the results or ignore them altogether. Models can leave marketers vulnerable to misguided decisions and the underutilization of data and analytics, which inhibits them from realizing the full value of the insights the model can provide.
Fortunately, marketers don’t need PhDs in econometrics to get more out of marketing analytics and modeling. There are some simple questions about marketing mix and attribution modeling all marketers can – and should – consider.
Is the model complete and strong?
Does the model include all marketing investments and drivers of sales and outcomes? If not, the model can’t be trusted to accurately indicate the contribution of each investment. It’s important for the model to account for spend on digital and television walled gardens, linear or analog media and many of the non-marketing marketplace factors (e.g., economy, weather) that influence sales. Attribution modeling doesn’t tend to include these factors, but it’s still important to recognize exactly what channels are accounted for in the model. If something is left out, the model will overstate the contribution of the media included in the model.
Model strength matters, too. In general, marketing mix models should fit outcome data with an R2 in the 90% range, indicating causality between the model and outcome data, and a MAPE (Mean Average Percent Error) of less than 5%, relative to a holdout sample.
Marketers should also consider whether specific model results match the results from past analyses or in-market tests. If not, marketers need to press the modeler to find possible explanations for the inaccuracies. The model forecast is built on what’s occurred in the past. What’s different about the situation now?
Does the model account for outside factors, like brand and sales effects or initial inputs?
Marketing mix models must capture the complexity of the marketplace. It’s vital to consider interactions like advertising’s effect on price elasticity, halo effects of advertising on other brands and advertising’s less-immediate carryover effect (adstock).
Carryover also exists in attribution – it’s the attribution window. Without accounting for these effects, the contribution of advertising to sales or other outcomes is misrepresented and shortchanged.
Data inputs can have an impact on outcomes, too. Often, the data sources used for the model are not the same ones used to guide your business day-to-day. To ensure your data strategy is holistic, consider both internal and external sources. Do the outcome data KPIs (e.g., category penetration, brand shares, sales, website visits, traffic trends) look right? What about the marketing investment data? Ensuring data inputs properly reflect the brand and category is imperative.
How clear and actionable are the results?
Everyone wants near real-time attribution results. But can your operation realistically deal with daily outcome data? Or even weekly or monthly? Does your marketing mix modeling fit your business cycle? It’s important that modeling insights arrive when you need them and can act on them.
It’s also important that the modeler can communicate in layman’s terms and is experienced enough to interpret results for your specific business opportunities and risks. Are their explanations for how your marketing is working relatively close to your assumptions? Do they make sense? Do they triangulate to any other business/test results you’re seeing? This is the reality-check phase. Make sure results make sense.
Once all the insights are in place, it’s time to act against model insights. At this point, it’s crucial to have alignment between all stakeholders within your organization. Otherwise, modeling is just an interesting, time-consuming and expensive exercise.
Modeling is an important characteristic of sophisticated marketing today. Marketers must ask the right questions to understand inputs, outputs and implications. And there may be even more questions to ask relative to your specific business growth and revenue needs.
Follow AdExchanger (@adexchanger) on Twitter.
