“To say that [cross-device] is hard to do is … kind of an understatement,” said Evan Neufeld, a principal analyst at market research firm Storyline Development, speaking at a cross-device workshop hosted by the Direct Marketing Association and the Coalition for Innovative Media Measurement on Monday.
“There’s no standardization even in the way measurement terms are used from vendor to vendor – it’s a confusing vendor landscape,” Neufeld said. “That’s not the fault of the vendors. This is the way technology evolves.”
The DMA released an RFI template on Monday to help advertisers and publishers educate themselves about cross-device and ask vendors the “right questions.”
“The buyers are as responsible here as the vendors,” said Jane Clarke, CEO and managing director of CIMM. “Buyers have to demand transparency. Keep asking questions.”
For example, what is the difference between the accuracy and precision of a graph? The two terms are used interchangeably, but in the cross-device context the nuance is quite significant.
Accuracy is a measure that shows how well a probabilistic graph is performing based on matches and non-matches. In other words, if a graph is comprised entirely of correctly identified non-matches – aka this phone is not connected to this tablet, this laptop is not connected to this phone, etc. – that graph can be considered 100% accurate.
What it’s not is 100% precise. Precision only measures the accuracy of matches, not non-matches.
With that in mind, it sounds like a foregone conclusion that precision is the only metric an advertiser should look at when evaluating cross-device technology. But it’s not as simple as that.
Precision is great, but it generally comes at the cost of scale, unless you’ve got an enormous amount of deterministic data at your disposal (or you’re Facebook, Google or Amazon).
“If you have tons of data, you can afford a higher precision,” said Michael Celona, global CTO at Publicis Media and former CTO of programmatic platform Run, which Publicis acquired in 2014. “These numbers don’t really mean anything unless you look at them in context, though.”
But walled gardens are not the only option for cross-device precision at scale. Facebook et al. have a lot of deterministic data, but there’s a trend now toward combining deterministic and probabilistic methodologies for greater scale without sacrificing precision.
“If you talk to folks like Facebook, they’ll tell you that everybody else is doing it wrong,” Neufeld said. “The flip side of that, which is important for brands, publishers, agencies and ad tech startups, is to be thinking about what exists beyond these walled gardens and how we can share data. It would make for a better ecosystem.”