Home Data-Driven Thinking It’s Time To Rethink Cross-Device Accuracy

It’s Time To Rethink Cross-Device Accuracy

SHARE:

timabraham“Data-Driven Thinking” is written by members of the media community and contains fresh ideas on the digital revolution in media.

Today’s column is written by Tim Abraham, director of data platforms at Adbrain.

Digital channels have provided a great return in dollar value and consumer insights. The emergence of cross-device solutions measurement has become a priority, and rightfully so, but it remains a poorly understood concept.

In the context of cross-device advertising, there’s a fixation on the accuracy of probabilistic solutions compared to the login-based matches produced by the likes of Facebook. Marketers typically think of accuracy in terms of cross-device technology being able to correctly attribute cross-device behavior to a single consumer, something Facebook does well.

The rise of competing cross-device solutions has created confusion around this idea of accuracy and, at worst, it is purposely misused. It’s about time we move past this simplified idea of probabilistic vs. deterministic cross-device solutions and focus on metrics that reveal a different story.

Rethinking Accuracy

Accuracy is a metric that doesn’t necessarily mean what people think it does. In the context of cross-device identification, accuracy is calculated as the number of matches correctly identified, as well as number of non-matches correctly identified. In other words it’s the number of times a probabilistic prediction was correct but also includes “non-match” predictions from the total pool of predictions it made.

Marketers don’t care very much about the non-match predictions because they want the predictions of correct device matches. But there will actually be many more non-matches than correct matches so this massively skews the accuracy score, making it look much better than it really is.

For example, if you wanted to identify two people who are related to one another and you took a random sample of a crowd, there is very little chance any two people would be related, but there is a very high chance they will not be related. If you’re looking for matches, but all you end up with is non-matches, that knowledge isn’t very valuable because you haven’t actually found what you’re looking for.

Why Precision And Recall Matter

Recall and precision are relevant metrics that deserve attention. Recall is the number of matches that can be correctly identified out of the total cross-device matches that truly exist. A high recall means that a solution has predicted a large number of the matches that exist, but tells us nothing about the number of matches predicted that didn’t actually exist.

Optimizing toward recall means you get a mixed bag to speak, but this ultimately produces a larger pool of matches. Precision is the number of matches that are actually correct out of the total numbers of matches believed to be correct. This is what most people incorrectly think of as accuracy.

The Perfect Mix?

In practice cross-device metrics for probabilistic solutions can be thought of as a pulley system where improving one metric often means making a trade-off between the others. This is particularly true for recall and precision.

Depending on the application, precision might not be all you’re after if, for example, the goal is to achieve scale, in which case precision can be traded for higher recall. Scale is a major selling point for media buyers and it’s likely that finding a sweet spot between precision and recall will actually yield better results than a highly precise but inherently limited reach.

The focus on accuracy is really just scratching the surface. By looking at different metrics, such as precision and recall, marketers can effectively exceed their goals. Accuracy may, in fact, play just a small part in that.

Follow Adbrain (@AdbrainTech) and AdExchanger (@adexchanger) on Twitter.

Tagged in:

Must Read

A white Philips Sonicare electric toothbrush standing upright inside of a water ripple

Philips Sonicare Polishes Its Streaming TV Strategy Using Amazon Ads Agent

Philips Sonicare, a maker of electric toothbrushes and water flossers, decided to consolidate its streaming TV ad buys within Amazon, which has also been doing some consolidating itself.

Jed Dederick

Why Trade Desk Veteran Jed Dederick Joined The Startup ShopMy To Sell Creator Marketing

After selling ads at The Wall Street Journal and WebMD and rising to CRO at The Trade Desk, Jed Dederick is now the first-ever chief revenue officer of creator startup ShopMy.

Arielle Garcia, Chief Operating Officer, DecryptAds

Ad Tech Is Getting Better At Fixing Its Flaws Instead Of Just Calling Them Out, Says DecryptAds’ New COO Arielle Garcia

DecryptAds’ goal, Garcia said, is to give researchers, ad agencies, tech companies and regulators the evidence they need to fix security gaps in the programmatic supply chain.

Privacy! Commerce! Connected TV! Read all about it. Subscribe to AdExchanger Newsletters

Liftoff Sees Big Wins Ahead For Mobile As AI Pushes Budgets Into Apps

The mobile growth marketing platform Liftoff reported its second-ever quarterly earnings last week. Shares were up by almost 25% after the company exceeded its revenue target and projected an optimistic profit margin outlook moving forward.

A measuring tape around a TV remote, symbolizing screen time, binge-watching, and daily media habits. top right, copy space

CTV Has Trust Issues

CTV sellers are promising measurable outcomes – even guaranteed store visits – but winning advertiser trust takes more than just measurement.

Google Will Now Tell You Exactly How Much It’s Saving You On Invalid Traffic

For the first time, a new feature in Google Ads breaks down the dollar amount a campaign saved from IVT blocking and exactly how much Google is crediting the campaign for bots caught post-billing.