Home Data-Driven Thinking Marketers Can’t Overlook Simpson’s Paradox In Programmatic Buying

Marketers Can’t Overlook Simpson’s Paradox In Programmatic Buying

SHARE:

Data-Driven Thinking” is written by members of the media community and contains fresh ideas on the digital revolution in media.

Today’s column is written by Cameron Wertheimer, director of corporate development and strategy at Vertical Mass.

As budgets continue to shift toward programmatic, it is more important than ever for marketers to use statistics when analyzing their campaigns. Even a moderate amount of statistical analysis can have an outsized impact, given the amount of data a single spend can generate.

One little-known statistical fallacy called Simpson’s paradox deserves special attention in digital advertising because it can cause marketers to misinterpret the results of their campaigns and waste money.

“Simpson’s paradox, or the Yule–Simpson effect, is a phenomenon in probability and statistics in which a trend appears in several different groups of data but disappears or reverses when these groups are combined,” according to Wikipedia.

Don’t worry if you had trouble following that. Simpson’s paradox is much easier to understand when illustrated with an example.

Suppose an agency is running a campaign with click-through rate (CTR) as the main objective. The campaign manager pulls the first weekly performance report and breaks out the results by gender.

Female Male
Clicks 500 750
Impressions 50,000 50,000
CTR 1% 1.5%

 

The chart shows that males have a 50% greater CTR. The campaign manager concludes that more budget should be allocated to males. However, that would be a mistake if other variables aren’t taken into account.

Here is the same data set broken down by age.

Female Male
18-24 25-34 18-24 25-34
Clicks 470 30 740 10
Impressions 25,000 25,000 40,000 10,000
CTR 1.88% 0.12% 1.85% 0.10%

 

Subscribe

AdExchanger Daily

Get our editors’ roundup delivered to your inbox every weekday.

The female grouping still has an aggregate 1% CTR, and the male grouping still has a 1.5% CTR. However, this new data indicates that the campaign manager should increase spending on the female 18-to-24-year-old cohort.

The age grouping is a confounding variable. It plays a major role in determining CTR, but it was not observable in the first table due to how the data was broken out. There are two lessons we can take away from this.

First, planners should not rely on overly broad audience segments. Many buyers use reach as their primary criterion to ensure that their campaigns scale. This approach leaves them susceptible to Simpson’s paradox.

In the case of a brand trying to reach pop fans, it would most likely be helpful to break out those pop fans by their passions, such as whether they are concertgoers, heavy music streamers, merchandise purchasers or social engagers.

These distinct groups of pop fans may react differently to different messaging styles. For example, concertgoers may be more likely to engage with an ad showing people at a concert, whereas heavy streamers may be partial to seeing imagery featuring someone listening to music at home.

The buyer will not understand these distinct groups unless they plan ahead of the campaign to surface those groups. Working with granular, robust audiences is key to combating this common mistake.

Second, planners should work with their campaign managers to break out other targeting variables as much as possible to surface other confounding variables. Time of day, day of week, ad exchange and browser are some variables that often are not surfaced in campaign reports and therefore muddle the conclusions drawn from campaigns.

These variables should be considered not only for reporting but also during the campaign setup process. This entails creating targeting groups that correspond to the variables. Campaign managers tend to check the performance of targeting groups on a daily basis, and they tend to pull more detailed reporting on a weekly basis. Matching targeting groups to key reporting variables during the campaign setup will make the campaign manager more aware of the key variables on a day-to-day basis.

There are drawbacks to setting up campaigns with too many data sets and targeting groups, however. The added complexity increases the likelihood of mistakes and makes optimizations more cumbersome. Having too many targeting groups can be also inefficient when there is not enough budget to deliver a statistically significant number of impressions against each group.

Planners should try to deploy appropriately granular audiences, and the campaign setup should aim to surface confounding variables without being disproportionately complex relative to the overall budget.

Follow Vertical Mass (@VerticalMass) and AdExchanger (@adexchanger) on Twitter.

Must Read

After The Election, News Corp Has Harsh Words For Advertisers Who Avoided News

News Corp’s chief exec blasted “the blatant biases of ad agencies and ad associations,” which are “boycotting certain media properties” due to “personal political prejudices.”

LiveRamp Outperforms On Earnings And Lays Out Its Data Network Ambitions

LiveRamp reported an unexpected boost to Q3 revenue, from $160 million last year to $185 million in 2024, during its quarterly call with investors on Wednesday.

Google in the antitrust crosshairs (Law concept. Single line draw design. Full length animation illustration. High quality 4k footage)

Google And The DOJ Recap Their Cases In The Countdown To Closing Arguments

If you’re trying to read more than 1,000 pages of legal documents about the US v. Google ad tech antitrust case on Election Day, you’ve come to the right place.

Privacy! Commerce! Connected TV! Read all about it. Subscribe to AdExchanger Newsletters

NYT’s Ad And Subscription Revenue Surge As WaPo Flails

While WaPo recently lost 250,000 subscribers due to concerns over its journalistic independence, NYT added 260,000 subscriptions in Q3 thanks largely to the popularity of its non-news offerings.

Mark Proulx, global director of media quality & responsibility, Kenvue

How Kenvue Avoided $3 Million In Wasted Media Spend

Stop thinking about brand safety verification as “insurance” – a way to avoid undesirable content – and start thinking about it as an opportunity to build positive brand associations, says Kenvue’s Mark Proulx.

Comic: Lunch Is Searched

Based On Its Q3 Earnings, Maybe AIphabet Should Just Change Its Name To AI-phabet

Google hit some impressive revenue benchmarks in Q3. But investors seemed to only have eyes for AI.