AI can help mitigate bias in advertising – but it’s still early days.
On Wednesday, IBM Watson Advertising and the 4A’s released findings from a six-month research project that used AI to analyze data from The Ad Council’s “It’s Up to You” COVID-19 vaccine education campaign.
The data set consisted of 10 million impressions and more than 108 variations on ad creative derived from predictive dynamic creative optimization.
IBM Watson Advertising used the data to develop an open-source toolkit called AI Fairness 360 as part of an incubation project with LF AI and Data Foundation, a nonprofit within the Linux Foundation that supports open-source work related to AI, machine learning and deep learning.
AI Fairness 360 uses 10 bias mitigation algorithms to measure ad campaigns and campaign-related decision-making processes against more than 70 fairness metrics. For this application, the toolkit was used to apply post-processing bias mitigation so researchers could understand how to equalize the outcomes of a similar campaign in the future.
Rather than just looking at the creative or audience reach, the toolkit aims to take the complexities of the digital ad ecosystem into consideration.
For example, it looks for bias in the processes driving an ad campaign and weighs factors such as the advertiser’s development process for the campaign, the publisher’s methodology for serving ads to its audience and the processes of other parties and machine learning workflows involved in the supply chain.
In some cases, it may also be able to find unconscious bias programmed into AI-driven solutions.
The Ad Council’s “It’s Up To You” COVID-19 campaign, for instance, was geotargeted at liberal- and conservative-leaning areas that were then further divided into age groups. But the predictive model for targeting the ads also took into account education level, gender and income, which is where bias in the modeling became apparent during the study.
Turns out the dynamic creative prediction model more aggressively targeted ads to certain groups, including women, people between the ages of 45 and 65 and those with higher education levels and incomes. This likely resulted in a lower number of conversions among other audience groups, such as those with lower education levels.
Looking back to understand where bias was present in a campaign is the first step toward using AI to build a more inclusive advertising ecosystem. The second step is using AI to help overcome biases lurking within these internal processes.
As part of this study, the researchers sought to transform the predicted probabilities of the campaign’s machine learning predictive models based on imbalances being observed in the targeting criteria.
For groups that were identified as being underserved by the campaign’s targeting, the actual conversion rate was less than .01%. This represented a large imbalance between the number of people who didn’t convert and those who did.
In order to allow the bias mitigation algorithm to learn more about those in underserved groups who did convert, the researchers reduced the number of non-converters so that it nearly equaled the number of converters. Adopting this type of common practice in cases with a heavy data imbalance may make it possible to mitigate bias across multiple disadvantaged groups, according to the study.
This type of mitigation could be used to adjust the targeting of a campaign in future iterations.
“We’ve been playing with mitigation strategies, and we’ve seen some mitigation possibility and the propensity of that mitigation to potentially increase performance, or at least allow a broader audience to have access to the model’s intention of delivering the right message to them,” said Robert Redmond, head of AI ad product design at IBM Watson Advertising. “We’re looking to expand that work and pull more data in so that we can continue to prove out that these biases are present – and that we can deal with them.”
The plan is to expand the analysis that IBM did with the Ad Council to allow for the more actionable examination of campaign data while it’s being run, Redmond said.
To that end, IBM and the 4A’s are issuing a call to action to the ad tech industry to share data that can help efforts to mitigate bias in advertising in the future.
“It’s about getting people to think differently about that campaign journey, about the people who are engaged and what they’re trying to deliver, not just when it comes to DEI for their talent, but how they do business across the enterprise,” said the 4A’s president and CEO Marla Kaplowitz.
With actionable, AI-driven data at its disposal, the ad tech industry can be more empowered to make good on its commitments to diversity, equity and inclusion, Kaplowitz said.
“It has to go beyond intent,” she said. “We need to start seeing the impact, and there has to be real accountability.”
