Large marketers are constantly running scores of experiments on their media investments. That can make optimization tricky, said Sadie Daryan, eBay’s global head of display and app marketing.
And so eBay has been playing around with a tool from marketing intelligence startup Admetrics, released Tuesday, that uses AI to automatically run thousands of concurrent tests and make recommendations on how to best tweak them in real time.
The tool, called Newton (because it’s seemingly impossible to not name an AI-related marketing solution after anyone other than a brilliant male scientist or mathematician), is trained to understand specific marketing use cases, said Markus Repetschnig, CEO and co-founder of Admetrics.
There’s a Newton model for optimizing Facebook campaigns, for example, and another for programmatic advertising. Marketers set up their tests within Admetrics, including their desired KPI, and Newton monitors the tests in real time and makes suggestions on the fly to improve results.
Newton may recommend shifting spend from one channel to another or changing targeting parameters to increase ROI. For a performance campaign, Newton will try to suss out the most efficient cost per click; for a branding campaign, it may advise funneling traffic from one creative variation to another based on how well it’s working.
If it looks like something’s going wrong with a campaign, the system automatically sends out an alert with tips on how to fix the problem, which is better than making a mistake and burning through budget because you didn’t deal with it quickly enough, Repetschnig said.
Over time, Newton also learns from what advertisers tell it through a feedback loop. Advertisers can mark whether a recommendation was good or not, which refines the algorithm.
The idea, Repetschnig said, is to enable “always-on experimentation” that allows marketers take a more exploratory approach to testing rather than the classic confirmatory approach of proving or disproving a hypothesis, which is time consuming, costly, requires a manual set up and takes a lot of data to reach statistical significance.
“Marketers know how important it is to run tests, but even some of our most advanced clients struggle to set up a real culture of experimentation,” he said.
Testing gets even more complicated at scale. Take eBay, which has myriad and near-constant tests on the boil up and down the customer funnel, targeting everyone from prospects at the top to known users at the bottom.
“Add in seasonality, sales events, the data science models we use to recommend different items – that all creates a matrix of campaigns running at any given point across the globe,” said eBay’s Daryan.
Ebay’s in-house team of data scientists has been using Newton in beta to “accelerate our decision-making process,” Daryan said.
“When you’re doing concrete A/B tests, you have to run them for two to three weeks at a time, and then you have to wait, which is what we did before,” she said. “Now we’re just moving much more quickly.”
