Criteo’s FLoC Tests Confirm There’s Way More Work Before It’s Ready For Prime Time

Based on extensive testing conducted by Criteo during the first set of public FLoC origin trials over the summer it's clear that more testing is needed.

With your permission, please endure yet one more Privacy Sandbox-related bird reference: Chrome’s Federated Learning of Cohorts (FLoC) proposal will not fly in its current form.

“We’ve got a long road to hoe before it’s possible to prove this can be a working model for advertising,” said Todd Parsons, chief product officer at Criteo.

The purpose of FLoCs is to enable ad targeting and prospecting by using on-device machine learning to group people according to their common browsing behavior.

Based on extensive testing conducted by Criteo during the first set of public FLoC origin trials over the summer – as extensive as possible considering the low volume of FLoC IDs available on programmatic exchanges – one thing is clear, which is that nothing is clear.

More testing is called for before FLoCs can be considered a viable solution for the post-cookie world.

Testing 1, 2, 3

There are three main issues with FLoCs as they stand and with the testing that’s been conducted thus far.

One, the experiments did not yield proof of performance anywhere near that of third-party cookie-based solutions – and definitely nothing near Google’s claim that “advertisers can expect to see at least 95% of the conversions per dollar spent when compared to cookie-based advertising.”

Two, cohorts based on the domains a person recently visited are not necessarily predictive of future conversion behavior. And although Google’s own research found that the FLoC API achieves better precision than a random cohort assignment, that baseline is an irrelevant measure of success since marketers don’t spend blindly or at random.

And three, FLoCs may not be any more privacy preserving than the current system. It’s still possible to do cross-site tracking using FLoC IDs by observing browsing habits and cohort assignments over time, for example, and FLoCs don’t address fingerprinting or sensitive category targeting. There’s nothing stopping advertisers from grouping people together based on their sexual orientation or ethnicity, tied to the content they browse.

Known unknowns

But beyond performance issues and privacy concerns, one of the biggest unanswered questions about FLoC has to do with its economic model.

Criteo has observed that there’s an inherent imbalance between the value a smaller publisher can bring to a FLoC compared with a large platform-cum-publisher. Visiting a niche recipe blog or a gaming site, for instance, tells marketers a lot more about an audience than would a visit to the main or domain. But the larger platforms benefit more from FLoCs.

By the same token, it’s not clear how exactly FLoC IDs are composed or what sort of user interests they contain.

Because they’re built upon top-level domains, FLoCs as they’re currently devised aren’t granular enough to create the commerce-related audience segments that are most valuable to marketers. A visit to could mean someone’s interested in back-to-school clothes, orange juice, sneakers, cat food, fishing gear – who knows.

“Even the same person will have widely varied interests; they might be interested in different things at different times of day within different contexts,” Parsons said. “That makes it a very hard math problem to try and assign a single attribute to a browser in a way that’s useful to marketers.”

Just getting started

So, what’s next?

This first phase of public FLoC origin testing ran between late March and mid-July with a small number of users in the US, Australia, Brazil, Canada, Indonesia, Japan, New Mexico, New Zealand and Philippines. Europe was notably absent from that list due to concerns about GDPR compliance, and FLoC testing has still not been conducted in the European market.

The next set of FLoC testing, as well as origin trials of the FLEDGE proposal – the one meant to enable remarketing without reliance on third-party cookies – were both set to kick off next quarter.

But Google recently pushed back the testing timeline for both the next phase of FLoC and the first phase of FLEDGE to Q1 2022.

Some might find these delays frustrating, but Parsons finds them encouraging.

“It’s better for us all to take our sweet time,” he said. “Then everybody has a better chance of winning together.”

Parsons is also reassured by the fact that the Chrome team recently reached out to Criteo proactively to set up a meeting.

“We’ll keep an open line of communication and see where we go from here,” he said. “I think Google realizes it needs to keep working on this thing.”

For more on the first phase of FLoC origin trials, Criteo chronicled its own testing process in a series of blog posts:

Enjoying this content?

Sign up to be an AdExchanger Member today and get unlimited access to articles like this, plus proprietary data and research, conference discounts, on-demand access to event content, and more!

Join Today!