Bots are to web traffic what broken turnstiles are to amusement parks – they make it impossible to get an accurate read on how many people are coming through the door, and that messes with optimization.
But at least in the case of a physical location, the visitors are human.
On Thursday, bot detection company Distil Networks announced it had closed its acquisition of Are You A Human, a Detroit-based digital security and human traffic verification company, in a bid to help publishers and websites rid their properties of bots without killing the experience for real users.
Distil declined to disclose the deal price of an acquisition that has all the markings of a classic acqui-hire. All 10 of Are You A Human’s employees will join Distil’s 200, and Distil will open an office in Detroit with plans to hire more engineering talent.
“We have data on who is a bot and who is not on more than 1 million sites,” said Reid Tatoris, Are You A Human’s co-founder and chief product officer, whose new title is still being ironed out.
“We’re combining that with Distil’s bot data so sites can have one version of the truth. That is the main purpose of the acquisition.”
The Distil and Are You Human technologies and skill sets are different but complementary, said Elias Terman, Distil’s VP of marketing.
Are You A Human analyzes how real users interact with a website and adds them to a whitelist of verified humans. Distil Networks attacks the bot problem by using machine learning to proactively intercept fishy-looking traffic and present it with a sort of Turing test for bots, such as an email validation or chat window.
“It’s so hard to find top engineering talent, especially people with a depth of experience fighting the bot problem,” Terman said. “Now we’ve got a brain trust of some of the smartest people who’ve been doing this the longest.”
The hookup is already starting to bear fruit with the release of the duo’s first product together, a free bot discovery plugin for Google Analytics that uses Are You A Human’s JavaScript tag to overlay bot and human data onto site reporting for a truer view of the real audience minus bots.
Publishers can then strip out the offending traffic.
Around 40% of all website traffic is generated by bots, according to Distil. Half of those are legit bots, including the web crawlers that index web pages for search engines, and the other half are nefarious little dudes that scrape content, commit ad fraud and skew analytics counts, among other crimes.
By Distil’s estimates, Google’s built-in bot filtering mechanism, which relies on the Interactive Advertising Bureau’s international spiders and robots list, catches less than 1% of the bots Distil sees in the wild.
“It’s just a drop in the bucket,” Tatoris said.
Without insight into how much of their traffic is coming from bots, site owners can’t properly measure their campaigns, run credible A/B tests or optimize their onsite experience.
“Say you put a new button on your site and there’s a bot programmed to hit the button,” Tatoris said. “You’re going to end up optimizing your site based on bot traffic. If you don’t have the right information, you can’t make the right decisions.”
Publishers also need to know more about why users – or bots – behave as they do. It’s often difficult to discern the difference between a human and a bot by simply observing mouse movements. For example, an authentic human swiping down on a mobile device creates what can look like telltale bot activity, even though it’s completely innocent.
Are You A Human had been shopping for a buyer since at least February, and data management platform Lotame was one of its serious suitors. But that deal fell through and Are You A Human ended up partnering with Lotame instead to integrate bot detection and traffic quality measurement into its stack so that customers can create, buy and sell human-verified segments.
Founded in 2011, Are You A Human raised nearly $8 million in funding, most recently a $4.2 million Series A in 2015. In addition to the new office it plans to open in Are You A Human’s hometown of Detroit, Distil has locations in San Francisco, Raleigh, N.C., Arlington, Va., the UK and Stockholm.
