“Data-Driven Thinking” is written by members of the media community and contains fresh ideas on the digital revolution in media.
Today’s column is written by Jonathan Lenaghan, head of data science at PlaceIQ.
Warnings of the coming Skynet-ization of digital advertising are becoming increasingly common. But rest assured, the near-term future of our industry is not going to be filled with self-aware, artificially intelligent machines that will replace all humans currently employed at ad tech companies.
However, digital advertising does seem to be on the cusp of a significant transformation, in the form of a rapid emergence of platform-centric ecosystems. The number of ad tech companies announcing the launch of a new platform seems to grow daily. These systems will be significantly more feature-rich than the real-time bidding or large Hadoop-based back-end platforms that have defined the industry for the past several years.
To unlock real value, though, these platforms need to enable business analysts, data scientists, campaign managers and an entire host of operations personnel. Many ad tech businesses rely on ingesting, processing and analyzing hundreds of terabytes of data coming from varied and disparate sources. Traditionally, large teams of engineers toting extensive experience within the Hadoop ecosystem were necessary to get actionable insights.
The next generation of platforms will still perform these functions, but aggregations, algorithms, internal languages and interactive visualization layers will empower this larger family of end users to better define and segment audiences, optimize campaigns based upon industry-specific KPIs or slice and pivot campaign data along many new dimensions. These platforms will no longer be under the exclusive purview of data teams but will be pushed deeper into organizations to those with perhaps less technical experience in big data but with deeper domain experience.
Checkmate
If the focus in the past decade has been to capture and process enormous amounts of data, the next step is to design platforms that strip away this complexity and scale and seamlessly incorporate the expertise of analysts and operations personnel. The emerging platform ecosystem will augment the intelligence of analysts and enable them to effortlessly make business decisions.
Chess is an excellent example of algorithms augmenting the intelligence of a human being. Ever since Deep Blue beat Gary Kasparov in 1997, computers and algorithms have been able to beat the best human grandmasters. The most formidable chess playing system, however, is a combination of top chess programs and human grandmasters. The sum of algorithms and human beings is greater than either individually.
Algorithmic Black Boxes
The term “platform” has a tendency to evoke images of the purely algorithmic black boxes that dominate the high-frequency equity-trading world. Similarly, bidding on ad inventory will always be algorithmic with little to no direct human interaction. Targeting and serving a digital ad needs to happen in a matter of milliseconds, and so it makes sense that on the surface much of the digital advertising ecosystem is loosely modeled after equity markets.
Algorithmic black boxes, however, are really only successful when they exploit time and capacity scales with a very narrow and specific purpose, such as optimally bidding on ad inventory with sub-millisecond latencies or taking advantage of tiny price discrepancies across multiple stock exchanges. The coming era where an analyst or data scientist is going to be replaced by a black box is a long ways off.
The R Project for Statistical Computing and other statistical packages, for example, have been around for many years but as a general population, we do not seem any better at understanding statistical concepts. Even companies that specialize in black box optimization utilize teams of analysts and data scientists to identify and implement optimization strategies. Those that do not rely on human intuition and experience, I conjecture, are doing a lot of optimization towards click and impression fraud.
The Power Of Deep Domain Expertise
To be sure, I am no Luddite. I have spent my career with machines, models and algorithms, and the greatest business leverage is found by combining the analyst with the algorithm. There is a common saying among data scientists in which bigger data beats better algorithms. I posit that deep domain understanding beats both.
Given the choice between doubling my data size, spending a few months investigating more sophisticated algorithms, or incorporating the work and expertise of a knowledgeable analyst into my platform, I’ll take the human being. Rules and heuristics defined by experts have more utility and can be implemented more quickly and efficiently that building fully automated systems that learn a domain.
This model has been very successful for companies like Palantir or Quid. The Palantir platform works “at the intersection of data, technology and human expertise” to yield actionable results for governments, as well as businesses. Quid, likewise, has built a platform to ingest large amounts of unstructured data to provide analysts with a means of interrogating complex relationships. In these platforms, data and algorithms are used to leverage human experience and intuition.
At the end of the day, the role of domain expertise will tend to outweigh both the sophistication of methodologies and access to more data. The next “Rise of the Machines” will aid analysts and managers, rather than replace them.
Follow PlaceIQ (@PlaceIQ) and AdExchanger (@adexchanger) on Twitter.
