‘Twas the fight before Christmas?
Now that Nielsen has completely switched over to its brand new Big Data + Panel offering, the Video Advertising Bureau (VAB) isn’t happy with how the company’s fall TV ratings are turning out.
The industry org is accusing Nielsen’s Big Data + Panel of producing “instability, inconsistency and irrational” TV audience measurement results, particularly when compared to its legacy panel-only product.
The report documents large swings in audience numbers over a month-long period this fall when the VAB compared Nielsen’s newer Big Data + Panel (BD+P) with its older Panel Only (PO) ratings, which Nielsen plans to no longer offer as a standalone product as of this year.
When asked for comment, a Nielsen spokesperson had this to say: “This report is seriously flawed and manipulated. From what we have seen, the VAB incorrectly pulled our data and the bureau does not know how to do a proper ratings analysis.”
The report
The VAB looked at audience data spanning 33 broadcast and cable networks and six demographic audiences – specifically, HH (households), P2+ (persons over the age of 2), P18-34, P18-49, P25-54 and P55+ – as captured over a four-week period from September 15 through October 12.
According to the VAB, the data suggests that between 48% and 58% of total hours during that time had over a 20% audience variance across several key buying demographics.
In other words, for almost half of television hours watched over a four-week period, the BD+P data reported ratings that were 20% higher or 20% lower than the PO data did.
These audience variances were especially extreme for 18-34 year olds. Almost one third of total hours varied by 50%, and one in 10 varied by at least 75%.
In a press briefing to discuss the report’s findings, VAB CEO and President Sean Cunningham said that these discrepancies can’t be easily explained away by changes in viewing behavior.
Even if younger audiences are tuning into broadcast and cable channels less frequently than they once did, the numbers are “still representative of millions upon millions of data points” and shouldn’t swing quite so wildly on an hour-by-hour basis, said Cunningham.
Likewise, although common sense would suggest that NFL audiences are more stable given the sport’s popularity and audience loyalty, the VAB frequently found evidence of double-digit variances across the 30 games it analyzed.
The report even suggests there were higher concentrations of negative variances in higher-rated broadcasts, meaning that the most-watched games were actually more likely to be underreported in the BD+P data.
However, Nielsen’s spokesperson pushed back against this assertion as well, claiming that the VAB “pulled data for live sports without accounting for the same games airing in different time zones across the US.”
“The VAB is wasting the time and money of its members,” they added.
What it means
Part of the reason these findings are so noteworthy to the VAB, said Cunningham, is that roughly 80% of Nielsen’s Big Data + Panel audience count is still driven by the original Panel Only audience – suggesting that these big fluctuations are being driven by issues in how the data points are extrapolated and modeled outward.
If that tune sounds familiar, it’s because the VAB has been playing it for a while. The organization has expressed similar concerns about Nielsen’s Household Demographic Assessment Model (HDAM), which came to a head earlier this year during a meeting with Nielsen and other publisher and agency leaders.
It’s also worth pointing out that the VAB’s report is based on live-only viewing, not Nielsen’s more typically cited live plus same day (Live + SD) ratings. The report also does not include digital in TV ratings (DVTR) or OOH, which Nielsen has previously argued is a critical component of complete, currency quality measurement.
In any case, if TV audiences cannot be accurately and predictably counted, said Cunningham, then that poses a serious problem for an industry where financial deals and campaign goals are so often based on audience guarantees.
After all, if an advertiser expects their commercial to reach 20 million people and it doesn’t, why would they want to spend money on another one?
