Home Daily News Roundup Robots.txt Can’t Fight Generative AI; Streamers On The Margins

Robots.txt Can’t Fight Generative AI; Streamers On The Margins

SHARE:
Comic: Room For More?

Here’s today’s AdExchanger.com news round-up… Want it by email? Sign up here.

Pub Crawling

Generative AI is breaking established internet etiquette to satisfy a bottomless appetite for training data.

For example, Microsoft-backed OpenAI and Amazon-supported Anthropic ignore robots.txt to scrape publisher sites without permission, Business Insider reports.

Robots.txt is the standard by which web publishers designate whether and which parts of their sites can be accessed by search engine indexers and other web crawlers. But there’s no legal standard or entity to enforce compliance. 

So OpenAI and Anthropic are apparently choosing to bypass robots.txt and scrape all parts of a given page, according to TollBit, a startup that facilitates paid licensing between publishers and AI firms. 

Both OpenAI and Anthropic publicly claim to respect robots.txt. But generative AI companies, including OpenAI, have argued to regulators that any publicly accessible content on the internet is open to fair use for training AI models.

If generative AI companies ignore robots.txt, might it affect whether ad tech firms honor the standard, too? Contextual data and brand safety vendors could likewise bypass robots.txt protections.

It’s another example of how little control online publishers have over who makes money off their copyrighted content.

The Streaming Squeeze

Entertainment execs are reconciling how dang hard it is to run a profitable streaming business during the upfront negotiations this year.

Streaming subscriber churn is at an all-time high while more platforms are competing for traditional TV ad budgets. 

The general consensus among executives and analysts is that four streaming services survive in the long run. Netflix is a given – it makes 28% margins on streaming operations, which no competitor comes close to, The New York Times reports. Analysts also expect Prime Video and the triumvirate of Disney, Hulu and ESPN to outlast the pack.

Which leaves Peacock, Paramount+ and Warner Bros. Discovery’s Max to fight over the scraps. 

One tip for these three, ironically, is this: Stop trying to be Netflix.

Peacock, for example, only has 34 million subs, but doesn’t need to be all things to all viewers. It’s limiting operations to North America, where it’s most profitable, for instance, says Comcast Chairman Brian Roberts.

Streaming services should also be more cautious with bundling plans, these TV insiders advise, since adding too many discounted sign-ups can painfully lower the total lifetime value of those subscribers.

There’s No Easy Flix

It isn’t all sunshine for Netflix, though.

Netflix’s next step is advertising. It’s had its chance to acquire the majority of potential subscribers; it’s in practically every global market; it cannot raise subscription prices on and on.

But, while Netflix has rock-solid cultural cache and enviable streaming profit margins, it does not have scale on the advertising side. 

In terms of total video ad spend, Netflix trails Roku, Univision and Amazon, not to mention the category leaders: Fox, NBCU, WBD, Paramount, Disney and YouTube, Bloomberg reports. 

One thing Netflix has considered is a free streaming tier with ads for some markets in Asia and Europe, according to Bloomberg. That would make sense, if it can add new pools of users who see ads and at a higher ad rate. (It’s free, after all.) Because Netflix also has self-imposed limitations on its ads business. It brags about its comparatively low ad rate and reluctance to interrupt content. 

Amazon’s video ad sales are low, too. But Amazon has strong industry inroads and seems likelier to scale that business in a big way and fast. 

Ad execs just don’t seem that urgent to be on Netflix. “There is not a clamoring for them,” says one exec.

But Wait, There’s More!

At Cannes, publishers attempt to quell marketers’ open programmatic apostasy. [Digiday]

How Amazon is using Amazon Outlet to keep shoppers around with discounts. [Business Insider]

How streaming services are faring in their fight against subscriber churn. [Marketing Brew]

Forget privacy, young internet users want to be tracked. [Financial Times]

BuzzFeed wants to sell First We Feast, which makes the YouTube show “Hot Ones,” for $70 million. But it’s struggling to find a buyer. [Bloomberg]

X CEO Linda Yaccarino fired her right-hand man Joe Benarroch, partly for failing to warn clients about the platform changing its adult content policy and partly due to pressure from X owner Elon Musk to cut costs. [Financial Times]

Must Read

The Arena Group's Stephanie Mazzamaro (left) chats with ad tech consultant Addy Atienza at AdMonsters' Sell Side Summit Austin.

For Publishers, AI Gives Monetizable Data Insight But Takes Away Traffic

Traffic-starved publishers are hopeful that their long-undervalued audience data will fuel advertising’s automated future – if only they can finally wrest control of the industry narrative away from ad tech middlemen.

Q3: The Trade Desk Delivers On Financials, But Is Its Vision Fact Or Fantasy?

The Trade Desk posted solid Q3 results on Thursday, with $739 million in revenue, up 18% year over year. But the main narrative for TTD this year is less about the numbers and more about optics and competitive dynamics.

Comic: He Sees You When You're Streaming

IP Address Match Rates Are a Joke – And It’s No Laughing Matter

According to a new report, IP-to-email matches are accurate just 16% of the time on average, while IP-to-postal matches are accurate only 13% of the time. (Oof.)

Privacy! Commerce! Connected TV! Read all about it. Subscribe to AdExchanger Newsletters
Comic: Gamechanger (Google lost the DOJ's search antitrust case)

The DOJ And Google Sharpen Their Remedy Proposals As The Two Sides Prepare For Closing Arguments

The phrase “caution is key” has become a totem of the new age in US antitrust regulation. It was cited this week by both the DOJ and Google in support of opposing views on a possible divestiture of Google’s sell-side ad exchange.

create a network of points with nodes and connections, plain white background; use variations of green and grey for the dots and the connctions; 85% empty space

Alt Identity Provider ID5 Buys TrueData, Marking Its First-Ever Acquisition

ID5 bought TrueData mainly to tackle what ID5 CEO Mathieu Roche calls the “massive fragmentation” of digital identity, which is a problem on the user side and the provider side.

CTV Manufacturers Have A New Tool For Catching Spoofed Devices

The IAB Tech Lab’s new device attestation feature for its Open Measurement SDK provides a scaled way for original device manufacturers to confirm that ad impressions are associated with real devices.