Home Daily News Roundup Robots.txt Can’t Fight Generative AI; Streamers On The Margins

Robots.txt Can’t Fight Generative AI; Streamers On The Margins

SHARE:
Comic: Room For More?

Here’s today’s AdExchanger.com news round-up… Want it by email? Sign up here.

Pub Crawling

Generative AI is breaking established internet etiquette to satisfy a bottomless appetite for training data.

For example, Microsoft-backed OpenAI and Amazon-supported Anthropic ignore robots.txt to scrape publisher sites without permission, Business Insider reports.

Robots.txt is the standard by which web publishers designate whether and which parts of their sites can be accessed by search engine indexers and other web crawlers. But there’s no legal standard or entity to enforce compliance. 

So OpenAI and Anthropic are apparently choosing to bypass robots.txt and scrape all parts of a given page, according to TollBit, a startup that facilitates paid licensing between publishers and AI firms. 

Both OpenAI and Anthropic publicly claim to respect robots.txt. But generative AI companies, including OpenAI, have argued to regulators that any publicly accessible content on the internet is open to fair use for training AI models.

If generative AI companies ignore robots.txt, might it affect whether ad tech firms honor the standard, too? Contextual data and brand safety vendors could likewise bypass robots.txt protections.

It’s another example of how little control online publishers have over who makes money off their copyrighted content.

The Streaming Squeeze

Entertainment execs are reconciling how dang hard it is to run a profitable streaming business during the upfront negotiations this year.

Streaming subscriber churn is at an all-time high while more platforms are competing for traditional TV ad budgets. 

The general consensus among executives and analysts is that four streaming services survive in the long run. Netflix is a given – it makes 28% margins on streaming operations, which no competitor comes close to, The New York Times reports. Analysts also expect Prime Video and the triumvirate of Disney, Hulu and ESPN to outlast the pack.

Which leaves Peacock, Paramount+ and Warner Bros. Discovery’s Max to fight over the scraps. 

One tip for these three, ironically, is this: Stop trying to be Netflix.

Peacock, for example, only has 34 million subs, but doesn’t need to be all things to all viewers. It’s limiting operations to North America, where it’s most profitable, for instance, says Comcast Chairman Brian Roberts.

Streaming services should also be more cautious with bundling plans, these TV insiders advise, since adding too many discounted sign-ups can painfully lower the total lifetime value of those subscribers.

There’s No Easy Flix

It isn’t all sunshine for Netflix, though.

Netflix’s next step is advertising. It’s had its chance to acquire the majority of potential subscribers; it’s in practically every global market; it cannot raise subscription prices on and on.

But, while Netflix has rock-solid cultural cache and enviable streaming profit margins, it does not have scale on the advertising side. 

In terms of total video ad spend, Netflix trails Roku, Univision and Amazon, not to mention the category leaders: Fox, NBCU, WBD, Paramount, Disney and YouTube, Bloomberg reports. 

One thing Netflix has considered is a free streaming tier with ads for some markets in Asia and Europe, according to Bloomberg. That would make sense, if it can add new pools of users who see ads and at a higher ad rate. (It’s free, after all.) Because Netflix also has self-imposed limitations on its ads business. It brags about its comparatively low ad rate and reluctance to interrupt content. 

Amazon’s video ad sales are low, too. But Amazon has strong industry inroads and seems likelier to scale that business in a big way and fast. 

Ad execs just don’t seem that urgent to be on Netflix. “There is not a clamoring for them,” says one exec.

But Wait, There’s More!

At Cannes, publishers attempt to quell marketers’ open programmatic apostasy. [Digiday]

How Amazon is using Amazon Outlet to keep shoppers around with discounts. [Business Insider]

How streaming services are faring in their fight against subscriber churn. [Marketing Brew]

Forget privacy, young internet users want to be tracked. [Financial Times]

BuzzFeed wants to sell First We Feast, which makes the YouTube show “Hot Ones,” for $70 million. But it’s struggling to find a buyer. [Bloomberg]

X CEO Linda Yaccarino fired her right-hand man Joe Benarroch, partly for failing to warn clients about the platform changing its adult content policy and partly due to pressure from X owner Elon Musk to cut costs. [Financial Times]

Must Read

Single continuous line drawing robot riding pencil rocket flying in the sky. Tech education and creativity. Future technology development. Artificial intelligence. One line design vector illustration

Q3: Omnicom’s Media Business Is Growing, But Is It Agent-Proof?

If AI agents can plan and buy media, who needs a holding company? Good question. It came up during Omnicom’s Q3 earnings on Thursday.

Netflix Touts New Measurement Capabilities As It Struggles With “Modest” Engagement Growth

Netflix’s revenue grew 12% YOY to $12.9 billion in Q3, which it maintains is in line with its forecast. Its ads business, however, is lagging slightly compared with expectations, causing Netflix to tweak its forecast for the rest of the year.

Apple Publishes Its Plan To Wipe Out Web Tracking

Apple is coming for identity resolution. WebKit’s latest tracking crackdown targets the infrastructure behind post-cookie IDs – and switching to a first-party setup won’t necessarily spare vendors.

Privacy! Commerce! Connected TV! Read all about it. Subscribe to AdExchanger Newsletters

AI Slop Sites Are On The Rise. Is Ad Tech Ready For Its New MFA Challenge?

Advertising supply chain intelligence company DecryptAds uncovered more than 2,100 AI-generated recipe sites that are monetized with programmatic ads and feature fabricated author profiles.

How Australian Home Lender Lendi Measures The Invisible Work Of AI

Most brands and businesses are trying to figure out how to improve their metrics when it comes to AI – and also what those metrics even are.

AI Startup Minerva Built An Agentic CDP, But Don’t Call It That

Minerva wants to turn fragmented customer data into something marketers can use, with AI agents doing the work behind the scenes.