web crawlers

Reuters wraps its series on Meta’s shrinking headcount; The Trade Desk has a new centralized AI interface; and USA Today Co. will restructure its content for bots.

The newly introduced Stealth Bot Prohibition Act would prevent AI stealth crawlers from hiding their identities and illicitly scraping web content.

The IAB Tech Lab’s new guidance for bot management encourages content owners to figure out which bots are worth their time.

Cloudflare restricts how bots can scrape content; TiVo’s customer base stays loyal, even as TiVo leaves the DVR business; and Nestlé announces a stark reduction in headcount.

From one perspective, publishers are up a creek thanks to the rise of generative AI search; the impact on discoverability and traffic is palpable. But that doesn’t mean publishers can’t adapt and find new ways to make money, says Mediavine CRO Amanda Martin.

The growth of Walmart’s ads business and third-party marketplace are separate, it says. Plus, Google and Meta’ve been targeting teens for a while.