Get alerts on this situation
We’ll email you as it develops, and you can follow the whole thread from day one.
Unsubscribe anytime.
[SITUATION] · [QUIET]
19 clusters · 73 sources · 52 days · First seen · Last updated
Categories: TECHNOLOGY · BUSINESS
AI bots dominate web traffic, toll‑booth charging expands
Entities: HTTP 402 status code · Cloudflare, Inc. · Amazon Web Services, Inc. · Web publishers · Artificial Intelligence bots
Overview
Cloudflare’s July 4, 2024 briefing revealed AI‑driven bots were responsible for roughly 57 % of HTML requests, prompting a Pay‑Per‑Use policy that began charging mixed‑use crawlers on ad‑supported pages on 1 July and default‑blocking AI‑training bots on ad‑carrying pages from 15 September 2024. Publishers quickly adopted these controls; in July 2026 Patreon partnered with Cloudflare to block AI‑training crawlers from scraping creator content while still allowing search‑engine bots.
Building on that framework, Cloudflare announced in July 2025 a toll‑booth model that leverages the long‑dormant HTTP 402 status code. Site owners can specify which AI agents may access their pages and the fee each must pay, targeting the majority of crawlers used for large‑scale model training while permitting benign indexing. On 15 June 2026 Amazon Web Services incorporated the same capability into its firewall service, offering a comparable “toll‑booth” for AI traffic.
These developments deepen the technical and commercial toolkit available to publishers, shifting part of the economic tug‑of‑war from ad‑based revenue toward direct monetisation of AI‑generated traffic and consent‑based extraction.
Timeline
-
10 days ago
[TECHNOLOGY] 4 sourcesCloudflare and AWS launch AI bot toll‑booth charging modelCloudflare (July 2025) and AWS (June 2026) introduced AI‑bot toll‑booth fees, using HTTP 402 to let sites charge AI crawlers for access.
-
17 days ago
[TECHNOLOGY] 6 sourcesPatreon blocks AI training bots with Cloudflare partnershipPatreon, together with Cloudflare, has blocked AI‑training crawlers, cutting weekly bot attempts from thousands to zero while keeping search‑engine bots active.
-
about 1 month ago
[TECHNOLOGY] 4 sourcesCloudflare to Block AI Training Crawlers on Ad Pages, Giving Publishers New LeverageCloudflare will block AI training crawlers on ad‑supported pages from Sept 15, while Meta rolls out an AI ad generator that could flood the web with synthetic content, raising concerns about model collapse.
-
about 1 month ago
[TECHNOLOGY] 2 sourcesBots now account for the majority of web traffic, Cloudflare data showsCloudflare says bots now make up 57 % of web traffic, outpacing humans. AI agents and automated tools are reshaping online content consumption and ad revenue models.
-
about 1 month ago
[TECHNOLOGY] 8 sourcesCloudflare restricts AI crawlers as AI reshapes web, journalism and SEOCloudflare will block mixed‑use AI crawlers, while AI reshapes search, journalism and SEO, sparking debate on control and accountability.
-
about 1 month ago
[TECHNOLOGY] 2 sourcesCloudflare to Block Mixed‑Use AI Crawlers on Ad‑Supported Pages Starting Sep 2026From 15 Sept 2026 Cloudflare will block mixed‑use AI crawlers on ad‑pages by default, keeping search‑only bots allowed, and is rolling out a Pay‑Per‑Use payout model for AI‑generated content.
-
about 1 month ago
[TECHNOLOGY] 5 sourcesCloudflare mandates AI crawlers to pay for publisher contentCloudflare will block mixed‑use AI crawlers from ad‑hosting pages starting 15 Sept 2026, offering a Pay‑Per‑Use model that lets publishers charge AI firms for content use.
-
about 1 month ago
[BUSINESS] 15 sourcesVisa launches live AI‑agent purchases across Europe with banks and fintech partnersVisa, together with European banks, fintech Nuvei, BBVA and Hungary’s K&H, has demonstrated live AI‑agent purchases that select, order and pay for goods using Visa Intelligent Commerce and Mastercard Agent Pay,
-
about 1 month ago
[TECHNOLOGY] 2 sourcesAI models uncover tax loopholes and become primary users of the internetAI can be trained to spot and exploit regulatory loopholes, while bots and AI agents now generate a large share of internet traffic, shifting information search to machines and risking content quality.
-
about 2 months ago
[TECHNOLOGY] 2 sourcesAI agents generate majority of global internet traffic, overtaking human usersCloudflare data shows AI agents now generate 57.4 % of global web traffic, surpassing human users and prompting security and infrastructure concerns.
-
about 2 months ago
[TECHNOLOGY] 2 sourcesFastly reports AI-generated web traffic grows 6.5 times faster than human trafficFastly finds AI web traffic rose 30 % Jan‑May 2026, growing 6.5 × faster than human traffic; 51 % of AI requests hit origin servers versus <9 % for humans, with Claude‑related traffic up 555 %.
-
about 2 months ago
[TECHNOLOGY] 4 sourcesCloudflare Radar data shows bots outpacing humans in web trafficCloudflare Radar shows bots exceed human traffic in HTML‑only views, but across all content types humans dominate, highlighting filter‑driven misinterpretations.
-
about 2 months ago
[TECHNOLOGY] 2 sourcesCloudflare CEO says AI bots now drive over half of internet trafficAI bots have overtaken humans, generating about 57 % of global web traffic, Cloudflare CEO Matthew Prince said, earlier than expected.
-
about 2 months ago
[TECHNOLOGY] 2 sourcesCloudflare reports AI bots now drive majority of website trafficCloudflare says AI bots now account for over half of web traffic, while a German blogger details a personal AI workflow for smart‑home automation.
-
about 2 months ago
[TECHNOLOGY] 2 sourcesAI traffic now dominates internet, surpassing human usersCloudflare data shows AI bots now generate 57.4% of internet traffic, overtaking human users and challenging ad‑based revenue models and the authenticity of online discourse.
-
2 months ago
[TECHNOLOGY] 2 sourcesCloudflare says AI bots now generate majority of worldwide web page requestsCloudflare reports AI bots now account for about 57% of global webpage requests, surpassing human traffic for the first time, signalling a shift toward agent‑driven browsing.
-
2 months ago
[TECHNOLOGY] 3 sourcesCloudflare reports AI agents now generate majority of internet trafficCloudflare says AI agents now make up 57.4% of web traffic, overtaking humans at 42.6%, after a 1,500% surge in 2025 driven by chatbot‑powered browsing.
-
2 months ago
[TECHNOLOGY] 5 sourcesAI agents now generate most global web trafficCloudflare data shows AI agents now account for 57.4% of global web traffic, overtaking humans and prompting debates on advertising models and the “dead internet theory.”
-
2 months ago
[TECHNOLOGY] 5 sourcesCloudflare reports AI bots now generate the majority of web trafficCloudflare says AI bots now account for 57% of web traffic, overtaking humans earlier than expected, and discusses a pay‑to‑crawl model amid industry reaction.
Sources
24matins.es · agrigentooggi.it · ainalnaql.com · altonivel.com.mx · b-und-i.de · basilio.com.br · begeek.fr · berlin-live.de · blogspan.net · bohos.blog.sme.sk · borncity.com · breitbart.com · diariosigloxxi.com · digitalmarketreports.com · digitalphablet.com · eoldal.hu · es.gizmodo.com · examiner.co.ug · expressindia.com · financialit.net · fourweekmba.com · freundin.de · fstoppers.com · gamereactor.nl · gazeta-lokalna.pl · gfm-nachrichten.de · gradschool.umn.edu · hirek.prim.hu · hobbyblogging.de · hwupgrade.it · ictmagazine.nl · ilariogobbi.it · independentmediacenter.org · ipadizate.es · it-daily.net · it-finanzmagazin.de · ithome.com · joseantoniomarina.net · josefacchin.com · koszykorama.pl · kronikaonline.ro · lavozdemichoacan.com.mx · lightnet.co.uk · luxfind.lu · memeburn.com · muycomputerpro.com · napi.hu · news.sk