# AI bots dominate web traffic, toll‑booth charging expands > Live situation record from CLSTR: https://clstr.news/situations/ai-bots-dominate-web-traffic > Updated: 2026-07-25T20:34:49.000Z. Sources: 73. Developments: 19. Cloudflare’s July 4, 2024 briefing revealed AI‑driven bots were responsible for roughly 57 % of HTML requests, prompting a Pay‑Per‑Use policy that began charging mixed‑use crawlers on ad‑supported pages on 1 July and default‑blocking AI‑training bots on ad‑carrying pages from 15 September 2024. Publishers quickly adopted these controls; in July 2026 Patreon partnered with Cloudflare to block AI‑training crawlers from scraping creator content while still allowing search‑engine bots. Building on that framework, Cloudflare announced in July 2025 a toll‑booth model that leverages the long‑dormant HTTP 402 status code. Site owners can specify which AI agents may access their pages and the fee each must pay, targeting the majority of crawlers used for large‑scale model training while permitting benign indexing. On 15 June 2026 Amazon Web Services incorporated the same capability into its firewall service, offering a comparable “toll‑booth” for AI traffic. These developments deepen the technical and commercial toolkit available to publishers, shifting part of the economic tug‑of‑war from ad‑based revenue toward direct monetisation of AI‑generated traffic and consent‑based extraction. ## Timeline ### 2026-07-25: Cloudflare and AWS launch AI bot toll‑booth charging model Cloudflare (July 2025) and AWS (June 2026) introduced AI‑bot toll‑booth fees, using HTTP 402 to let sites charge AI crawlers for access. 4 sources. https://clstr.news/cluster/cloudflare-and-aws-launch-ai-bot-tollbooth-charging-model ### 2026-07-18: Patreon blocks AI training bots with Cloudflare partnership Patreon, together with Cloudflare, has blocked AI‑training crawlers, cutting weekly bot attempts from thousands to zero while keeping search‑engine bots active. 6 sources. https://clstr.news/cluster/patreon-blocks-ai-training-bots-with-cloudflare-partnership ### 2026-07-05: Cloudflare to Block AI Training Crawlers on Ad Pages, Giving Publishers New Leverage Cloudflare will block AI training crawlers on ad‑supported pages from Sept 15, while Meta rolls out an AI ad generator that could flood the web with synthetic content, raising concerns about model collapse. 4 sources. https://clstr.news/cluster/cloudflare-to-block-ai-training-crawlers-on-ad-pages-giving-publishers-new-leverage ### 2026-07-04: Bots now account for the majority of web traffic, Cloudflare data shows Cloudflare says bots now make up 57 % of web traffic, outpacing humans. AI agents and automated tools are reshaping online content consumption and ad revenue models. 2 sources. https://clstr.news/cluster/bots-now-account-for-the-majority-of-web-traffic-cloudflare-data-shows ### 2026-07-02: Cloudflare restricts AI crawlers as AI reshapes web, journalism and SEO Cloudflare will block mixed‑use AI crawlers, while AI reshapes search, journalism and SEO, sparking debate on control and accountability. 8 sources. https://clstr.news/cluster/artificial-intelligences-impact-on-freedom-and-cinema-explored ### 2026-07-02: Cloudflare to Block Mixed‑Use AI Crawlers on Ad‑Supported Pages Starting Sep 2026 From 15 Sept 2026 Cloudflare will block mixed‑use AI crawlers on ad‑pages by default, keeping search‑only bots allowed, and is rolling out a Pay‑Per‑Use payout model for AI‑generated content. 2 sources. https://clstr.news/cluster/cloudflare-to-block-mixeduse-ai-crawlers-on-adsupported-pages-starting-sep-2026 ### 2026-07-01: Cloudflare mandates AI crawlers to pay for publisher content Cloudflare will block mixed‑use AI crawlers from ad‑hosting pages starting 15 Sept 2026, offering a Pay‑Per‑Use model that lets publishers charge AI firms for content use. 5 sources. https://clstr.news/cluster/cloudflare-mandates-ai-crawlers-to-pay-for-publisher-content ### 2026-07-01: Visa launches live AI‑agent purchases across Europe with banks and fintech partners Visa, together with European banks, fintech Nuvei, BBVA and Hungary’s K&H, has demonstrated live AI‑agent purchases that select, order and pay for goods using Visa Intelligent Commerce and Mastercard Agent Pay, 15 sources. https://clstr.news/cluster/ai-agents-face-consumer-scepticism-as-cloudflare-moves-to-charge-ai-tools-for-web-content ### 2026-06-28: AI models uncover tax loopholes and become primary users of the internet AI can be trained to spot and exploit regulatory loopholes, while bots and AI agents now generate a large share of internet traffic, shifting information search to machines and risking content quality. 2 sources. https://clstr.news/cluster/ai-models-uncover-tax-loopholes-and-become-primary-users-of-the-internet ### 2026-06-14: AI agents generate majority of global internet traffic, overtaking human users Cloudflare data shows AI agents now generate 57.4 % of global web traffic, surpassing human users and prompting security and infrastructure concerns. 2 sources. https://clstr.news/cluster/ai-agents-generate-majority-of-global-internet-traffic-overtaking-human-users ### 2026-06-10: Fastly reports AI-generated web traffic grows 6.5 times faster than human traffic Fastly finds AI web traffic rose 30 % Jan‑May 2026, growing 6.5 × faster than human traffic; 51 % of AI requests hit origin servers versus <9 % for humans, with Claude‑related traffic up 555 %. 2 sources. https://clstr.news/cluster/fastly-says-ai-generated-web-traffic-grows-65-times-faster-than-human-traffic ### 2026-06-07: Cloudflare Radar data shows bots outpacing humans in web traffic Cloudflare Radar shows bots exceed human traffic in HTML‑only views, but across all content types humans dominate, highlighting filter‑driven misinterpretations. 4 sources. https://clstr.news/cluster/cloudflare-radar-data-shows-bots-outpacing-humans-in-web-traffic ### 2026-06-06: Cloudflare CEO says AI bots now drive over half of internet traffic AI bots have overtaken humans, generating about 57 % of global web traffic, Cloudflare CEO Matthew Prince said, earlier than expected. 2 sources. https://clstr.news/cluster/cloudflare-ceo-says-ai-bots-now-drive-over-half-of-internet-traffic ### 2026-06-05: Cloudflare reports AI bots now drive majority of website traffic Cloudflare says AI bots now account for over half of web traffic, while a German blogger details a personal AI workflow for smart‑home automation. 2 sources. https://clstr.news/cluster/cloudflare-reports-ai-bots-now-drive-majority-of-website-traffic ### 2026-06-05: AI traffic now dominates internet, surpassing human users Cloudflare data shows AI bots now generate 57.4% of internet traffic, overtaking human users and challenging ad‑based revenue models and the authenticity of online discourse. 2 sources. https://clstr.news/cluster/ai-traffic-now-dominates-internet-surpassing-human-users ### 2026-06-05: Cloudflare says AI bots now generate majority of worldwide web page requests Cloudflare reports AI bots now account for about 57% of global webpage requests, surpassing human traffic for the first time, signalling a shift toward agent‑driven browsing. 2 sources. https://clstr.news/cluster/cloudflare-says-ai-bots-now-generate-majority-of-worldwide-web-page-requests ### 2026-06-05: Cloudflare reports AI agents now generate majority of internet traffic Cloudflare says AI agents now make up 57.4% of web traffic, overtaking humans at 42.6%, after a 1,500% surge in 2025 driven by chatbot‑powered browsing. 3 sources. https://clstr.news/cluster/cloudflare-reports-ai-agents-now-generate-majority-of-internet-traffic ### 2026-06-04: AI agents now generate most global web traffic Cloudflare data shows AI agents now account for 57.4% of global web traffic, overtaking humans and prompting debates on advertising models and the “dead internet theory.” 5 sources. https://clstr.news/cluster/ai-agents-generate-more-web-traffic-than-humans-worldwide ### 2026-06-04: Cloudflare reports AI bots now generate the majority of web traffic Cloudflare says AI bots now account for 57% of web traffic, overtaking humans earlier than expected, and discusses a pay‑to‑crawl model amid industry reaction. 5 sources. https://clstr.news/cluster/cloudflare-reports-ai-bots-now-generate-the-majority-of-web-traffic --- Cite as: AI bots dominate web traffic, toll‑booth charging expands. CLSTR, https://clstr.news/situations/ai-bots-dominate-web-traffic