AI Bot Traffic Surged 300%, Publishers Got Hit the Hardest, And SEO Feels It Next
Akamai published its latest State of the Internet report this week, and the findings should concern anyone who creates content, builds links, or invests in organic visibility. The report, titled “Protecting Publishing: Navigating the AI Bot Era,” tracked AI bot activity across Akamai’s global network from July through December of last year. The numbers are significant. AI bot activity surged by 300%, and the media industry ranked second globally with 13% of all AI bot traffic, with publishing organizations making up 40% of that activity.
Those percentages translate into an enormous volume of automated requests hitting content rich websites. And the implications go well beyond server load.
What Are AI Bots and Why Are They Scraping Content
To understand what’s happening, it helps to know the difference between the types of AI bots that are hitting websites right now. The Akamai report breaks them into distinct categories, each with different objectives and different consequences for publishers.
AI training crawlers are bots that systematically scrape website content to feed it into large language model training datasets. When a company like OpenAI or Google builds the next version of its language model, the training data has to come from somewhere. These crawlers visit websites at scale, download text content, and store it for use in model training. AI training crawlers made up 63% of all AI bots targeting the media industry. The content they collect becomes part of the model’s knowledge base permanently.
AI fetchers are different. These bots retrieve content in real time to power AI driven search tools and chatbot responses. When someone asks ChatGPT or Perplexity a question and gets an answer that references a specific article or dataset, a fetcher likely pulled that content moments before. AI fetchers represented 24% of all AI bot activity targeting media, with publishing accounting for 43% of that segment.
AI search crawlers operate similarly to traditional search engine crawlers, but they index content specifically for AI powered search interfaces rather than conventional search results pages.
The distinction between training crawlers and fetchers matters because they create different problems. Training crawlers take content once and bake it into a model forever. Fetchers take content repeatedly, in real time, and use it to generate answers that may keep users from ever visiting the original source. Both consume server resources. Neither generates pageviews, ad impressions, or subscription conversions for the publisher.
The Referral Traffic Problem
The most striking data point in the report concerns what happens after AI systems consume publisher content. AI chatbots drove approximately 96% less referral traffic than traditional Google search. That number was measured in Q4 2024, and the trend has only accelerated since then.
Consider what that means in practice. A user searches Google for a topic, clicks a result, and lands on a publisher’s website. The publisher gets a pageview, serves ads, has a chance to convert a subscriber, and builds brand familiarity. That exchange has powered the economics of online publishing for two decades.
Now compare that to what happens with an AI chatbot. A user asks a question, the AI fetcher pulls content from one or several publisher sites, synthesizes an answer, and delivers it in the chat interface. The user gets what they need. The publisher gets nothing. No visit, no ad impression, no brand exposure, no opportunity to convert. The content was used, but the value was captured entirely by the AI platform.
OpenAI generated the highest volume of AI bot traffic targeting media companies, with publishing organizations accounting for 40% of all OpenAI requests. That makes OpenAI the single largest identified source of AI bot traffic hitting publishers.
Why Content Creators Should Pay Attention
The Akamai data describes a structural shift in how content gets consumed and who captures the value from it. For anyone creating content as part of a link building or digital PR strategy, this shift has real operational consequences.
Content published through guest posting on third party sites has traditionally served two purposes: earning a backlink for SEO value and placing a brand in front of that publisher’s audience. The second part of that equation depends on readers actually visiting the publisher’s site, reading the article, and encountering the brand in context. If AI bots are scraping that content and serving it through chatbot interfaces without attribution or links, the guest post still exists on the publisher’s site, but fewer people are reading it there.
The same logic applies to any content that lives on a third party domain. Thought leadership articles, contributed columns, data driven research that earns editorial coverage. All of these depend on the publisher’s ability to attract and retain readers. As AI tools pull content out of publisher sites and serve it directly, the traffic those publishers receive declines, their ad revenue drops, and the economic model that supports their editorial operations gets squeezed.
That squeeze has downstream effects. Publishers under revenue pressure cut editorial staff, tighten contributor guidelines, reduce content output, or shut down entirely. The ecosystem that SEO and link building depends on, a healthy network of publishers creating high quality content, doesn’t function the same way when the economic incentives that sustain it are being systematically undermined.
The Scraping vs. Blocking Dilemma
Publishers face an uncomfortable choice. They can try to block AI bots, which risks losing visibility in AI powered search results that are rapidly growing in usage. Or they can allow scraping, which means their content gets used to generate answers that compete with their own traffic.
Blocking AI crawlers through robots.txt or other technical measures might protect content from being scraped, but it also means that content won’t appear in AI generated answers. As more users shift toward AI interfaces for information retrieval, being absent from those answers means being absent from a growing share of discovery. The trade off is between protecting today’s revenue model and maintaining relevance in tomorrow’s information ecosystem.
Some publishers are negotiating licensing deals directly with AI companies. Others are exploring technical solutions that allow AI systems to access content under specific conditions, like providing attribution and linking back to the source. But no industry wide standard has emerged, and the pace of bot development continues to outrun the pace of policy.
What This Means for SEO Strategy
The AI bot surge doesn’t change the fundamentals of SEO overnight, but it does change the environment in which SEO operates.
First, the value of direct brand visibility becomes more important as intermediary platforms (including AI chatbots) capture a larger share of content discovery. If users are getting answers from AI tools instead of visiting websites, the brands that get mentioned by name in those AI generated answers are the ones that maintain visibility. Building entity recognition, the kind of consistent brand and topic association that helps AI systems identify a source as authoritative, becomes a competitive advantage. Companies investing in content strategies that associate their brand with specific topics and services are better positioned to appear in AI generated responses, even when direct referral traffic declines.
Second, the quality of backlinks matters more when the quantity of available publishers is under pressure. If AI driven revenue erosion causes publishers to consolidate or close, the supply of high authority sites available for link insertion and guest contributions contracts. Links from surviving, well maintained, authoritative publications become more valuable precisely because there are fewer of them. Building relationships with quality publishers now, before further consolidation occurs, is a hedge against a shrinking ecosystem.
Third, content that gets scraped and repackaged by AI systems doesn’t carry the same SEO value as content that drives engagement on a publisher’s site. Google’s algorithms still reward content that generates real user signals: clicks, time on page, engagement. If AI bots are consuming content without producing any of those signals, the content may still exist but its contribution to rankings could weaken over time as engagement metrics shift.
The Bigger Picture
The Akamai report measures something that has been building for the past two years but is now reaching a scale that’s hard to dismiss. AI systems are consuming publisher content at an accelerating rate, generating less referral traffic in return, and reshaping the economics that have supported online content creation since the early days of search.
For anyone working in SEO, content marketing, or link building, the response isn’t to panic. The response is to recognize that the ecosystem is changing and to build strategies that account for that change. Brand visibility in AI generated answers, strong relationships with quality publishers, and content that creates genuine engagement are all more important now than they were a year ago, and they’ll be more important a year from now than they are today.
The sites and brands that adapt to this shift won’t just survive it. They’ll benefit from the fact that many of their competitors haven’t noticed it yet.