AI crawlers are not free traffic. They consume bandwidth, origin compute, and edge cache that publishers pay for, and unlike a human visitor they rarely view an ad or convert. This page models the real cost of AI bot traffic to publishers in 2026 and weighs it against the referral value those bots return. The figures draw on request volumes observed on the Presenc AI network and customer properties combined with public cost benchmarks. The short version is that for many sites, AI crawling is a net cost, not a net benefit, unless it is metered or converted into citations.
Modeled Crawl Cost per Million AI Requests
The table breaks down the infrastructure cost a publisher incurs to serve one million AI bot requests, by hosting tier. Costs assume an average page weight of 1.4 MB and typical cache hit rates.
| Hosting tier | Bandwidth cost | Origin compute | Total per 1M requests |
|---|---|---|---|
| Budget shared hosting | $84.00 | $41.00 | $125.00 |
| Standard cloud plus CDN | $38.00 | $22.00 | $60.00 |
| Enterprise edge cached | $11.00 | $7.00 | $18.00 |
| Dynamic or paywalled site | $47.00 | $96.00 | $143.00 |
Crawl Cost vs Referral Value by Operator
The cost only makes sense next to the value returned. The table compares the modeled cost to serve a typical operator against the referral revenue it sends back, per million crawls, for a mid-size content publisher.
| Operator | Cost to serve 1M crawls | Referral value returned | Net per 1M crawls |
|---|---|---|---|
| GPTBot | $60 | $34 | -$26 |
| Google-Extended | $60 | $9 | -$51 |
| ClaudeBot | $60 | $41 | -$19 |
| Bytespider | $60 | $2 | -$58 |
| PerplexityBot | $60 | $118 | +$58 |
Key Findings
- Most AI crawl traffic runs at a loss. Four of the five major operators return less referral value than they cost to serve for a mid-size publisher.
- Dynamic sites pay the most. Paywalled and dynamic properties incur the highest cost at $143 per million requests, driven by origin compute that cannot be cached.
- Perplexity is the exception. Its heavy live-retrieval mix returns $118 per million crawls, the only operator that is clearly net positive on referral alone.
- Bytespider is the worst trade. It costs the same to serve as any bot but returns just $2 per million, making blocking or metering an easy call.
Turning Cost into Revenue
Publishers have three levers. They can block low-value bots to cut cost, meter crawls through pay-per-crawl to charge for access, or optimize for citations so retrieval crawls convert into traffic. The right mix depends on the bot profile, which is why measurement comes first. Sites that blindly allow all AI traffic are subsidizing model training with no offsetting return.
Methodology
Request volumes and bot attribution were observed on the Presenc AI network and customer properties. Cost models were built from public cloud and CDN pricing, and referral values were compiled from public sources and analyst reports with Presenc AI estimates where public data was unavailable. Projections use compound growth modeling. Figures are reviewed quarterly. Last update June 2026.
How Presenc AI Helps
You cannot manage what you cannot see. Presenc AI logs every AI bot request to your origin, attributes the cost to each operator, and tracks how many citations and clicks each one returns, so you know exactly which crawlers are paying their way. See which AI bots crawl your site and turn on crawl-to-citation tracking to find the bots worth keeping.