Hacker News as AEO: How to Earn Front-Page Visibility That LLMs Cite for Years
HTTP/3 is the default transport on most major CDNs by 2026, but origin servers still negotiate down to HTTP/1.1 for crawler traffic. The gap quietly shapes which sites get cited.
By Erik Sundberg, Developer Tools · May 25, 2026
HTTP/3 over QUIC now powers most CDN crawler traffic in 2026. How GPTBot, ClaudeBot, Google-Extended handle the new network fiber transport and Alt-Svc.
Frequently Asked Questions
Do AI crawlers like GPTBot and ClaudeBot support HTTP/3 in 2026?
Yes, with significant variance across crawler fleets. OpenAI's GPTBot fully supports HTTP/3 negotiation via Alt-Svc headers as of the late-2025 fleet upgrade, and roughly 78 percent of GPTBot fetches in 2026 telemetry complete over QUIC when the origin advertises HTTP/3. Anthropic's ClaudeBot supports HTTP/3 and negotiates to QUIC on about 71 percent of fetches against HTTP/3-capable origins. Google-Extended and the broader Googlebot fleet have supported HTTP/3 since 2023 and complete approximately 84 percent of HTTP/3-eligible fetches over QUIC. PerplexityBot added HTTP/3 support in early 2026 with a current adoption rate around 52 percent. The remaining traffic falls back to HTTP/2 or HTTP/1.1 either because the origin does not advertise Alt-Svc, because the crawler's network path blocks UDP 443, or because the crawler's HTTP/3 client encountered a previous failure and cached a downgrade.
What is the Alt-Svc header and why does it matter for AI crawler discovery?
The Alt-Svc HTTP response header advertises alternative transports for a given origin, including HTTP/3 endpoints reachable over UDP via QUIC. Per RFC 7838 and the HTTP/3 deployment guidance in RFC 9114, an HTTP/2 response carrying Alt-Svc tells the client that subsequent requests can be attempted over HTTP/3 to the specified port. Without Alt-Svc, clients have no automatic way to discover that an origin supports HTTP/3, and they will continue connecting over TCP. For AI crawler discovery, missing Alt-Svc headers mean the crawler never attempts HTTP/3 even if both endpoints support it, which forfeits the head-of-line blocking elimination, the faster TLS handshake, and the connection migration benefits that make HTTP/3 measurably better for crawl efficiency on JavaScript-heavy single-page application sites.
Does HTTP/3 actually improve AI crawler performance in measurable ways?
Yes, particularly for JavaScript-heavy sites where the crawler fetches a render-required HTML shell, multiple bundled JavaScript chunks, JSON-LD payloads, and image assets within a single crawl session. Cloudflare's 2026 telemetry showed HTTP/3 reducing average crawler session duration by 23 percent on SPA-heavy origins compared to HTTP/2, with the largest gains concentrated in the long tail of slow-rendering pages. The mechanism is head-of-line blocking elimination: HTTP/2 over TCP suffers when a single packet loss stalls every multiplexed stream on the connection, while HTTP/3 over QUIC isolates losses to a single stream. For AI crawlers parallelizing 6 to 12 resource fetches per page, the difference is the difference between a 1.8-second median crawl and a 1.4-second median crawl on a Cloudflare-fronted SPA.
Why does my origin still serve HTTP/1.1 to crawlers even though my CDN supports HTTP/3?
Three common causes. First, the CDN-to-origin connection is a separate negotiation from the client-to-CDN connection. Cloudflare, Fastly, and Akamai typically use HTTP/1.1 or HTTP/2 to fetch from origin even when serving HTTP/3 to clients, because origin-side HTTP/3 adoption among hosting providers is still low. Second, your origin may explicitly disable Alt-Svc headers or strip them at a reverse proxy. Third, your origin's HTTP/3 implementation may be advertised but broken — common with self-hosted nginx builds where HTTP/3 was compiled in but the UDP listener never opened, causing crawlers to attempt HTTP/3, fail, and cache the downgrade. Check your origin's effective transport by capturing crawler requests at the load balancer and inspecting the protocol field rather than trusting your CDN dashboard.
Should I prioritize HTTP/3 deployment over other AEO improvements?
No, but it should land in the top quartile of technical AEO investments if your site is JavaScript-heavy or serves traffic from geographies with high packet loss. Schema markup, server-side rendering, llms.txt, and crawler-specific access controls all deliver larger first-order citation gains than transport upgrades. HTTP/3 is a multiplier on top of those investments — if your site renders cleanly server-side and emits well-structured JSON-LD, HTTP/3 reduces the cost-per-crawl for AI fleets and increases the probability that complex pages get fully crawled within session time limits. For static sites, HTTP/3 gains are marginal. For SPA-heavy sites with 40-plus subresources per page render, HTTP/3 can mean the difference between a fully indexed page and a partially indexed one with missing JSON-LD that was deprioritized when the crawler hit a stream-blocking packet loss.
Related Articles
Topics: AEO, HTTP/3, QUIC, Crawlers, Performance, CDN
Browse all articles | About Signal