Website owners now have a new, though still experimental, tool for verifying whether bot traffic claiming to be a Google AI agent is genuine. Google published new developer documentation introducing Web Bot Auth, describing it as "a new cryptographic protocol that helps websites to validate that bots are authentic."
The Problem Web Bot Auth Is Designed to Solve
The current standard for bot verification relies on user-agent strings and IP address checks, methods that carry a fundamental weakness. The HTTP user-agent request header used by Googlebot is often spoofed by other crawlers, making it important to verify that a problematic request actually comes from Google. That same vulnerability applies to all automated traffic, including AI agents. Right now, user agents are self-reported. Any bot can call itself Googlebot. Web Bot Auth changes that by adding cryptographic signatures to HTTP requests.
This creates a direct problem for website operators: blocking all unverified bot traffic aggressively risks cutting off Googlebot and legitimate AI agents, while being too permissive opens the door to scrapers and spoofed requests that drain server resources and harvest content without authorization.
What Web Bot Auth Does
Google defines Web Bot Auth as "an experimental cryptographic protocol used to authenticate requests sent by bots." Instead of relying solely on self-reported headers and IP addresses, Web Bot Auth allows agents to cryptographically sign their requests. Using Web Bot Auth helps website owners identify automated traffic on their sites and prevents other actors from attempting to spoof reputable agents.
The protocol adds another verification layer by letting agents sign HTTP requests with cryptographic keys. Websites can then verify those signatures against published public keys to confirm the request came from who it claims to be. Specifically, Web Bot Auth uses HTTP Message Signatures (RFC 9421) to let automated clients sign outgoing requests.
The underlying draft, `draft-meunier-web-bot-auth-architecture`, currently at version -05, published March 2, 2026, authored by Thibault Meunier of Cloudflare and Sandor Major of Google, applies HTTP Message Signatures (RFC 9421) to automated traffic. Each bot operator generates an Ed25519 keypair, publishes the public key as a JWKS at `/.well-known/http-message-signatures-directory` on a domain the operator controls, and signs every outbound HTTP request. Websites verify the signature and know, with cryptographic certainty, that the visitor came from the claimed operator.
Google's signed AI agent traffic uses the identity `https://agent.bot.goog`. Site owners can retrieve and cache Google's public keys from that endpoint and validate incoming `Signature` and `Signature-Input` headers labeled with the `g=` label, as described in Google's documentation.
Current Scope: Limited and Experimental
Google is not deploying Web Bot Auth across all of its bot traffic. Google added that this is "experimental," saying the company is "testing the protocol with some AI agents hosted on Google infrastructure." The documentation is explicit that not all Google user agents are using Web Bot Auth, and Google is not yet signing every request of agents using the protocol. Google recommends that in addition to Web Bot Auth, sites continue relying on IP addresses, reverse DNS, and user-agent strings as Google gradually rolls out signed traffic.
The experimental status also reflects where the underlying standard sits in the standards process. The IETF Web Bot Auth Working Group was chartered in early 2026 with milestones for standards-track specifications and a best current practice document. The webbotauth working group has published milestones targeting standards-track specifications for authentication techniques and bot information mechanisms, and a Best Current Practice operational document by August 2026.
Industry Adoption Already Underway
Google is not the only major infrastructure provider behind this protocol. Amazon, Cloudflare, Akamai, and OpenAI are pushing the same standard forward. AWS WAF has announced the addition of Web Bot Auth support, providing a secure and standardized way to authenticate legitimate AI agents and automated tools accessing web applications. On the CDN side, Cloudflare has published its own implementation documentation and supports Ed25519 key-based verification within its WAF products.
For sites already using a CDN or WAF that supports the protocol, verification may happen automatically.
What This Means for Site Owners and Developers
The privacy design of the protocol limits its use to operator-level identity, not individual user tracking. The key used for signing must not be tied to a specific human individual. Keys should represent a role, company, or automation identity. This avoids accidental exposure of personally identifiable information and prevents the misuse of keys for user tracking or profiling.
For developers who want to implement verification manually, Google's documentation points to the HTTP Message Signatures for Automated Traffic Architecture specification and example implementations on GitHub. The process involves fetching Google's public key set from `https://agent.bot.goog/.well-known/http-message-signatures-directory`, caching keys according to the `Cache-Control` header, and validating the `Signature` header on participating requests.
For site owners and SEOs using major CDN or WAF providers, the practical implication is to confirm whether their provider already supports Web Bot Auth verification, in which case signed Google-Agent requests may be handled automatically. Those managing their own infrastructure should review the IETF draft architecture and Google's developer documentation before implementing, since the protocol remains in active development and the specification may change.
The Web Bot Auth protocol may eventually provide more granular controls. The IETF working group has milestones targeting mid-2026 for initial specifications. Google's documentation states that until the protocol reaches wider adoption, falling back to established IP-based and reverse DNS verification methods remains the recommended default.
