RankEcho

Crawling and indexing

What is Web Bot Auth?

Web Bot Auth is a way for a bot to prove who it is by signing each request with HTTP Message Signatures. The operator publishes its public key, and the site or CDN checks the signature. Unlike a user agent, a signature cannot be copied, and unlike IP lists, it survives a change of network.

How to read it

User agents can be faked and IP ranges change. A signed request carries a Signature-Agent header naming the operator and a signature only the operator's private key could make.

You rarely check signatures yourself. CDNs such as Cloudflare verify them and mark the request as a verified bot, so your rules can trust that mark.

Example from rankecho.io

ExaSearchBot: 1,172 verified requests

ExaSearchBot, the crawler of the AI search engine Exa, signs its requests with Web Bot Auth. In 30 days Cloudflare verified 1,172 of its requests to rankecho.io, on 30 of 30 days, and 8 more used its name without verifying.

Source: rankecho.io's Cloudflare logs, verified requests only, 17 October to 15 November 2026.

Questions people ask

What is Web Bot Auth?

Web Bot Auth is a way for a bot to prove who it is by signing each request with HTTP Message Signatures. The operator publishes its public key, and the site or CDN checks the signature. Unlike a user agent, a signature cannot be copied, and unlike IP lists, it survives a change of network.

Do I need to set anything up for Web Bot Auth?

Usually not. If your CDN supports it, signed bots are verified for you. Check your bot rules allow the verified bots you want.

Which crawlers sign their requests?

A growing number, including some AI agents and crawlers. Each operator's documentation says whether it signs.

Related terms

Read more on rankecho.io

See it on your own site

RankEcho reads your Search Console data and asks AI assistants whether they cite you, then shows each of these numbers for your pages. The audit is free.

Definition and example checked 2026-11-16.