AI crawler
What is ExaSearchBot?
ExaSearchBot crawls the web for Exa, a search engine that AI applications query through an API. Exa says a group for ExaSearchBot in robots.txt comes first; without one it follows your rules for major search engines, then the * group.
- Operator
- Exa
- What it does
- Builds an AI search index
- robots.txt token
ExaSearchBot- Verified on rankecho.io, 30 days
- 1,172 requests
What ExaSearchBot did on rankecho.io
From 17 October to 15 November 2026, Cloudflare verified 1,172 requests from ExaSearchBot, on 30 of the 30 days. It fetched 204 different pages, read robots.txt 32 times, sitemaps 2 times and llms.txt 0 times.
- Verified requests
| Date | Verified requests |
|---|---|
| Oct 17 | 20 |
| Oct 18 | 80 |
| Oct 19 | 60 |
| Oct 20 | 10 |
| Oct 21 | 50 |
| Oct 22 | 70 |
| Oct 23 | 50 |
| Oct 24 | 30 |
| Oct 25 | 43 |
| Oct 26 | 29 |
| Oct 27 | 58 |
| Oct 28 | 39 |
| Oct 29 | 38 |
| Oct 30 | 35 |
| Oct 31 | 34 |
| Nov 1 | 35 |
| Nov 2 | 66 |
| Nov 3 | 39 |
| Nov 4 | 37 |
| Nov 5 | 40 |
| Nov 6 | 42 |
| Nov 7 | 34 |
| Nov 8 | 50 |
| Nov 9 | 27 |
| Nov 10 | 31 |
| Nov 11 | 24 |
| Nov 12 | 32 |
| Nov 13 | 33 |
| Nov 14 | 24 |
| Nov 15 | 12 |
| Path | Requests |
|---|---|
/benchmarks/state-of-ai-citations-q3-2026 | 70 |
/product/ai-visibility-audit | 47 |
/learn/ai-visibility-for-healthcare | 40 |
/compare/agencyanalytics-alternatives | 38 |
/resources/find/prompt-gap-analysis | 37 |
Also in those days, 8 requests used ExaSearchBot's user agent without Cloudflare verifying it.
Source: rankecho.io's Cloudflare logs, visitor requests only, queried 2026-11-16. Cloudflare sampled 380 of the verified requests, so those counts are estimates.
Allowing or blocking ExaSearchBot
RankEcho's audits do not read the ExaSearchBot token yet, so there is no count of sites blocking it. The audit checks the AI and search crawlers that decide whether you can be cited.
Exa says a group for ExaSearchBot in robots.txt comes first; without one it follows your rules for major search engines, then the * group. robots.txt stops crawling, so to drop a page Exa already indexed, add a noindex tag.
Allow ExaSearchBot
User-agent: ExaSearchBot Allow: /
Block ExaSearchBot
User-agent: ExaSearchBot Disallow: /
Put the lines in the robots.txt file at the root of your domain. A more specific group for ExaSearchBot replaces the rules under User-agent: * for that crawler.
How to check a visit is really ExaSearchBot
Exa signs its requests with HTTP Message Signatures (Web Bot Auth), with a Signature-Agent header naming Exa. A request without a valid signature is not Exa.
Source: Exa: ExaSearchBot.
Questions about ExaSearchBot
What is ExaSearchBot?
ExaSearchBot crawls the web for Exa, a search engine that AI applications query through an API.
Should I block ExaSearchBot?
Exa says a group for ExaSearchBot in robots.txt comes first; without one it follows your rules for major search engines, then the * group. robots.txt stops crawling, so to drop a page Exa already indexed, add a noindex tag.
How do I know a visit is really from ExaSearchBot?
Exa signs its requests with HTTP Message Signatures (Web Bot Auth), with a Signature-Agent header naming Exa. A request without a valid signature is not Exa.
How often does ExaSearchBot visit a site?
On rankecho.io, ExaSearchBot made 1,172 Cloudflare-verified requests from 17 October to 15 November 2026, on 30 of the 30 days. Your site will differ with its size and how often it changes.
Related crawlers
Check what your robots.txt allows
The free crawler policy check reads your robots.txt and says which AI crawlers it lets in. A free audit also asks ChatGPT, Perplexity and Gemini whether they cite you, and marks a missing answer as crawler access when a crawler is blocked.
Facts checked 2026-11-16 against Exa's documentation. Log numbers from 17 October to 15 November 2026. Read our crawler log study for the method.
