SEO tool crawler
What is AhrefsBot?
AhrefsBot crawls the web to build the link and page index behind Ahrefs' SEO tools. Blocking AhrefsBot keeps your pages and links out of Ahrefs' fresh data, so Ahrefs users see less about your site.
- Operator
- Ahrefs
- What it does
- Builds an SEO tool's index
- robots.txt token
AhrefsBot- Verified on rankecho.io, 30 days
- 3,586 requests
What AhrefsBot did on rankecho.io
From 17 October to 15 November 2026, Cloudflare verified 3,586 requests from AhrefsBot, on 29 of the 30 days. It fetched 2,527 different pages, read robots.txt 161 times, sitemaps 54 times and llms.txt 2 times. 459 of its requests got an error status (4xx or 5xx).
- Verified requests
| Date | Verified requests |
|---|---|
| Oct 17 | 0 |
| Oct 18 | 80 |
| Oct 19 | 10 |
| Oct 20 | 10 |
| Oct 21 | 50 |
| Oct 22 | 50 |
| Oct 23 | 290 |
| Oct 24 | 80 |
| Oct 25 | 63 |
| Oct 26 | 42 |
| Oct 27 | 4 |
| Oct 28 | 7 |
| Oct 29 | 10 |
| Oct 30 | 28 |
| Oct 31 | 52 |
| Nov 1 | 534 |
| Nov 2 | 661 |
| Nov 3 | 279 |
| Nov 4 | 318 |
| Nov 5 | 95 |
| Nov 6 | 36 |
| Nov 7 | 33 |
| Nov 8 | 93 |
| Nov 9 | 47 |
| Nov 10 | 20 |
| Nov 11 | 34 |
| Nov 12 | 115 |
| Nov 13 | 417 |
| Nov 14 | 74 |
| Nov 15 | 54 |
| Path | Requests |
|---|---|
/benchmark/homefurniturenyc.com | 20 |
/benchmark/subsconscious.com | 16 |
/benchmark/darussalamny.com | 14 |
/benchmark/greatmany.com | 14 |
/benchmark/jennynguyen51.glossgenius.com | 14 |
Also in those days, 1,144 requests used AhrefsBot's user agent without Cloudflare verifying it.
Source: rankecho.io's Cloudflare logs, visitor requests only, queried 2026-11-16. Cloudflare sampled 680 of the verified requests, so those counts are estimates.
Allowing or blocking AhrefsBot
RankEcho's audits do not read the AhrefsBot token yet, so there is no count of sites blocking it. The audit checks the AI and search crawlers that decide whether you can be cited.
Blocking AhrefsBot keeps your pages and links out of Ahrefs' fresh data, so Ahrefs users see less about your site. It does not change Google, Bing or the AI assistants' own crawlers.
Allow AhrefsBot
User-agent: AhrefsBot Allow: /
Block AhrefsBot
User-agent: AhrefsBot Disallow: /
Put the lines in the robots.txt file at the root of your domain. A more specific group for AhrefsBot replaces the rules under User-agent: * for that crawler.
How to check a visit is really AhrefsBot
Ahrefs publishes the IP ranges its crawlers use. A request from outside them is not Ahrefs. Cloudflare marks real requests as verified bots.
Source: Ahrefs: AhrefsBot.
Questions about AhrefsBot
What is AhrefsBot?
AhrefsBot crawls the web to build the link and page index behind Ahrefs' SEO tools.
Should I block AhrefsBot?
Blocking AhrefsBot keeps your pages and links out of Ahrefs' fresh data, so Ahrefs users see less about your site. It does not change Google, Bing or the AI assistants' own crawlers.
How do I know a visit is really from AhrefsBot?
Ahrefs publishes the IP ranges its crawlers use. A request from outside them is not Ahrefs. Cloudflare marks real requests as verified bots.
How often does AhrefsBot visit a site?
On rankecho.io, AhrefsBot made 3,586 Cloudflare-verified requests from 17 October to 15 November 2026, on 29 of the 30 days. Your site will differ with its size and how often it changes.
Why does AhrefsBot crawl more than Googlebot?
On rankecho.io it did, over the same 30 days. SEO tool crawlers refresh link data across the whole web, so they can fetch a site more often than a search engine does.
Related crawlers
Check what your robots.txt allows
The free crawler policy check reads your robots.txt and says which AI crawlers it lets in. A free audit also asks ChatGPT, Perplexity and Gemini whether they cite you, and marks a missing answer as crawler access when a crawler is blocked.
Facts checked 2026-11-16 against Ahrefs's documentation. Log numbers from 17 October to 15 November 2026. Read our crawler log study for the method.
