RankEcho

AI agent

What is Claude-User?

Claude-User fetches a page when a Claude user asks a question that needs it. It acts for that person, not on a crawl schedule. Anthropic says Claude-User respects robots.txt.

Operator
Anthropic
What it does
Fetches a page when a user asks
robots.txt token
Claude-User
Verified on rankecho.io, 30 days
0 requests

What Claude-User did on rankecho.io

Verified requests030 days
Requests using its name100not verified by Cloudflare
Secret-file probes among them23/.env, compose.yml and similar

Cloudflare verified no request from Claude-User to rankecho.io from 17 October to 15 November 2026. In the same days 100 requests used Claude-User's user agent without Cloudflare verifying it, and 23 of those asked for secret files such as /.env or compose.yml, which no real crawler needs. Cloudflare checks requests against the operator's published addresses, so an unverified request may come from an address Cloudflare does not know yet, or from someone borrowing the name. Treat them as unconfirmed.

Source: rankecho.io's Cloudflare logs, visitor requests only, queried 2026-11-16.

How many sites block Claude-User

0 of 13 sites whose robots.txt RankEcho read in audits from 3 November 2026 to 16 November 2026 block Claude-User from the whole site. The sample is small and grows with every audit, so read it as a first look rather than a rate for the web.

Anthropic says Claude-User respects robots.txt. Blocking it stops Claude from reading your pages when a user asks, which can stop Claude citing them in that answer.

Allow Claude-User

User-agent: Claude-User
Allow: /

Block Claude-User

User-agent: Claude-User
Disallow: /

Put the lines in the robots.txt file at the root of your domain. A more specific group for Claude-User replaces the rules under User-agent: * for that crawler.

How to check a visit is really Claude-User

Anthropic publishes the IP addresses its crawlers use. Compare the request IP with that list, or let Cloudflare do it: real requests are marked as verified bots.

Source: Anthropic: Does Anthropic crawl data from the web?.

Questions about Claude-User

What is Claude-User?

Claude-User fetches a page when a Claude user asks a question that needs it. It acts for that person, not on a crawl schedule.

Should I block Claude-User?

Anthropic says Claude-User respects robots.txt. Blocking it stops Claude from reading your pages when a user asks, which can stop Claude citing them in that answer.

How do I know a visit is really from Claude-User?

Anthropic publishes the IP addresses its crawlers use. Compare the request IP with that list, or let Cloudflare do it: real requests are marked as verified bots.

How often does Claude-User visit a site?

On rankecho.io, Cloudflare verified no request from Claude-User from 17 October to 15 November 2026, although 100 requests used its user agent.

How many sites block Claude-User?

0 of the 13 sites whose robots.txt RankEcho read in audits from 3 November 2026 to 16 November 2026 block Claude-User from the whole site. It is a small sample, so read it as a first look, not a web-wide rate.

Related crawlers

Check what your robots.txt allows

The free crawler policy check reads your robots.txt and says which AI crawlers it lets in. A free audit also asks ChatGPT, Perplexity and Gemini whether they cite you, and marks a missing answer as crawler access when a crawler is blocked.

Facts checked 2026-11-16 against Anthropic's documentation. Log numbers from 17 October to 15 November 2026. Read our crawler log study for the method.