RankEcho

Crawling and indexing

What is a user-triggered fetcher?

A user-triggered fetcher is a bot that fetches a page because a person asked for it, such as ChatGPT-User when someone asks ChatGPT about a link. It is not a crawler on a schedule, so several operators say robots.txt does not apply to it, and its visits follow people's questions.

How to read it

Fetchers are the bots closest to a reader: each visit is someone asking an assistant about your page right now. Blocking them removes you from those answers.

Because they act for a person, Google and Perplexity say their user-triggered fetchers generally ignore robots.txt, and OpenAI and Mistral leave it to their own pages. A firewall rule is the only hard block.

Example from rankecho.io

ChatGPT-User: 448 requests, 341 on one day

ChatGPT-User made 448 verified requests to rankecho.io in 30 days, 341 of them on 12 November. Fetchers come in bursts like that when one question gets asked a lot.

In the same days Cloudflare verified no request from Claude-User or Perplexity-User. Their visits depend on what their users ask, not on the site.

Source: rankecho.io's Cloudflare logs, verified requests only, 17 October to 15 November 2026.

Questions people ask

What is a user-triggered fetcher?

A user-triggered fetcher is a bot that fetches a page because a person asked for it, such as ChatGPT-User when someone asks ChatGPT about a link. It is not a crawler on a schedule, so several operators say robots.txt does not apply to it, and its visits follow people's questions.

Do user-triggered fetchers obey robots.txt?

It depends on the operator. Several say robots.txt does not apply because a person asked; Anthropic says Claude-User respects it. Check each operator's page.

Are fetches used for training?

The major operators say fetches made for a user are not used to train models.

Related terms

Read more on rankecho.io

See it on your own site

RankEcho reads your Search Console data and asks AI assistants whether they cite you, then shows each of these numbers for your pages. The audit is free.

Definition and example checked 2026-11-16.