robots.txt token
What is Applebot-Extended?
Applebot-Extended is a robots.txt token, not a crawler. It tells Apple whether pages Applebot crawled may be used to train Apple's foundation models for its generative AI features. Disallowing Applebot-Extended does not remove your pages from Siri, Spotlight or Safari suggestions, because Applebot still crawls them.
- Operator
- Apple
- What it does
- robots.txt control token, no crawler of its own
- robots.txt token
Applebot-Extended- Verified on rankecho.io, 30 days
- Never visits
What Applebot-Extended did on rankecho.io
Nothing, and that is expected: Applebot-Extended is a robots.txt token, so no request ever carries it. The fetches it controls are Applebot's. Over 17 October to 15 November 2026 Applebot made 1,206 verified requests to rankecho.io, read robots.txt 61 times and fetched 81 different pages. See Applebot's numbers.
How many sites block Applebot-Extended
2 of 13 sites whose robots.txt RankEcho read in audits from 3 November 2026 to 16 November 2026 block Applebot-Extended from the whole site. The sample is small and grows with every audit, so read it as a first look rather than a rate for the web.
Disallowing Applebot-Extended does not remove your pages from Siri, Spotlight or Safari suggestions, because Applebot still crawls them. No request ever arrives with Applebot-Extended in its user agent, so it never shows in logs.
Allow Applebot-Extended
User-agent: Applebot-Extended Allow: /
Block Applebot-Extended
User-agent: Applebot-Extended Disallow: /
Put the lines in the robots.txt file at the root of your domain. A more specific group for Applebot-Extended replaces the rules under User-agent: * for that crawler.
How to check a visit is really Applebot-Extended
Nothing to verify: Applebot-Extended never visits. The fetches are Applebot's.
Source: Apple: About Applebot.
Questions about Applebot-Extended
What is Applebot-Extended?
Applebot-Extended is a robots.txt token, not a crawler. It tells Apple whether pages Applebot crawled may be used to train Apple's foundation models for its generative AI features.
Should I block Applebot-Extended?
Disallowing Applebot-Extended does not remove your pages from Siri, Spotlight or Safari suggestions, because Applebot still crawls them. No request ever arrives with Applebot-Extended in its user agent, so it never shows in logs.
How do I know a visit is really from Applebot-Extended?
Nothing to verify: Applebot-Extended never visits. The fetches are Applebot's.
How many sites block Applebot-Extended?
2 of the 13 sites whose robots.txt RankEcho read in audits from 3 November 2026 to 16 November 2026 block Applebot-Extended from the whole site. It is a small sample, so read it as a first look, not a web-wide rate.
Does blocking Applebot-Extended remove me from Siri?
No. Siri and Spotlight use pages Applebot crawled. Applebot-Extended only controls whether those pages may train Apple's models.
Related crawlers
Check what your robots.txt allows
The free crawler policy check reads your robots.txt and says which AI crawlers it lets in. A free audit also asks ChatGPT, Perplexity and Gemini whether they cite you, and marks a missing answer as crawler access when a crawler is blocked.
Facts checked 2026-11-16 against Apple's documentation. Log numbers from 17 October to 15 November 2026. Read our crawler log study for the method.
