RankEcho

Link preview fetcher

What is Twitterbot?

Twitterbot fetches a page when its link is posted on X, to build the card preview from the page's tags. X says its crawler respects robots.txt, so blocking Twitterbot stops cards for your pages.

Operator
X
What it does
Fetches a page to build a link preview
robots.txt token
Twitterbot
Verified on rankecho.io, 30 days
60 requests

What Twitterbot did on rankecho.io

From 17 October to 15 November 2026, Cloudflare verified 60 requests from Twitterbot, on 7 of the 30 days. It fetched 2 different pages, read robots.txt 40 times, sitemaps 0 times and llms.txt 0 times.

Verified requests607 of 30 days
Different pages fetched25 page requests
robots.txt reads40sitemaps 0, llms.txt 0
Requests faking its name2same user agent, not verified
  • Verified requests
Verified Twitterbot requests per day to rankecho.io, 17 October to 15 November 2026
DateVerified requests
Oct 170
Oct 1810
Oct 190
Oct 200
Oct 210
Oct 220
Oct 230
Oct 2410
Oct 2510
Oct 260
Oct 270
Oct 280
Oct 290
Oct 300
Oct 313
Nov 110
Nov 20
Nov 30
Nov 40
Nov 50
Nov 60
Nov 70
Nov 813
Nov 90
Nov 100
Nov 110
Nov 120
Nov 130
Nov 140
Nov 154
Pages Twitterbot fetched most
PathRequests
/benchmarks/state-of-ai-citations-q3-20263
/2

Also in those days, 2 requests used Twitterbot's user agent without Cloudflare verifying it.

Source: rankecho.io's Cloudflare logs, visitor requests only, queried 2026-11-16. Cloudflare sampled 20 of the verified requests, so those counts are estimates.

Allowing or blocking Twitterbot

RankEcho's audits do not read the Twitterbot token yet, so there is no count of sites blocking it. The audit checks the AI and search crawlers that decide whether you can be cited.

X says its crawler respects robots.txt, so blocking Twitterbot stops cards for your pages. Allow it if you want links on X to show a preview.

Allow Twitterbot

User-agent: Twitterbot
Allow: /

Block Twitterbot

User-agent: Twitterbot
Disallow: /

Put the lines in the robots.txt file at the root of your domain. A more specific group for Twitterbot replaces the rules under User-agent: * for that crawler.

How to check a visit is really Twitterbot

Cloudflare checks requests against X's known addresses and marks real ones as verified bots, which is how this page tells real visits from borrowed names. The user agent alone proves nothing.

Source: X: Cards, getting started.

Questions about Twitterbot

What is Twitterbot?

Twitterbot fetches a page when its link is posted on X, to build the card preview from the page's tags.

Should I block Twitterbot?

X says its crawler respects robots.txt, so blocking Twitterbot stops cards for your pages. Allow it if you want links on X to show a preview.

How do I know a visit is really from Twitterbot?

Cloudflare checks requests against X's known addresses and marks real ones as verified bots, which is how this page tells real visits from borrowed names. The user agent alone proves nothing.

How often does Twitterbot visit a site?

On rankecho.io, Twitterbot made 60 Cloudflare-verified requests from 17 October to 15 November 2026, on 7 of the 30 days. Your site will differ with its size and how often it changes.

Related crawlers

Check what your robots.txt allows

The free crawler policy check reads your robots.txt and says which AI crawlers it lets in. A free audit also asks ChatGPT, Perplexity and Gemini whether they cite you, and marks a missing answer as crawler access when a crawler is blocked.

Facts checked 2026-11-16 against X's documentation. Log numbers from 17 October to 15 November 2026. Read our crawler log study for the method.