SeenLensBot: user agent, request limits and opt-out
SeenLensBot is the scanner behind SeenLens. It visits a site only when someone scans one of its pages. It fetches that page, robots.txt, llms.txt and a few well-known files, at most about 25 requests per scan. It obeys robots.txt rules written for SeenLensBot.
Last updated
User agent and robots.txt block
Every request from SeenLensBot carries this token in its User-Agent header:
SeenLensBot/0.1 (+https://seenlens.com/bot)The full header starts with a standard browser prefix, because many firewalls reject bare bot strings, and ends with the token above. Match on SeenLensBot in your logs or firewall rules.
To block SeenLensBot from the whole site, add this group to your robots.txt:
User-agent: SeenLensBot
Disallow: /SeenLensBot reads robots.txt before it fetches the page. If the page path is disallowed for SeenLensBot, it does not fetch the page. You can also disallow only some paths, for example Disallow: /private/. Blocking SeenLensBot has no effect on other crawlers, and it does not change how AI search engines see your site.
What SeenLensBot fetches and why
A scan checks how visible one page is to AI search engines and AI agents. To do that, SeenLensBot requests only what those systems would request:
| Path | Why SeenLensBot requests it |
|---|---|
| The page URL that was submitted (redirects are followed) | It is the page that gets scored. |
/robots.txt | To read your crawler rules, including any group for SeenLensBot, AI crawlers and Content-Signal lines. |
/llms.txt and /llms-full.txt | To check whether the optional llms.txt files exist and are well formed. |
| A Markdown version of the page | To check whether agents can get clean text. SeenLensBot asks for the page with Accept: text/markdown, follows a rel="alternate" Markdown link if the page has one, and tries the page path with .md. |
/.well-known/api-catalog, /.well-known/oauth-protected-resource, /.well-known/oauth-authorization-server, /.well-known/agent-card.json, /.well-known/ucp, /.well-known/ai-catalog.json | To check the discovery files that AI agents look for. |
One made-up path that starts with /claude-seo-404-probe- | To confirm that unknown paths return 404 and not a page with status 200. |
That is about 25 requests at most per scan. SeenLensBot does not follow links, does not crawl other pages, does not submit forms and does not try to log in. If the raw HTML is an empty JavaScript app shell, it loads the same page once in a headless browser so the content can be read; that browser load also fetches the page's own scripts and styles.
What a SeenLensBot request looks like in your log
This is an example line in combined log format. The IP address is a documentation address, not a real SeenLensBot address.
203.0.113.7 - - [06/Nov/2026:09:14:02 +0000] "GET /robots.txt HTTP/2.0" 200 412 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/150.0.0.0 Safari/537.36 SeenLensBot/0.1 (+https://seenlens.com/bot)"A scan produces a short burst of lines like this one and then stops. SeenLens does not publish an IP range for SeenLensBot, so match requests on the user agent token. To check a request you are unsure about, email the time and the IP address to the contact address below.
How often it visits
SeenLensBot has no crawl schedule. It never revisits a site on its own. Each visit is triggered by one person submitting one URL on seenlens.com, and the scan finishes in one short burst of requests. We rate-limit scans so that nobody can use SeenLens to send heavy traffic to a server.
Contact
If SeenLensBot causes a problem on your site, or you want us to stop scanning your domain altogether, email support@seenlens.com. Include your domain and, if you can, the time and IP address of the requests. We reply within two business days. Other ways to reach us.
Want to see what SeenLensBot reports? Scan your own page or read how the checks work. SeenLens is run as described on the About page.