---
title: "Fix: ChatGPT Can't Access My Website (Page Not Loading)"
description: "ChatGPT and other AI crawlers fail to load a page when it returns an error, a bot challenge or a long redirect chain. Find the cause with curl and fix it."
url: https://seenlens.com/fixes/ai-crawlers-cannot-load-page
last_updated: 2026-11-06
---

# ChatGPT can't access my website: how to fix it

ChatGPT cannot access your website when a plain request for your page returns an error, a bot challenge or a long redirect chain. The usual causes are a firewall rule, a server error or a wrong redirect. Fixing the cause lets AI crawlers read the page. Most fixes take under one hour.

By SeenLens. Last updated: 2026-11-06

## Why can't ChatGPT access my website?

OpenAI runs three crawlers. OAI-SearchBot finds pages for ChatGPT search. GPTBot collects training data. ChatGPT-User fetches a page when a person asks ChatGPT to open it. All three need your server to answer a plain request with the page. Perplexity and Anthropic crawlers work the same way.

| What the crawler sees | Likely cause | Where to fix it |
| --- | --- | --- |
| HTTP 403, 429 or 503, often with a small body | Firewall or bot protection | [Firewall and Cloudflare fix](https://seenlens.com/fixes/ai-crawlers-blocked-by-waf) |
| HTTP 200 and an empty page | Content needs JavaScript | [JavaScript fix](https://seenlens.com/fixes/content-needs-javascript) |
| HTTP 500 to 599 | Server error or timeout | Section below |
| HTTP 404, or a redirect to a login page | Wrong URL, or a page behind a login | Section below |
| Page loads, but the crawler stays away | A robots.txt rule | [robots.txt fix](https://seenlens.com/fixes/robots-txt-blocks-ai-crawlers) |

## page-reachable: can a plain request load the page?

### What this check tests

SeenLensBot sends one GET request for your page. It uses a browser style user agent that ends with the `SeenLensBot` token, follows redirects and runs no JavaScript. The check passes when the answer is below HTTP 400 and no bot challenge appears. It fails when the request times out, returns HTTP 400 or higher, or returns a challenge page from a firewall.

### How to confirm the problem

Run this command. It prints the final status code, the size of the answer and the number of redirects. A small size next to a 403 or 503 points to a firewall.

```text
curl -sS -o /dev/null -L -w "status %{http_code}, %{size_download} bytes, %{num_redirects} redirects\n" https://example.com/your-page
```

Then repeat it with `-A "OAI-SearchBot"` and compare. The embedded AI crawler checker above does the same comparison for several crawlers.

### How to fix it

- **HTTP 403, 429 or 503:** a firewall or bot rule blocks plain clients. Follow the [firewall fix](https://seenlens.com/fixes/ai-crawlers-blocked-by-waf).
- **HTTP 500 to 599:** read your server error log for the same time. Fix the failing code, database call or upstream service.
- **Timeouts:** the origin answers too slowly. Check server load, cold starts and slow database queries.
- **HTTP 404:** the URL is wrong or the page moved. Link to the live URL and add a 301 redirect from the old one.
- **Redirect to a login page:** a crawler cannot sign in. Public pages must open without a login.

## http-status: does the page return status 200?

### What this check tests

| Result | What SeenLens saw |
| --- | --- |
| Pass | Status 200 after at most two redirects |
| Warn | A status below 400 that is not 200 (for example 202 or 204), or more than two redirects |
| Fail | Status 400 or higher, or no status at all |

Google follows up to 10 redirect hops and treats 301 and 308 as strong signals. Other crawlers may stop sooner, so a short chain is safer.

### How to fix it

Link to the final URL and send each old address to it in one hop. This Nginx example sends HTTP and www requests straight to the HTTPS apex URL:

```text
server {
    listen 80;
    server_name example.com www.example.com;
    return 301 https://example.com$request_uri;
}
```

In Apache, use `Redirect permanent "/" "https://example.com/"` inside the virtual host. In Next.js, set `permanent: true` on a redirect in `next.config.js`; it sends status 308.

## How to verify the fix

1. Run the curl command again. Expect status 200 and at most one redirect.
2. Run the [AI crawler checker](https://seenlens.com/tools/ai-crawler-checker) on the page. The status codes for each crawler should match the browser.
3. Check your access log for the crawler names. On a combined log format, `grep -E "OAI-SearchBot|GPTBot|ChatGPT-User" access.log | awk '{print $9}' | sort | uniq -c` counts the status codes they received.

[Run the free SeenLens scan on your page](https://seenlens.com/)

## When can you ignore it?

Ignore a failure on a page you want to keep private, such as a staging site, an admin page or a page behind a login. A crawler that cannot load those pages is the correct result.

## Related checks

- [Cloudflare and firewall blocks](https://seenlens.com/fixes/ai-crawlers-blocked-by-waf)
- [Content that needs JavaScript](https://seenlens.com/fixes/content-needs-javascript)
- [robots.txt blocks AI crawlers](https://seenlens.com/fixes/robots-txt-blocks-ai-crawlers)
- [ChatGPT-User](https://seenlens.com/bots/chatgpt-user) and [OAI-SearchBot](https://seenlens.com/bots/oai-searchbot)

## Sources

- [OpenAI: crawlers and user agents](https://developers.openai.com/api/docs/bots) (OAI-SearchBot, GPTBot and ChatGPT-User roles, IP lists, checked 2026-11-06)
- [Google Search Central: HTTP status codes and network errors](https://developers.google.com/search/docs/crawling-indexing/http-network-errors) (redirect hops, 4xx and 5xx handling, checked 2026-11-06)
- [nginx: return directive](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html) (redirect syntax, checked 2026-11-06)
- [Apache: mod_alias Redirect](https://httpd.apache.org/docs/2.4/mod/mod_alias.html) (permanent redirect syntax, checked 2026-11-06)
- [Next.js: redirects](https://nextjs.org/docs/app/api-reference/config/next-config-js/redirects) (permanent redirects send status 308, docs updated 2026-06-30)