# AI Readiness Checker for AI Search and Agents

Audit a URL for AI-search and agent readiness, as a checklist with evidence (not a score). Fetches the page, robots.txt, the sitemap and a few well-known files (llms.txt, the API catalog, an MCP server card, Markdown for agents) and returns checks grouped as access, content, discovery and agents, each with pass, warn, fail or info, what was seen, where, and how to fix it. Includes a crawler matrix (GPTBot, ClaudeBot, PerplexityBot and others: allowed or blocked, and whether blocked by choice) and a level word with counts: Ready, Mostly ready, Partly ready or Not ready (Unclear when too little could be read). If the site refuses us, blockedUs is true. A crawler blocked on purpose never lowers the level. Reads public pages only, about 6 to 10 requests.

**Not for:** Ranking or traffic prediction: nothing here is a ranking factor we can promise.

- Tool id: `ai-readiness-check`
- Category: web
- Execution: network
- MCP tool `ai_readiness_check` at https://lookupowl.com/mcp
- REST `GET https://lookupowl.com/api/v1/tools/ai-readiness-check` (or POST a JSON body)
- OpenAPI: https://lookupowl.com/api/openapi.json

## Input

| Field | Type | Required | Default | Description |
|---|---|---|---|---|
| `url` | string | yes |  | A website or a page address, such as lookupowl.com. We read the page, robots.txt, the sitemap and a few well-known files. Only http and https on the usual ports. min length 1, max length 2048 |

## Output

Results are wrapped in an envelope: `tool`, `version`, `requestId`, `data`, `summary`, `meta` (cache status, timing), `permalink`. `data` has these fields:

| Field | Type | Description |
|---|---|---|
| `target` | string |  |
| `finalUrl` | string |  |
| `redirectedTo` | string \| null | The host the address sent us to, when it is not the one typed. |
| `blockedUs` | boolean | The site refused to let us read it, so we cannot say what AI crawlers see. |
| `blockedStatus` | integer | The HTTP status the site answered when it refused us. |
| `summary` | object |  |
| `groups` | object[] |  |
| `crawlerMatrix` | object[] |  |
| `llmsTxt` | object | The llms.txt section the /llms-txt-checker page leads with. |
| `page` | object |  |
| `notes` | string[] |  |
| `partial` | boolean |  |

## Examples

### lookupowl.com, built for agents

```json
{"url":"lookupowl.com"}
```

```bash
curl 'https://lookupowl.com/api/v1/tools/ai-readiness-check?url=lookupowl.com'
```

Web page: https://lookupowl.com/ai-readiness-checker?q=lookupowl.com

### A page address

```json
{"url":"https://lookupowl.com/learn"}
```

```bash
curl 'https://lookupowl.com/api/v1/tools/ai-readiness-check?url=https%3A%2F%2Flookupowl.com%2Flearn'
```

Web page: https://lookupowl.com/ai-readiness-checker?q=https%3A%2F%2Flookupowl.com%2Flearn

## Web pages

- https://lookupowl.com/ai-readiness-checker
- https://lookupowl.com/llms-txt-checker

## Limits and caching

- Cost: 10 units for a live lookup, 1 unit for a cached one. Budgets are per network and published in the `RateLimit` response headers.
- Cache: results are kept for 1800 s.
- Timeout: 30 s.

## Related tools

- [HTTP Headers Checker](https://lookupowl.com/tools/http-headers.md): Fetch a URL, follow its redirects and see every response header, with a security-header grade and caching summary.
- [Security Headers Checker](https://lookupowl.com/tools/security-headers.md): Grade a site's HTTP security headers from A+ to F, with what each one does and the exact header to add.
- [Is It Down? Check a Website from Many Places](https://lookupowl.com/tools/http-check-global.md): Check a website from probes in many places: name lookup, connection, certificate and web server, with a plain verdict.
- [SSL Certificate Checker](https://lookupowl.com/tools/ssl-checker.md): Check a site's SSL/TLS certificate: days until it expires, whether it is trusted, matches the host and has a complete chain.
- [Who Hosts This Website](https://lookupowl.com/tools/website-hosting.md): Find the web host or CDN, DNS provider and email provider behind a domain, with the evidence, and whether the origin is hidden.

## Guide: AI readiness checker

Find out whether AI search engines and assistants can reach and read your page, as a checklist with the evidence for every line and no invented score.

### What the checker looks at

AI search and AI assistants read the web differently from people. Most of them fetch the HTML your server sends and do not run your scripts, they obey robots.txt, and they lean on plain structure: a title, a description, headings, links and a sitemap. This checker fetches your page the way such a crawler would, then reads the few files that tell crawlers and agents more, and reports what it found in five groups of questions.

- **Can AI read it?** Whether the page answers, whether robots.txt exists, which AI crawlers it lets in, any Content-Signal line, and page-level directives such as noindex.
- **What does it see before JavaScript?** The text, the main heading and the links in the first response. A page that builds itself in the browser can look empty to a crawler.
- **Does it describe itself?** The title, meta description, canonical link, Open Graph title and structured data (JSON-LD).
- **Can it be found?** A sitemap, an llms.txt file and how fast the first byte arrives.
- **Can an agent use it?** A Markdown version of the page, an API catalog, an MCP server card and Link headers that point to them.

Each line shows what we saw and where we looked, and for most gaps it gives the sentence to fix it and a snippet to paste. We make eight or nine requests to the site itself, in one short burst, and none to anyone else.

### How to use it

1. **Enter a site or a page**: Type `example.com` or a full address. We follow redirects and check the page you land on, and we say when that is a different host from the one you typed.
2. **Read the level, then the groups**: The answer at the top is a word with counts. Below it, five tiles show which group set the level. Open the one that is not green.
3. **Work through the checklist**: Failures come first, then warnings, passes and information. Open a row to see where we looked, how to fix it and a snippet to copy. Run the check again after you deploy.

```bash
# The same check as JSON, for a script or an agent
curl "https://lookupowl.com/api/v1/tools/ai-readiness-check?url=example.com"
```

> **From an AI assistant** With the LookupOwl MCP server connected, ask "Is example.com ready for AI search?" The assistant calls `ai_readiness_check` and answers from the checklist. See [using LookupOwl from AI agents](https://lookupowl.com/learn/lookupowl-mcp-ai-agents).

### How the level is worked out

There is no score here, because a number would need weights, and any weights would be a guess. The level is a word, and it comes from one rule that you can check by hand: it is the lowest result among the five groups. Checks are counted, never weighted. Information lines are not counted for or against anything.

| Level | When it applies |
|---|---|
| Ready | Every group has at least one pass and no warnings or failures. |
| Mostly ready | No group fails, but at least one has a warning. |
| Partly ready | No group fails, but at least one has nothing that passes yet. Optional agent features that are missing do this. |
| Not ready | At least one group has a failure, such as a page with almost no text before JavaScript. |
| Unclear | Fewer than three groups could be read, for example because the site was slow or refused us. |

### Blocking AI crawlers is a choice, not a defect

Many publishers disallow AI training crawlers such as GPTBot or CCBot in robots.txt on purpose. This checker shows that as "Blocked by choice", marks each such crawler in the crawler tab, and never lets it lower the level. What it does tell you is what the choice costs: blocking a search crawler such as OAI-SearchBot keeps the page out of that assistant's search results, and blocking an assistant such as ChatGPT-User stops it fetching the page when a person asks.

A block that comes from the rule for every crawler, not from a line naming the bot, is different. That often means a staging setting that was never removed, so it is shown as a warning with the lines to allow the crawlers you want.

### What most sites need to fix first

- Put the main content in the first response. Server-render or prerender pages that are built in the browser.
- Give every page a title, a description, a canonical link and one clear h1.
- Publish a sitemap and name it in robots.txt.
- Decide on purpose which AI crawlers you welcome, and say so in robots.txt, with a Content-Signal line if you like.
- Optional extras for agents: a Markdown version of your pages, an API catalog and an MCP server card.

None of this is a ranking factor we can promise, and the page says so. It is the list of things that decide whether a crawler can read you at all. For the security side of the same page, try the [security headers checker](https://lookupowl.com/security-headers), and to see every header the page sends, the [HTTP headers checker](https://lookupowl.com/http-headers-checker).

### Frequently asked questions

**What is AI readiness?**

It is whether AI search engines and assistants can find your page, read what is on it and understand what it is. That depends on crawler access in robots.txt, on content being in the first response, on basic metadata and structured data, and on a sitemap. For agents, Markdown versions and API descriptions help too.

**Will a better level improve my ranking or get me cited?**

We cannot promise that, and nothing on this page claims it. The checklist covers the preconditions: a crawler that cannot reach or read a page cannot use it. Whether an assistant then cites it is up to that assistant.

**Why does it say "Blocked by choice" and not a failure?**

Because disallowing AI crawlers by name in robots.txt is a legitimate decision about your content. We show it so you can confirm it is what you meant, and we leave the level alone. Only a failure, a warning or a group with nothing passing changes the level.

**Do AI crawlers run JavaScript?**

Most fetch the HTML and do not run scripts, which is why the text in the first response matters. Some crawlers do render pages, so a script-built page may still be read by them, but you cannot count on it. The "What AI sees" tab shows the first 600 characters a crawler that does not run scripts reads.

**Does this tool pretend to be an AI crawler?**

No. It reads robots.txt and applies the rules for each crawler name, but it fetches your pages as LookupOwl, with our own user agent. We never send another crawler's name, so what you see in your logs is a single, identifiable visitor.

**Why did it say it could not check something?**

A file did not answer in time, or the connection failed. That one line is marked "Couldn't check" and counted apart, and the rest of the list is complete. If a site refuses our request entirely, we tell you that instead of guessing.