This is the complete specification for our Agent Readiness Auditor: what it checks, what each check is worth, how the total is calculated, and which things we deliberately refuse to score. You should be able to reproduce any result by hand from this page.
What existing scanners get wrong
Public agent-readiness scanners already exist. We ran a production marketing site through one in July 2026 and audited every check it made. The verdict was fair warning, wrong panic.
The scanner weighted draft-stage agent infrastructure heavily — DNS-AID records, OAuth discovery endpoints, MCP server cards, agentic-commerce protocols — none of which any marketing site has, and none of which any AI assistant consults before reading or recommending a site. Meanwhile the things that demonstrably feed AI answers today were under-weighted, and several were not checked at all.
A site owner reading that report would spend a fortnight implementing protocols with no consumer, and never discover that their CDN was returning a challenge page to ClaudeBot.
The weighting principle
In one sentence: score what agents use today, report what exists only in drafts, and show the evidence for every point.
Which means the tool has to be able to tell you you are fine, ignore the panic as readily as you are invisible to agents, here is why. A scanner that can only produce alarm is a marketing instrument, not a diagnostic.
Assumptions behind this method
- Every check is deterministic. Same site, same result, no model involved anywhere in the scoring.
- We check whether a site is readable, not whether it is good. Content quality is out of scope.
- Bot User-Agent strings identify themselves honestly. A site that serves different content to a spoofed UA than to the real crawler will read differently to us than to the crawler.
- Point values are our editorial judgement about what matters today, not an industry standard. They are published here precisely so you can disagree with them.
- The scanner respects robots.txt for its own token. If you disallow us, we read only your robots.txt, llms.txt and sitemap, and report no score.
Every check and its points
A. Access & crawlability — 30 points
If bots cannot fetch you, nothing else matters. We do not trust robots.txt declarations — we fetch with real bot User-Agents and compare.
| ID | Check | Logic | Pts |
|---|---|---|---|
| A1 | robots.txt exists and parses | 200, parseable, and not an HTML page served with a 200 status. | 4 |
| A2 | AI crawlers allowed in robots.txt | Evaluates can_fetch("/") for 14 tokens: GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, Claude-SearchBot, anthropic-ai, PerplexityBot, Perplexity-User, Google-Extended, CCBot, Amazonbot, meta-externalagent, DuckAssistBot. Score is the proportion allowed. | 10 |
| A3 | Live fetch parity under AI user-agents | Fetches the homepage as a browser, then as GPTBot, ClaudeBot, PerplexityBot and Googlebot. Passes per bot when the status matches and content length is within 15%. This catches edge blocking that robots.txt never shows. | 10 |
| A4 | HTTPS and redirect hygiene | http upgrades to https, and the canonical host resolves without a chain. | 3 |
| A5 | Missing pages return a real 404 | A random missing path must return 404 or 410, not a soft 200. | 3 |
B. Machine-readable content — 25 points
Agents read the same HTML crawlers do. Content that only exists after JavaScript runs is invisible to most answer engines.
| ID | Check | Logic | Pts |
|---|---|---|---|
| B1 | Content exists without JavaScript | Visible text extracted from raw HTML. 400+ words passes, 150–399 warns, under 150 fails as a shell. | 10 |
| B2 | Title, meta description, one H1 | All three present. A title over 60 characters warns rather than fails. | 5 |
| B3 | Semantic structure | A main or article landmark, and heading levels that do not skip. | 4 |
| B4 | Markdown content negotiation | Accept: text/markdown returns a non-HTML structured response. Rewarded, never punished — absence scores zero rather than failing. | 3 |
| B5 | Content page depth | A discovered content page carries at least 300 raw-extractable words. | 3 |
C. Structured data & entity — 20 points
Schema is how agents work out who you are and what a page is, and it is where sites most often get confused with a similarly-named competitor.
| ID | Check | Logic | Pts |
|---|---|---|---|
| C1 | JSON-LD present and valid | At least one ld+json block that parses. Invalid JSON-LD is discarded silently by consumers. | 5 |
| C2 | Organization or WebSite entity | An Organization, NGO or WebSite node with a name and url. sameAs links are noted as supporting evidence. | 5 |
| C3 | Content-type schema | Article, BlogPosting, FAQPage, Product or similar on the content page. | 4 |
| C4 | Schema matches visible content | If FAQPage is declared, the question strings must appear in visible text. A mismatch fails — Google has treated this as a spam policy violation since May 2026. | 3 |
| C5 | Canonical link | An absolute, self-referencing rel=canonical. | 3 |
D. Agent affordances — 15 points
The files that exist specifically so machines can find their way around.
| ID | Check | Logic | Pts |
|---|---|---|---|
| D1 | llms.txt | Exists, is markdown, has at least 3 absolute links, under 100 KB. Labelled honestly: a proposed standard adopted by tooling, not ratified. | 6 |
| D2 | Sitemap and lastmod sanity | A sitemap that parses, is declared in robots.txt, and whose lastmod values are not all identical. Generation-stamped lastmod is worse than none. | 4 |
| D3 | Content-Signal and X-Robots coherence | A Content-Signal line in robots.txt, and no contradicting noindex header. | 2 |
| D4 | llms.txt discoverability | A Link rel="describedby" header, or a robots.txt reference. | 1 |
| D5 | Feed autodiscovery | An RSS or Atom autodiscovery link that resolves. | 2 |
E. Performance & hygiene — 10 points
Agents run on short timeouts and abandon slow responses.
| ID | Check | Logic | Pts |
|---|---|---|---|
| E1 | Time to first byte | Under 1.5s passes, under 3s warns. | 4 |
| E2 | Raw page weight | HTML under 1 MB passes, under 3 MB warns. | 2 |
| E3 | Compression | brotli, gzip or zstd on HTML responses. | 2 |
| E4 | Homepage is indexable | No noindex in meta or headers. | 2 |
How the score is calculated
A weighted sum over applicable checks, expressed out of 100. A warn earns half the points, rounded down. An info earns nothing either way and never counts against you.
Checks that do not apply are removed from the denominator rather than scored as zero. A site with no shop is not penalised for lacking commerce signals, and a scan that we were not permitted to complete produces no score at all instead of a low one.
| Score | Band | Meaning |
|---|---|---|
| 85–100 | Agent-ready | Agents can read this site. |
| 70–84 | Good | Agents can read most of this site. |
| 50–69 | Partial | Agents can read some of this site. |
| 30–49 | Limited | Agents struggle with this site. |
| 0–29 | Not agent-readable | Agents cannot meaningfully read this site. |
The sentence printed under every score: This measures whether agents can read your site — not whether they will cite it. In our 23-prompt study, AI answers overwhelmingly cited pages that already ranked in classic search. Readiness is the floor, not the ceiling.
The watch list
A sixth category is probed, reported, and worth zero points: DNS-AID and
_agent TXT records, /.well-known/oauth-authorization-server,
/.well-known/mcp.json, agents.json, and Web Bot Auth signature headers.
Each is shown as found or not found with a one-line maturity note. These are IETF drafts and proposals without consumer implementations. Telling a site owner they have "failed" them manufactures urgency about work with no payoff, which is the failure mode this tool exists to correct.
When any of them leaves draft status and gains a real consumer, it moves into the scored categories and this page changes to say so.
Our own score
We ran the auditor against this website while building it. The first run scored 75 out of 100 — Good — with full marks for crawler access but 1 out of 15 for agent affordances, because we had published no llms.txt, no feed and no Content-Signal line. It also found that our sitemap sat at a non-conventional path and that our own articles carried no page-level schema.
We fixed all of it and re-ran the deployed auditor against the live site on 17 August 2026: 89 out of 100 — Agent-ready — with agent affordances at 15 out of 15 and no failing checks. What remains, published rather than excused: markdown content negotiation we have not implemented; organisation pages that carry no Article schema because they are not articles; and a compression reading the scanner cannot see when it audits a site on its own network, although external clients receive brotli.
Publishing both numbers is the point. A tool whose authors quietly exempt themselves is worth less than no tool, and a score that only ever appears after the fixes proves nothing. We will not backdate the record.
Limits of this method
The auditor measures readability, and readability is necessary rather than sufficient. It cannot tell you whether an assistant will cite you, what it will say, or how you compare with a competitor — those depend on content quality, authority and ranking, none of which this measures.
It also sees one moment from one network location. Edge configuration varies by region, and a site that serves us correctly may behave differently elsewhere. Where a result surprises you, the evidence line under each check is the actual header or measurement we received, so you can reproduce the fetch yourself and compare.
Found an error? Tell us and show your working — we correct in place and record what changed. Contact us.