# AgentReady — full content > AgentReady scans any public URL and scores how easily AI agents can read, navigate, and act on that site. ## What AgentReady checks AgentReady evaluates five categories. Discoverability (robots.txt, AI crawler access, sitemap, canonical). Agent instructions (llms.txt, llms-full.txt, meta description, page title). Machine-readable structure (JSON-LD, Open Graph and Twitter cards, heading hierarchy, semantic landmarks, server-side rendering). Actionability (form labels, accessible button names, CAPTCHA usage, public API discovery, navigable nav). Trust and freshness (HTTPS enforcement, Last-Modified or dateModified, image alt text coverage, security headers). Each check returns pass, warn, fail, or info with the evidence found on your site and a copy-paste fix. ## Scoring Each category is scored 0-100 from its checks, then combined using weights: Discoverability 20, Agent Instructions 20, Machine-Readable Structure 20, Actionability 25, Trust and Freshness 15. The final score is 0-100. 75 or above is Agent-ready, 50-74 is Partially readable, below 50 means agents will struggle. ## Privacy Scans cache for 24 hours keyed by normalized domain. No personal data is stored. Rate limits: 5 scans per hour, 20 per day per IP. ## What is llms.txt? llms.txt is a markdown file served at the root of your domain (https://your-domain.com/llms.txt) that gives AI agents a curated, structured summary of your site: what it is, what is on it, and which pages matter most. It follows a convention described at llmstxt.org: one H1 title, a blockquote summary, then H2 sections containing markdown links. ### Why llms.txt exists AI agents do not browse your site the way a person does. They fetch pages, keep only what fits in a limited context window, and guess at what matters. A homepage clogged with navigation, cookie banners, and marketing copy wastes that budget. llms.txt hands the agent a clean table of contents so its first fetch already points at your best pages. ### The format The file is plain markdown with three required parts. First, an H1 line with the name of your project or site. Second, a blockquote (a line starting with >) summarizing what the site is in one paragraph. Then any number of H2 sections, each containing markdown links with optional descriptions after a colon. The last section is conventionally called "Optional" for material agents can skip. ### Where it goes Serve it at /llms.txt on your apex domain. No special headers are required — text/plain is fine. A companion file, /llms-full.txt, can hold the full text of your documentation in one file for agents with larger context windows. ### How to write a good one Lead with what the site does, not who you are. Link to the pages an agent would actually need: docs, pricing, API reference, contact. Write the link descriptions as if the agent will never click through — because sometimes it will not. Keep the whole file under a few kilobytes; it is a map, not a dump. --- ## How AI agents read websites AI agents read websites by fetching the raw HTML over HTTP — without executing JavaScript — and converting it into plain text, a list of links, and a map of form fields. They rely on the same signals a screen reader uses: semantic landmarks, heading hierarchy, accessible names, and metadata like JSON-LD. If your content only appears after JavaScript runs, most agents see an empty page. ### The fetch pipeline An agent requests your page, receives HTML, strips scripts and styles, and keeps the text. Then it extracts links with their anchor text, images with their alt text, and form fields with their labels. Anything that required JavaScript to render does not exist yet. The agent decides what to do next based entirely on this flattened view. ### What agents struggle with Client-side rendered apps that ship an empty body. Icon-only buttons with no accessible name. Unlabeled form fields. Pages that hide their main content behind a login wall without saying so in text. Infinite scroll with no real links. All of these are invisible or ambiguous in the flattened view. ### What helps most Server-render your primary content. Use real semantic HTML: header, nav, main, footer, one h1, logical heading levels. Give every interactive element an accessible name. Publish llms.txt and JSON-LD so the agent does not have to infer structure. Expose real links instead of click handlers. --- ## The agent-readiness checklist Agent readiness comes down to five categories: discoverability (robots.txt that allows AI crawlers, a sitemap, a canonical URL), agent instructions (llms.txt, meta description, page title), machine-readable structure (JSON-LD, Open Graph, one h1, semantic landmarks, server-rendered content), actionability (labeled forms, named buttons, a navigable nav), and trust (HTTPS, freshness signals, alt text). Work through them in that order. ### 1. Discoverability Publish robots.txt. Make sure GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Bingbot, and CCBot are not disallowed from the pages you want read. Reference your sitemap from robots.txt. Add a canonical link to every page. ### 2. Agent instructions Add /llms.txt with a title, a summary, and sections of links. Add a meta description of 70-160 characters and a page title of 30-60 characters on every page. Optionally add /llms-full.txt with your full documentation. ### 3. Machine-readable structure Add JSON-LD for Organization and WebSite at minimum; add Article, Product, or FAQPage where relevant. Include Open Graph and Twitter card tags. Use exactly one h1 with no skipped heading levels. Wrap the page in header, nav, main, and footer. Ensure the primary content is in the raw HTML, not added by JavaScript. ### 4. Actionability Label every form field with a label element or aria-label. Give every button and link an accessible name. Keep CAPTCHAs off primary public forms where possible, or offer an alternative path. Link to your API docs or OpenAPI spec. Keep at least three internal links in your nav. ### 5. Trust and freshness Redirect all HTTP traffic to HTTPS. Expose Last-Modified or a dateModified property. Provide alt text on every image. Set Content-Security-Policy and Strict-Transport-Security headers. --- ## JSON-LD for AI JSON-LD is a JSON block embedded in a script tag that describes your page using schema.org vocabulary: what kind of thing it is (an article, a product, an organization), its name, dates, and relationships. AI agents parse it directly from the raw HTML, so it works even when they skip your JavaScript. Put it in your page head and validate it with Google's Rich Results Test. ### Why agents prefer JSON-LD Reading prose is ambiguous. Reading "@type": "FAQPage" with a list of questions and answers is not. Structured data removes the guesswork from entity extraction, which means fewer wrong citations and better answers about your site. ### The types that matter Organization and WebSite sitewide. Article or BlogPosting for content, with dateModified so agents know how fresh it is. Product for commerce. FAQPage for question pages. BreadcrumbList for site structure. SoftwareApplication for tools and apps. ### A minimal template One script tag in the head is enough to start. Keep the JSON valid — a single trailing comma makes the whole block unparseable, which is one of the most common failures the scanner sees. ### Validating Paste your URL into the Rich Results Test or Schema.org validator. Then run your domain through AgentReady — the JSON-LD check reports exactly which types it found and whether each block parsed. --- ## Bot AgentReadyBot fetches your robots.txt, llms.txt, llms-full.txt, sitemap.xml, and your homepage HTML. It sets the User-Agent "AgentReadyBot/1.0 (+https://agenticai.vin/bot)". It respects robots.txt and never executes JavaScript. To block it, add "User-agent: AgentReadyBot / Disallow: /" to your robots.txt.