Three plain files decide what AI crawlers can read on your site. Here is the 30-minute setup, checked on November 11, 2026 against what OpenAI, Anthropic, Google and Perplexity say on their own pages.
Open your robots.txt, search it for GPTBot, and look for a bare Disallow: / under User-agent: *. A snippet pasted from a 2024 "protect your content" post can still be turning away the crawlers that fetch pages for AI answers, and you only find out by looking.
Short answer: 3 files: robots.txt (so the crawler, a computer robot that automatically requests web pages, knows if it can crawl or not). Your sitemap of your pages. llms.txt is an optional short description for the AIs. Copy/paste for every config as it is, it would take you about 30 minutes or so.
This checklist comes from RankControl, an AI SEO and AEO platform. On November 11, 2026 we reread the crawler pages from OpenAI, Anthropic, Google and Perplexity against it, so the file below follows what those companies document today.
The file and the full checklist are in the write-up on Medium.