The channel your existing playbook half-covers
Most of what makes a page rank also helps it get cited, but not all of it, and the gaps are the expensive part. Crawler access is governed by different user agents, extraction favours different content shapes, and there is no rank position to report. Teams that treat AEO as a strict subset of SEO tend to plateau exactly where the differences begin.
This page is for the team that already owns technical SEO and content and wants to add answer engines without running a second programme. It covers where the two disciplines diverge, how to fold the new checks into an audit you already run, and what to put on the monthly report when there is no position 1 to point at.
Why SEO teams add this layer
Different crawlers, different rules
Googlebot access says nothing about the agents answer engines actually use. OAI-SearchBot and ChatGPT-User fetch pages for ChatGPT answers, PerplexityBot and Perplexity-User do the same for Perplexity, and ClaudeBot for Claude. GPTBot, CCBot and Google-Extended are training crawlers with a different job. A robots.txt that allows Googlebot and blocks everything else, or a WAF rule that challenges unknown agents, leaves a perfectly ranked site invisible to the answer layer. That is a configuration check rather than a strategy question, and it is where most teams find their first problem.
Rendering matters more, not less
Google renders JavaScript eventually and indexes what it finds. Several AI retrieval agents fetch the raw HTML and move on. A page that assembles its body copy, its FAQ block and its structured data after hydration is a complete page for Google and an empty shell for them. The fix is usually server-side rendering or pre-rendering for the templates that matter, and it is worth verifying with a non-JavaScript fetch before assuming the framework handles it.
The reporting metric changes
There is no rank position in a synthesised answer. What you can measure is whether the brand is mentioned at all for a given prompt, whether your page is cited as a source, how you are described, and how that compares with named competitors. Those numbers only mean something against a fixed prompt set tracked over time, so the baseline has to exist before anyone can report progress. The Visibility Tracker at /features/brand-visibility-tracker captures mention rate per engine.
The winning sources are often not you
Citation data for a category usually shows a mix of community threads, review sites and comparison posts alongside vendor pages. An SEO team is used to competing against other domains for a ranking; here you are competing to be one of a handful of sources the model trusts for a claim. That changes content priorities. Consolidating thin pages into one definitive answer beats publishing more of them, and off-site work becomes part of the on-site brief.
How AEO fits an existing SEO programme
The additions to a technical and content workflow that already exists, in the order that finds problems cheapest first.
- 1
Audit AI crawler access separately
Pull robots.txt from the live edge, not from the repo, because CDNs and bot-management products inject managed rules above your own. Test each agent by name: OAI-SearchBot, ChatGPT-User, PerplexityBot, Perplexity-User and ClaudeBot for retrieval; GPTBot, CCBot, Google-Extended and Applebot-Extended for training, which you may choose to treat differently. Then check server or WAF logs for 403 and 429 responses to those user agents, because a rule that allows a crawler in robots.txt and blocks it at the firewall is common. Use /tools/robots-txt-validator to catch syntax errors and contradictory groups.
- 2
Verify server-rendered output
Fetch your top templates with curl or a plain HTTP client and read the body. Confirm the H1, the body copy, any FAQ block and the JSON-LD are present in the response, not injected by a client bundle. Do this for the category page, the article template, the comparison template and the pricing page, because those are the pages answer engines pull from. Where the body is missing, the fix is server-side rendering or static generation for that template. Prioritise by citation potential rather than by traffic, since the two rankings differ.
- 3
Extend structured data coverage
Add FAQPage where a page genuinely answers discrete questions, HowTo where it describes a procedure, Article with author and dateModified on everything editorial, and Organization with sameAs on the home page. Go beyond what rich results reward: the point is to mark self-contained answers a model can lift with attribution. Generate and check the output with /tools/json-ld-generator and /tools/json-ld-validator, and make sure the schema describes what is actually on the page, because mismatches are treated as noise at best.
- 4
Publish an llms.txt and keep it honest
Write a short llms.txt that lists the pages that best answer your category questions, with a one-line description each, and leave out promotional pages. Treat it as a curated index for retrieval agents rather than a sitemap. Support for the file varies by engine, so it is an inexpensive addition rather than a guarantee, but it costs almost nothing and gives you a place to point crawlers at your definitive pages. Draft it with /tools/llm-txt-generator and check it with /tools/llm-txt-validator.
- 5
Restructure for extraction
Edit existing pages rather than rewriting them. Open each section with a direct answer in one or two sentences, then the supporting detail. Keep one claim per passage and make sure it makes sense with no surrounding context, because that is how it will be quoted. Merge overlapping thin pages that split the same intent into one page with depth. Add a short definition block for the terms your category uses. This is an editing discipline the content team already has; it just needs the extraction test applied to it.
- 6
Report on the AEO metric set
Fix a prompt set of the category, comparison and problem questions your buyers ask, and run it on a schedule against each engine. Report mention rate and citation share per engine next to rankings and clicks, with the cited-source list as the action item. Recover AI-referred sessions from Direct with /features/ai-traffic-decoder so the channel carries a traffic number. Keep the prompt set stable month to month; changing it resets the trend line, and the trend line is what leadership remembers.
Where SEO teams start
Technical AEO audit
Crawler access, rendering and structured data coverage, checked per template and per agent. This is the fastest work to verify and the most likely to produce a visible change, because a site that could not be read cannot be cited. Package it as a bounded project with a checklist output.
Content refresh prioritisation
Rank existing pages by how close they are to being citable rather than by traffic. A page with strong topical coverage but a buried answer needs an edit; a page cited by one engine but not another needs a rendering check. The gap list from citation data tells you which to touch first.
Cannibalisation cleanup
Thin, overlapping pages split the same intent and none of them reaches the depth a model prefers. Consolidate into a single page with the answer up front and the variants covered underneath, and redirect the rest. This usually helps rankings too, which makes it easy to justify.
Competitor and source analysis
See which domains each engine cites for your category questions and which competitors get named alongside you. The list shapes both the on-site brief and the off-site outreach, and it is the part of the report a marketing lead will actually read. Citation Intelligence at /features/citation-intelligence produces it.
Migration and replatform checks
Before a framework or CMS change ships, confirm the new templates still server-render body copy and JSON-LD, and that the new edge configuration has not introduced bot rules. Answer engine visibility is easy to lose in a migration and slow to recover.
What differs per engine for an SEO team
The same technical foundation, with a different access check and a different selection bias on each surface.
ChatGPT
Search-style answers fetch live pages through OAI-SearchBot and ChatGPT-User, so access for those two agents is the first check, separately from GPTBot. Citations appear as source links, which makes citation share measurable. Answer-first sections and clear headings help because retrieval works on passages, not whole pages. Check rendering carefully; the fetch does not wait for a client bundle.
Perplexity
Perplexity is the most citation-forward of the engines and shows sources prominently, so it is the fastest feedback loop for whether a rewrite worked. PerplexityBot indexes and Perplexity-User fetches at query time; allow both. It leans on recently updated pages and specific, well-labelled detail, so a dateModified that is real and a definition block that is precise both help.
Google AI Overviews and AI Mode
Both draw on the regular Google index via Googlebot, so existing crawlability carries over and Google-Extended is not the switch. What changes is selection: pages with a direct, well-structured answer to the sub-questions a query is split into are favoured. Search Console data is still relevant, but the overview can cite pages that do not rank on the classic results page.
Gemini
Gemini grounds answers with Google Search, so the same index applies, with the added factor that Google-Extended governs whether your content is used in Gemini training and grounding beyond search. If you have blocked it for policy reasons, expect thinner coverage here. Structured data and entity clarity, meaning a consistent Organization record with sameAs, shape how Gemini describes the brand.
Microsoft Copilot
Copilot grounds on the Bing index, so Bingbot access and Bing Webmaster Tools coverage are the levers, and a site that has only ever been tuned for Google can have gaps. IndexNow gives faster pickup of updated pages. Citation behaviour is similar to ChatGPT in search mode, with sources shown as links, so the same passage-level structure applies.
The numbers an SEO team reports
Each replaces something on the rankings report rather than duplicating it.
- Mention rate
- The share of tracked prompts where the brand is named in the answer, per engine. Read it as reach: it tells you whether you are in the conversation at all. Movement is slow and lumpy, so compare month to month on the same prompt set rather than week to week.
- Citation share
- Of the sources an engine cites for your prompt set, the fraction that are your pages. This is the metric closest to the technical work: when access, rendering and structure improve, citation share moves before mention rate does. Break it down by template to see which page types earn citations.
- Cited-source gap
- The list of domains cited for your prompts that are not yours, ranked by frequency. It is a to-do list more than a KPI: each entry is either a page you should be competing with on-site or a source you should be present on off-site.
- AI-referred sessions
- Sessions recovered from Direct and attributed to answer engines, plus their conversion rate. It is the number that connects the programme to revenue, but it under-counts influence that never produces a click, so present it as a floor rather than the total.
Mistakes SEO teams make on the first pass
Mistake · Testing crawler access against the robots.txt in the repo instead of the served file.
Fix · Fetch robots.txt from the live edge and compare it with the source. CDN bot management, WAF rules and hosting defaults add or override groups, and the served file is what the crawler actually sees.
Mistake · Blocking every non-Google agent with one wildcard to stop training scrapers.
Fix · Name the training crawlers you want to block individually and leave the retrieval agents open. A wildcard that catches OAI-SearchBot or PerplexityBot removes you from answers, not just from training sets.
Mistake · Adding FAQPage schema to pages that do not contain those questions and answers.
Fix · Only mark up what is visibly on the page, in the same wording. Mismatched schema is ignored at best and counts against the page at worst, and it undermines the parts of the markup that are honest.
Mistake · Reporting a single spot-check screenshot as the baseline.
Fix · Run a fixed prompt set on a schedule and report the aggregate. Answers vary by session and model version; one response proves nothing, and the trend line is what leadership will remember.
A worked example: a five-person SEO team at a mid-market HR software company
A hypothetical mid-market HR software company ranks well for its category terms but is rarely named when someone asks ChatGPT or Perplexity for options. The first audit finds that the CDN bot-management rule challenges any agent not on its allow list. Googlebot and Bingbot are on the list; OAI-SearchBot, PerplexityBot and ClaudeBot are not. Nothing in the robots.txt in the repo suggested a problem.
With the allow list fixed, a non-JavaScript fetch of the comparison template shows the H1 and hero copy present but the comparison table and FAQ block rendered client-side. The team moves those two components to server rendering, adds FAQPage markup that matches the visible questions, and adds real dateModified values to the Article schema. The pricing page, which had been a JavaScript-only experiment, gets a static fallback.
On the content side, four overlapping "what is" articles are merged into one page with a definition block at the top and the variants covered below; the other three URLs redirect to it. The team fixes a prompt set of thirty category, comparison and problem questions and starts tracking it on a schedule across ChatGPT, Perplexity, Google AI Mode and Gemini, using the cited-source list as the monthly action item.
Over the following quarters the report shows citation share moving first on Perplexity, where source display is most prominent, then mention rate catching up on ChatGPT. The cited-source gap list surfaces two review sites and one community thread that the team hands to marketing for off-site work. None of the changes cost rankings, and the AI-visibility line now sits on the same slide as the rankings chart.
Frequently asked questions
Is AEO just SEO with a new name?+
No. It overlaps heavily but is not identical. The differences that matter in practice are which crawlers need access, how much JavaScript rendering is tolerated, how much weight structured data carries, and the fact that the output is a citation in a synthesised answer rather than a ranked position. Most technical SEO work helps; the gaps are where teams stall.
Will optimising for AI hurt our Google rankings?+
No. Complete structured data, server-rendered content, answer-first sections and consolidated pages are all things classic search rewards too. The two disciplines pull in the same direction on almost everything. The one genuine trade-off is whether to allow training crawlers such as GPTBot, which is a policy decision rather than a rankings one.
Which crawlers do we actually need to allow?+
For retrieval: OAI-SearchBot and ChatGPT-User for ChatGPT, PerplexityBot and Perplexity-User for Perplexity, and ClaudeBot for Claude. GPTBot, CCBot, Google-Extended and Applebot-Extended are training crawlers and can be handled separately. Google AI Overviews and AI Mode use the regular Googlebot index, so no extra agent is needed there.
How long do the content pages need to be?+
Long enough to answer the question completely and no longer. Competitive category pages tend to be substantial because the questions are, but padding a thin page to a word count does not make it citable. Extractable structure does: a direct answer at the top of each section, one claim per passage, and a definition where a term needs one.
Where should a team start?+
Access and rendering, every time. They are cheap to check, fast to fix, and it is common to find that zero citations has a mechanical cause. Only once every retrieval agent can fetch a fully rendered page is it worth spending editorial time on structure and consolidation.
Does blocking GPTBot affect our ChatGPT visibility?+
Not directly for search-style answers, which are fetched by OAI-SearchBot and ChatGPT-User at query time. GPTBot is the training crawler. Blocking it is a legitimate policy choice, but make sure the rule is scoped to GPTBot alone and does not catch the retrieval agents with it, which is a frequent side effect of a broad wildcard.
How do we report this without a rank position?+
Use mention rate and citation share on a fixed prompt set, per engine, tracked over time. Add sentiment or description accuracy where the wording matters. Present it alongside rankings and clicks rather than instead of them, and attach the cited-source list so the number comes with an action.
Do we need a separate tool, or does our SEO suite cover this?+
Rank trackers do not query answer engines, and web analytics under-reports AI referrals. You need something that runs prompts against each engine on a schedule and records mentions and citations. AI Search Monitoring at /features/ai-search-monitoring is that layer; the technical checks can be done with the free validators.
Explore the features behind this solution
Go deeper
Audit what answer engines can actually see
Crawler access, rendering and structured data: the three things that decide whether the content work matters.