People increasingly get answers instead of a list of blue links. Google’s AI Overviews and AI Mode, Gemini, ChatGPT search, Perplexity and Claude all write a response and cite a handful of sources. Generative engine optimization (GEO) is the practice of making your brand one of those sources. Some of it is new, but most of it is good SEO done with more discipline. This guide explains how answer engines actually find and cite pages, what you can control (crawler access, entity clarity, structured data, content you can quote) and where the hype runs ahead of the evidence.
What generative engine optimization is, and what it isn’t#
A large language model doesn’t “search” in the classic sense when it answers from memory. When an assistant shows citations, though, it is almost always using retrieval-augmented generation. The system runs one or more searches, fetches or looks up candidate pages, pulls out relevant passages, and writes an answer grounded in them. The links you see are the passages it relied on.
That has a practical consequence. To be cited, you first have to be retrieved. Retrieval depends on search indexes and crawlers, which is familiar SEO territory. GEO then adds a second question: once your page is among the candidates, is it easy for the model to understand, trust and quote?
What GEO is not:
- A way to “inject” your brand into a model’s training data. What a model learned during training is beyond your control and changes only when it is retrained.
- A set of secret markup tricks. Google says plainly that there are no special requirements, machine-readable AI files or special schema needed to appear in its AI features (more on that below).
- Guaranteed. Answers vary between runs, users, locations and model versions. You are improving your odds, not buying a placement.
How the major answer engines find and cite sources#
Each provider combines its own crawlers, search indexes and partners differently, and the details change often. The broad picture as of late 2026:
Google AI Overviews and AI Mode#
Both are built on Google Search’s index. Google’s documentation on AI features and your website describes a “query fan-out” technique: the system issues several related searches across subtopics and can surface a wider set of supporting pages than a single classic query would. The requirement for inclusion is simply that a page is indexed and eligible to appear in Search with a snippet. Google lists no extra technical requirements.
Gemini#
The Gemini app can ground answers in Google Search results. Use of crawled content for Gemini model training and for grounding in other Google products is controlled by the Google-Extended robots.txt token. Google’s crawler documentation notes that it doesn’t affect inclusion or ranking in Google Search.
ChatGPT search#
OpenAI separates its crawlers by purpose: OAI-SearchBot surfaces websites in ChatGPT’s search features, GPTBot collects content that may be used to train foundation models, and ChatGPT-User fetches pages when a user’s request triggers a visit. Their crawler documentation notes that sites which opt out of OAI-SearchBot won’t appear in ChatGPT search answers.
Perplexity#
Perplexity runs PerplexityBot to index and link websites in its answers, and Perplexity-User for fetches a user triggers. It says PerplexityBot isn’t used to crawl content for training foundation models.
Claude#
Anthropic documents three agents: ClaudeBot collects content for model training, Claude-SearchBot improves the relevance of search results, and Claude-User fetches pages when a person asks Claude a question that needs web content.
| Token / user agent | Operator | Main purpose | Blocking it affects |
|---|---|---|---|
Googlebot |
Search indexing | Google Search and AI Overviews/AI Mode | |
Google-Extended |
Control token (no separate crawler) | Gemini training and grounding use; not Search ranking | |
OAI-SearchBot |
OpenAI | ChatGPT search | Appearing in ChatGPT search answers |
GPTBot |
OpenAI | Model training | Use of content in future training |
PerplexityBot |
Perplexity | Search index | Appearing in Perplexity answers |
ClaudeBot |
Anthropic | Model training | Use of content in future training |
Claude-SearchBot |
Anthropic | Search quality | Visibility in Claude’s search results |
CCBot |
Common Crawl | Open web archive | Inclusion in Common Crawl datasets, which many AI projects use |
- Agents like
ChatGPT-User,Perplexity-UserandClaude-Useract on a specific person’s request. Providers state that robots.txt rules may not apply to them in the same way as to automatic crawlers. Read each provider’s current documentation if this matters to you.
Crawler access: a deliberate robots.txt policy#
The most common GEO problem we see isn’t weak content. It is sites that block AI search crawlers by accident: a security plugin, a CDN bot-management rule or a copied robots.txt snippet that shuts out everything with “bot” in the name. Decide on a policy deliberately, along two separate axes:
- Search and citation crawlers: if you want to be cited, allow them.
- Training crawlers: a business and licensing decision. Blocking them doesn’t remove you from AI search with the providers that separate the two.
Here is a policy that welcomes AI search but opts out of training:
# Search engines and AI search/citation crawlers: allowed
User-agent: Googlebot
User-agent: Bingbot
User-agent: OAI-SearchBot
User-agent: PerplexityBot
User-agent: Claude-SearchBot
Allow: /
# Model-training crawlers and controls: opted out
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: Google-Extended
User-agent: CCBot
Disallow: /
# Everyone else
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Sitemap: https://www.example.com/sitemap_index.xmlTwo caveats. First, robots.txt is a voluntary standard. Reputable operators honour it, but it isn’t access control. Second, blocking Google-Extended is a trade-off you should make knowingly. It stops grounding use in Gemini-family products outside Search, which some brands want and others don’t. Whatever you choose, also check your CDN or firewall. Robots.txt allowing a crawler means nothing if a bot-protection rule returns a 403 or a challenge page to it.
Entity clarity: make it obvious who you are#
Language models reason about entities: organisations, people, products, places, and the relationships between them. If your brand name is ambiguous, inconsistent or rarely tied to what you do, a model has a hard time deciding that you are the right source to cite for a question in your field.
Practical steps#
- Use one canonical name. Use the same spelling, capitalisation and legal-versus-trading name everywhere: site header, footer, social profiles, directories and press mentions.
- Write a precise About page. Say in plain sentences what you are, what you do, who you serve and where you operate. “We are an independent web development studio building WordPress sites and custom web applications” is far more useful to a machine than a page of slogans.
- State relationships explicitly. Link your services, products and topic pages to each other with descriptive anchor text, so the site itself lays out what you are an authority on.
- Connect your profiles. Link from your site to your official profiles and from those profiles back to your site, and list them in
sameAsin your Organization markup.
Structured data: Organization, Article and FAQPage#
Let’s be clear about what structured data does here. Google says that no special schema is required for AI Overviews or AI Mode, and that your markup must match the visible content of the page. Still, JSON-LD built on schema.org types is an unambiguous, machine-readable statement of facts, and it costs little to maintain. It supports regular rich results and entity understanding, and it removes guesswork for any system that parses it.
Organization#
Put this on the homepage (or sitewide) with your name, URL, logo, contact point and sameAs profiles. This is where entity clarity becomes machine-readable.
Article#
On blog posts and guides, Article (or BlogPosting) markup states the headline, publication and modification dates, author and publisher. Accurate dates matter because answer engines care about freshness for many queries.
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "BlogPosting",
"headline": "Generative Engine Optimization: Getting Your Brand Cited by AI Search",
"datePublished": "2026-10-14",
"dateModified": "2026-10-14",
"author": { "@type": "Organization", "name": "Example Studio", "url": "https://www.example.com/" },
"publisher": {
"@type": "Organization",
"name": "Example Studio",
"logo": { "@type": "ImageObject", "url": "https://www.example.com/logo.png" }
},
"mainEntityOfPage": "https://www.example.com/blog/geo-guide/"
}
</script>FAQPage#
FAQPage markup describes real question-and-answer content on a page. Google has limited FAQ rich results to a small set of authoritative government and health sites since 2023, so don’t expect expandable FAQs in classic results. The markup still describes the content accurately, and question-shaped content maps naturally onto the questions people type into assistants. Only mark up questions and answers that are visible on the page.
Writing content that gets cited#
Retrieval systems work on passages, not whole pages. A page that buries its answer in the sixth paragraph of a narrative is harder to quote than one that states it plainly near a matching heading. Pages that get cited tend to share these traits:
- Answer first. Open each section with a direct one- or two-sentence answer, then expand. This is also good writing for skimming humans.
- Self-contained passages. Each section should make sense on its own, without “as mentioned above”. Repeat the subject noun instead of relying on “it”.
- Descriptive headings. Phrase headings the way people ask: “How long does a WordPress migration take?” rather than “Timelines”.
- Structure that extracts cleanly. Tables for comparisons, numbered lists for procedures and definitions in a single sentence. Semantic HTML makes this structure explicit.
- Specifics. Numbers, thresholds, version numbers, dates and named standards. A vague claim has nothing in it for a model to cite.
- Visible freshness and accountability. Show updated dates, a clear author or publisher, and an editorial policy explaining how content is checked.
Technical health still underpins all of this. A page that renders its main content only after heavy client-side JavaScript, or that responds slowly, may be retrieved less reliably. Start with our technical SEO checklist.
Original research and consistent brand mentions#
Give models something they can’t get elsewhere#
If twenty pages say the same thing, an answer engine can cite any of them, or none. Original information breaks that tie. You don’t need a research department to produce it:
- Publish aggregated, anonymised data from your own work, such as typical project timelines or the most common issues found in audits, with a clear methodology.
- Run your own benchmarks, documenting the setup well enough that someone else could reproduce them.
- Write definitive documentation for your own products, pricing and processes. For questions about you, you should be the best source.
- Create explanatory tools such as calculators, checklists and decision tables that others reference.
Be described consistently off-site#
Answer engines also learn about you from other sites: directories, industry publications, partner pages, reviews and community discussions. Consistency counts for more than volume here. Make sure the description, category and core facts about your business match everywhere you control them. Earn mentions in places that are relevant to your field. A short, accurate mention in a respected industry resource does more than dozens of low-quality listings.
The llms.txt proposal: what it is and how much it matters#
llms.txt is a proposal for a Markdown file at /llms.txt that gives language models a concise, curated guide to a site. It has a required H1 with the site name, an optional blockquote summary, and H2 sections listing key links with short notes. An “Optional” section holds secondary links. The proposal also suggests offering clean Markdown versions of important pages.
# Example Studio
> Independent web development studio building WordPress sites,
> WooCommerce stores and custom web applications.
## Services
- [WordPress development](https://www.example.com/services/wordpress/): Custom themes and block-based sites
- [Performance audits](https://www.example.com/services/audit/): Core Web Vitals and technical SEO reviews
## Guides
- [Technical SEO checklist](https://www.example.com/blog/technical-seo-checklist/): Pre-launch checks
## Optional
- [Company policies](https://www.example.com/policies/)Be realistic about it. llms.txt is a proposal, not a standard, and adoption is uneven. It is popular with developer-documentation sites and some AI coding tools. The major answer engines haven’t published consistent commitments to use it in ranking or citation, and Google explicitly says that no special AI text files are needed for its AI features. We see it as a cheap, harmless experiment that is worth doing if you maintain good documentation. It won’t fix weak fundamentals.
Measuring results#
Measurement is still rough, so combine several signals:
| Signal | Where to find it | What it tells you |
|---|---|---|
| Google AI features traffic | Search Console Performance report, “Web” search type | Clicks and impressions, including AI Overviews and AI Mode (not separated out) |
| Assistant referrals | Analytics referrers such as chatgpt.com, perplexity.ai, gemini.google.com, claude.ai | Which assistants send visitors, and to which pages |
| Crawler activity | Server or CDN logs filtered by user agent | Whether search and citation bots are reaching your pages at all |
| Prompt testing | A fixed list of customer questions, checked monthly | Whether, and how accurately, you are mentioned or cited |
Treat prompt testing as directional. Answers vary between sessions, accounts and locations, so look for trends across a stable question set, not single screenshots.
- To be cited, you must first be retrieved. Indexing, crawler access and technical health come first.
- Separate AI search crawlers from training crawlers in robots.txt, and check that your CDN doesn’t silently block them.
- Make your entity unambiguous: consistent name, precise About page, Organization markup with
sameAs. - Write answer-first, self-contained passages, and publish information others can’t.
- Treat llms.txt as an optional experiment, not a requirement.
Be the source worth citing#
Generative engine optimization rewards the same things good SEO always has, applied more strictly: crawlable pages, clear structure, unambiguous facts and content that adds something new. The new parts are mostly about access decisions and describing your brand consistently. Speed matters too. Pages that load and render quickly are easier for every crawler to process, which our Core Web Vitals guide covers in detail.
Want to know how AI search engines see your site today? Our Performance & Technical SEO Audit reviews crawler access, structured data, entity signals and content structure alongside the classic technical checks. Contact us and we’ll take a look.
Frequently asked questions
Is generative engine optimization different from SEO?
Mostly it is SEO with a different emphasis. AI search features retrieve pages from search indexes or their own crawls, so crawlability, indexing and quality content still come first. GEO adds attention to entity clarity, passages that can be quoted on their own, original information and how consistently your brand is described across the web.
Do I need an llms.txt file to appear in AI answers?
No. llms.txt is a community proposal, and support for it varies between AI providers. Google states that no special AI files or markup are needed to appear in AI Overviews or AI Mode. Publishing one is low-cost and harmless, but treat it as an experiment rather than a requirement.
If I block GPTBot, will my site disappear from ChatGPT?
Not necessarily. OpenAI documents GPTBot as the crawler for model training and OAI-SearchBot as the crawler for ChatGPT search. You can block training while still allowing search. Check each provider's documentation, because the separation between training, search and user-triggered fetching differs from company to company.
How can I tell whether AI search sends me traffic?
Google includes AI Overviews and AI Mode traffic in the Search Console Performance report under the Web search type. For other assistants, look for referrals from domains such as chatgpt.com, perplexity.ai, gemini.google.com and claude.ai in your analytics, and run regular manual checks of the questions your customers actually ask.
