Generative engine optimisation: being the source AI assistants cite

AI systems use their own crawlers with their own user agent strings, and those are separate from Googlebot. A site can be perfectly visible in Google results and entirely unreachable by the crawler behind an assistant, and nothing on the site indicates this.

The blocking usually happens in one of two ways, neither deliberate. Someone added agent names to robots.txt following advice about protecting content, without distinguishing between crawlers used for training and crawlers used for live retrieval. Or, more often, a bot protection service or firewall is challenging unfamiliar agents by default, which no one in the business knows because it was configured by the hosting provider.

The distinction that matters is between training crawlers and retrieval crawlers. A training crawler collects content that may inform a future model. A retrieval crawler fetches pages at the moment someone asks a question, so that the assistant can answer using current information and cite the source. Blocking the first and allowing the second is a coherent position: your content is not absorbed into a model, and you still appear in answers with attribution. Blocking both removes you from the answers entirely.

Whether to allow any of them is a genuine business decision and both answers are defensible. Publishers protecting subscription content reasonably block. Service businesses wanting to be found reasonably do not. What is not defensible is not knowing, and most sites we check have never looked.

The test is straightforward: read your live robots.txt, list which agents are named, and then check whether your firewall or bot protection is challenging requests the robots file permits. The second half is the part usually missed.

A statement with a named, linked primary source behind it is more likely to be repeated with your name attached than the same statement asserted alone. This is the most actionable finding in generative engine optimisation and the one businesses act on least, because sourcing takes effort and asserting does not.

The practical standard is ordinary journalism. Any figure, rule, date or external fact names where it came from, links to it, and says when it was checked. Regulations cite the regulator. Platform behaviour cites the platform’s own documentation. Research cites the study rather than the article that summarised it.

Dates matter more here than in traditional SEO. An assistant answering a question about current rules has to judge whether a page is still accurate, and an undated page asserting a fact is weaker than a dated page citing a source. Publication and update dates should be visible and honest, and updating the date without updating the substance is both dishonest and useless.

First hand experience is the other differentiator, and it aligns with Google’s guidance on helpful content. Material that could only be written by someone who does the work, with specifics about process, constraints and failure modes, is harder to synthesise from other sources and therefore more likely to be the thing cited.

The inverse is worth stating. Pages that assemble generally available information without adding anything are exactly what these systems can produce themselves, so there is no reason to cite them. Generative engine optimisation rewards material that contains something unavailable elsewhere.

Traditional SEO counts links. AI systems draw on the whole corpus they were trained on or can retrieve, which means a business discussed across many credible places is more likely to be surfaced than one that exists only on its own website, regardless of links.

This changes the emphasis of off page work. An unlinked mention in a respected trade publication, which a classic link building programme would discount entirely, is genuinely valuable here because it is a corroborating description of your business from a source that is not you.

Where it concentrates is in the places people go for recommendations. Industry publications, professional associations, review platforms, community forums and question sites. A business mentioned in the kinds of places a person would look when asking the same question is more likely to appear when a machine does.

The implication is that generative engine optimisation and off page SEO want the same work for partly different reasons, which is convenient. Genuine coverage, active association memberships, real reviews and participation in your sector’s conversation serve both.

What does not work is the shortcut version. Mass produced mentions on low quality sites are as useless here as in traditional search, and possibly more so, since credibility of source is central to how these systems weigh material.

The pattern is that generative engine optimisation does not replace search visibility, it builds on it. Systems grounded in a search index favour pages already performing there, so a site invisible to Google is usually invisible to the assistants built on it. The order of work follows: fix indexing and quality first, then the extraction and access details.

One consequence specific to this region is worth noting. Bing visibility is frequently neglected by UAE businesses and it matters for Copilot, which is present on a large number of corporate desktops. A business selling to other businesses here may be missing an audience it never considered.

Across the generative engine optimisation testing we have done, certain page types appear repeatedly as sources and others almost never do. Knowing which is which changes what is worth writing.

Explanatory pages answering a specific question are cited most. A page that clearly answers one question, with the answer near the top and evidence behind it, is exactly what a retrieval system is looking for. This is the highest return format.

Comparison content performs well because a great many questions put to assistants are comparative. A page setting out genuine trade offs between two approaches, including where each loses, is useful in a way a one sided page is not.

Process and how to material is frequently drawn on, particularly where it includes the steps that go wrong and what the constraints are. This is where first hand knowledge shows most clearly.

Pages with genuine original data are cited disproportionately, because the information exists nowhere else. Even modest figures from your own operations, published honestly with method stated, are hard for a system to find elsewhere.

Sales pages are rarely cited, and this is worth accepting rather than fighting. A page whose purpose is persuasion contains few extractable facts, and an assistant answering a question has no reason to quote it. That does not make the page unnecessary; it makes it the wrong place to invest for this particular outcome.

News and announcements age out quickly. A company update is rarely the answer to anything a buyer asks, and it does not accumulate value the way explanatory material does.

That generative engine optimisation order matters because the first three steps cost very little and frequently produce the whole of the available improvement. Businesses that begin at step four, commissioning new content before checking whether their crawlers are blocked, spend the most and learn the least.

It also keeps the investment proportionate. For most Dubai businesses this is a few days of work layered onto an existing search programme, not a separate retainer, and treating it as a major new budget line is usually premature given how uncertain the measurement remains.

Does an llms.txt file help? It is a proposed convention for describing a site to language models, and support for it is not established. Adding one is inexpensive and harmless, and Google has been clear it does not use such a file for Search. Treat generative engine optimisation claims built around it with caution, because nobody can currently demonstrate its effect.

Should we publish more content to be cited more? Volume is not the lever. A small number of genuinely useful, well sourced pages outperforms a large number of thin ones, and generative engine optimisation punishes padding for the same reason search does: there is no reason to cite a page that says nothing specific.

Will this cannibalise our traffic? Possibly. If an assistant answers a question completely, the person may not click through. That is a real effect and it applies most to purely informational queries with short answers. It applies least to questions where someone needs to choose a supplier, which is where your commercial interest sits.

Does our content management system matter? Only insofar as it affects rendering and structure. A platform that produces clean HTML with proper headings is easier to extract from than one that assembles everything in the browser, which is the same consideration covered under technical work.

How often should we re test? Monthly is reasonable for most businesses. More frequent testing produces noise, since outputs vary between sessions, and less frequent means a problem such as being described inaccurately can persist for a long time before anyone notices.