GEO Is Experimental Science

Ask the same AI engine the same buying question ten times and you will often get several different shortlists. That single fact rules out the playbook model of GEO and rules in the experimental one.

SEO trained a generation of marketers to think in checklists, because search rankings were stable enough to reverse-engineer. Do the list, wait, check your position. AI answers do not behave that way, and pretending they do is how teams end up celebrating noise or abandoning changes that worked.

How noisy the answers are

The 2026 measurements are blunt.

• In a SparkToro study, 600 volunteers ran 12 prompts through ChatGPT, Claude and Google's AI a combined 2,961 times. There was less than a 1 in 100 chance that any two responses gave the same brand list (SparkToro).
• Across more than 38,000 ChatGPT answers and 38,000 Gemini answers to 585 commercial questions, collected through the APIs, a brand moved into or out of the top five on about 39% of transitions (TechRepublic on MentionBird).
• SE Ranking ran 10,000 keywords through Google AI Mode three times on the same day and found only 9.2% average overlap in the cited URLs (SE Ranking).
• Across engines, BrightEdge found ChatGPT, AI Overviews and AI Mode disagreed on brand recommendations for 61.9% of identical prompts (BrightEdge).

Notice how this squares with Chapter 1, where engines converged on brands. Both are true at different scales. Any single answer is volatile. Aggregate enough answers and stable patterns appear, including which brands are consistently in the consideration set and which are not. Conductor's analysis of 14,000 calls makes a related point: recommendations are "predictably inconsistent," and for some intents, especially comparisons, the first pick is far more stable than the rest of the list (Conductor).

That is exactly the situation science was invented for: a noisy individual measurement hiding a real underlying signal.

Three reasons GEO resists the playbook

There are many engines. ChatGPT, Gemini, AI Overviews, AI Mode, Perplexity, Claude and Copilot each retrieve from different indexes and weigh sources differently. A change that helps on one can do nothing on another. There is no single scoreboard.

The answers are non-deterministic. Sampling, query fan-out, personalization, session context and frequent model updates mean the same input does not produce the same output. A single check proves almost nothing.

There is no rank to reverse-engineer. In classic search you could watch position 7 become position 3 and attribute it. In AI answers, a brand is mentioned or not, cited or not, recommended first or buried, described accurately or wrongly, with probabilities that shift underneath you.

A checklist assumes a stable system you can push in a known direction. GEO operates on an unstable system where you can only shift probabilities. That calls for a different method.

The four-step loop

The method is the one product teams already use for anything uncertain.

1. Form a hypothesis tied to a product signal. Not "publish more content," but something specific and falsifiable: "If our SSO feature page and docs state plainly that TrustBridge Connect is SAML SSO, our mention rate on SSO-intent prompts will rise across at least three engines within eight weeks." A good GEO hypothesis names the change, the prompts it should affect, the engines, the metric and the time window.

2. Run a controlled test. Change one thing where you can. Keep a control: a comparable feature, product line or prompt cluster you do not touch. Without a control, a model update that lifts everyone will look like your win.

3. Measure the distribution, not the anecdote. Run each prompt many times per engine before and after the change. Measure rates: how often you are mentioned, how often you are cited, how often you are first, and how often the engine describes you accurately. Compare treatment to control. One screenshot of a good answer is a story, not a result.

4. Iterate and record. Keep a change log of every product, content and technical change with its date. Retire hypotheses that fail. Expand the ones that work to adjacent features and verticals. Over a year, the log becomes your own evidence base, which is worth more than any generic best-practice list because it describes your domain and your engines.

What to measure

Most dashboards report share of voice or citation share. Those are useful, but they miss the measure that matters most for product-resident GEO.

Metric	Question it answers	Why it matters
Mention rate	How often are we named for this prompt cluster?	Presence in the consideration set
Citation rate	How often is one of our URLs cited?	Which of our surfaces engines trust
First-pick rate	How often are we recommended first?	The first pick is often the most stable, and the one a buyer reads first
Description accuracy	When named, is what the engine says about us correct?	The direct test of product-layer quality
Source mix	Which domains are cited when we are named?	Where our entity is actually being built

Description accuracy is the metric most teams skip and the one product teams should own. If engines name you but get your category, capabilities or integrations wrong, the fix is almost never more content. It is a product signal that is missing, contradictory or buried.

Sample sizes, honestly

How many runs are enough? There is no universal answer, and anyone who gives you one without asking about your prompts is guessing. My working rule: enough repeated runs per prompt per engine that a single unusual answer cannot move the rate much, across enough prompts per cluster that one odd prompt cannot either. In practice that usually means dozens of prompts per cluster and several runs each, repeated over weeks rather than days. Treat that as a starting point to tune, not a standard.

Watch for two confounders in particular. Model and index updates can shift everyone at once, which is why controls matter. And retrieval-based answers can respond within days, while anything that depends on training data can take months, so match your time window to the mechanism you are testing.

An example experiment

Here is the shape of a product-layer test. It is an illustrative composite, not a single client's results.

A B2B identity product had a feature with a branded name and no plain functional description. Hypothesis: adding the functional term to the feature's name line in docs, the pricing page and the product's structured product data would raise mention rate on prompts using the functional term.

The team picked a second, similar feature as control and left it alone. They ran a fixed prompt set across four engines, with repeated runs, for four weeks before the change and six weeks after. They tracked mention rate and description accuracy for both features.

The outcome I would look for is not "we got cited." It is: the treated feature's rate moved, the control's did not, and the movement held across more than one engine. That is a result you can scale to the rest of the product. Anything less is a hint worth another round, not a conclusion.

This is product methodology

None of this is new to product teams. Hypotheses, controls, holdouts, distributions, change logs and patience with noise are how good product organizations ship features and pricing changes. GEO asks for the same discipline, applied to a new surface.

That is the second reason GEO belongs near product. Not only do product teams own most of the signals (Chapter 1), they also own the method needed to change those signals responsibly. Marketing teams can learn it, and many do. But the instinct to test before declaring victory is already native to product.

What this chapter rules out

It rules out three common habits. Declaring success from a single good answer. Judging a change after a week. And buying a tool because its dashboard shows one number going up, without asking how it handles variance. Chapter 12 turns that last point into a buying criterion.

Key takeaways

• AI answers vary run to run and engine to engine, so a single check proves almost nothing.
• Use a four-step loop: a falsifiable hypothesis tied to a product signal, a controlled test, distributions instead of anecdotes, and a change log.
• Track mention rate, citation rate, first-pick rate, source mix and, above all, description accuracy.
• Match the time window to the mechanism: retrieval can respond in days, training-dependent changes take months.