AI Visibility Tracking: What to Measure, and What Not to Fake, When You Track Your Brand in AI Answers

AI visibility tracking means asking AI systems the same questions repeatedly, recording whether and how your brand appears, and using the tally to estimate visibility. The output is a rate with a margin of error, such as the share of runs that name your brand. It is not a ranking, and it is not a count of readers.

The reason to be careful is variance. In a study published by SparkToro with Gumshoe.ai, 600 volunteers ran 12 prompts through ChatGPT, Claude and Google's AI features, producing 2,961 responses in November and December 2025. The authors report fewer than 1 in 100 chances that ChatGPT or Google's AI would give the same list of brands in any two responses to the same prompt, and fewer than 1 in 1,000 for the same list in the same order. Rand Fishkin concluded that a visibility percentage across dozens to hundreds of prompts, each run multiple times, is a reasonable metric, while any tool giving a ranking position in AI is, in his word, baloney. Two cautions apply. The study is not peer reviewed, and Fishkin notes a possible conflict of interest because his co-researcher, Patrick O'Donnell, works at Gumshoe.ai, a startup in the AI tracking sector. It is also one study of a changing system.

The variance is not a quirk of one engine. A paper by Atil and colleagues tested five LLMs configured to be deterministic, across eight tasks and ten runs, and found that outputs varied for the same inputs. None of the models gave repeatable results across all tasks. That paper measured task accuracy, not brand mentions, so it does not tell you how often a brand appears. It does tell you that a single run, even at settings meant to be stable, is a poor measurement.

All three, separately. They answer different questions, and combining them into one number hides which part of the chain is working. We cover the crawl, citation and referral split for ChatGPT in our post on ChatGPT search visibility, and the difference between being listed and being used in our post on citation selection and absorption.

The distinction between selection and use is supported in the research literature. A 2026 preprint by Zhang, He and Yao proposes a two-stage framework: citation selection, where a platform decides to search and picks sources, and citation absorption, where a cited page actually contributes language, evidence or structure to the answer. A cited URL is not proof the answer used what your page said.

Google and Microsoft now give some first-party data, with limits. Google's Search Generative AI performance reports, announced in 2026 and, according to Google's post, available to all websites worldwide since 31 August 2026, show impressions for URLs appearing in generative AI features, broken down by page, country, device and date. The announcement lists impressions, pages, countries, devices and dates, and does not list clicks. Bing Webmaster Tools' AI Performance dashboard, in public preview since February 2026, reports total citations, average cited pages, grounding queries (shown as a sample) and page-level citation activity. Bing says its figures do not indicate placement or a page's role in an answer. GA4's default channel group has an AI Assistant channel, defined as visits from sources such as ChatGPT, Gemini, Deepseek, Copilot or Grok, and it excludes Google's AI Overviews and AI Mode. Our post on the GA4 AI Assistant channel covers how to reconcile it.

Build it from questions customers actually ask, not from your brand name and not from keywords alone. Good sources are sales call notes, Search Console queries, support tickets and the objections that appear in lost-deal notes. Group the prompts by stage so the set covers more than your best-case questions: a problem the buyer has, a category they are choosing in, a comparison with named alternatives, and a question about your brand itself.

Fix the wording once you have chosen it. Changing a prompt between runs changes the question, and you will not know which change moved the number. Where you want to test phrasing, add two or three paraphrases of the same intent as separate entries and track them as a group. In the SparkToro report, 142 prompts about choosing headphones for a traveling family member produced 994 responses, and the top brands appeared in 55 to 77 percent of them. That suggests that a well-defined category shows a stable core even when the lists wobble. It is one category in one study, so test your own.

For a small team, a set of thirty to a hundred prompts is a workable range. That is our recommendation, not a published standard. It is small enough to review by hand and large enough that no single odd prompt dominates the result. Add competitors to the tracked list by name, and decide before the first run which ones count. Choosing them afterward, based on who looks beatable, is a quiet way to fake a result.

Enough to put a boundary on the number. When you run a prompt many times and count the share of runs that name your brand, you have a proportion, and a proportion has a confidence interval. The NIST engineering statistics handbook describes interval methods for a proportion, including the one attributed to Wilson and the Agresti-Coull variant. You do not need a statistician. You need to stop reading a bare percentage as precise.

Here is an illustrative case. Suppose your brand is named in 12 of 40 runs, which is 30 percent. A 95 percent Wilson interval runs from roughly 18 to 45 percent. With 100 runs and the same 30 percent, the interval narrows to roughly 22 to 40 percent. The figures are made up, but the arithmetic is general. A move from 30 to 36 percent between two small samples is well inside the noise, and reporting it as growth would be a mistake. Treat a change as real only when the intervals separate, or when the change repeats across several periods.

Log every run with the fields that let you rerun it later: the engine and its mode, the date and time, the location setting, whether you were logged in, the exact prompt, and the full answer text and cited URLs. Keep the conditions the same from one period to the next, or note where they changed. Our GEO test protocol post covers how to run a before-and-after comparison, and our AI search visibility measurement workbook has a logging template. Decide your comparison before you run it. Running more samples until the number looks good is a form of fakery, even when each run is honest.

Tools solve a real problem: running hundreds of prompts on a schedule is laborious. They also package uncertain measurements into confident dashboards. We have not reviewed any individual vendor's method for this post, so the table lists claims to question, not verdicts. Our posts on LLM visibility tools and AI brand monitoring go deeper on the categories.

Ask each vendor how the prompts are chosen and by whom, how many runs sit behind each number, which engines and modes are sampled and whether they sample the consumer product or an API, how competitors are selected, and whether the method has been published for review. SparkToro's report makes the same request of the industry: it asks marketers to stop spending on AI tracking that offers no publicly reviewable research. A vendor that answers plainly is worth a trial. A vendor that answers with a score is not.

Report a small number of things in one place. Start with a coverage line: how many prompts, how many runs, which engines and modes, and which dates. Then give your visibility rate and each named competitor's, each with its interval. Show where the mentions come from, separating your own site from third-party pages, since the pages answers cite are your lever. Add the referral visits from GA4 with a note on the channel boundary, and then the changes you made to the site or content in the period. End with the decisions the numbers support. A report that cannot name a decision is a record, not a measurement.

On OpenAI's side, its crawler documentation states that OAI-SearchBot and GPTBot are separate settings, and that a site can allow OAI-SearchBot to appear in search results while disallowing GPTBot, which relates to training. Check that you are not blocking the crawler that supports search before concluding that you are invisible. Google's AI features documentation says there are no additional requirements to appear in AI Overviews or AI Mode, so for Google the work is still the ordinary work of being indexable and useful.

Then keep the discipline that gives the number any value. Do not re-run until the answer improves. Do not publish a screenshot of the one good run. Do not claim revenue from AI answers unless you can trace the inquiry to a source, and say so when you cannot. Do not manufacture visits or mentions, and do not hide text for crawlers. Evidence that any single tactic reliably earns citations is thin: the authors of the C-SEO Bench paper point out that earlier tests covered a limited range of domains and a single acting document, and they built a benchmark to test with several competing actors. Nothing in this post guarantees a citation. If a number cannot survive being explained to a skeptical colleague, it does not belong on the report.