Skip to content

How we measure brand visibility in AI

Every week we ask ChatGPT and Gemini the questions someone looking for a product in your category would ask, and check whether the answer names your brand and whether your site is among its sources. Below: how we score it, what the number is worth and what it does not tell you.

What we ask, and how

  • Engines: ChatGPT (OpenAI, GPT-6 Luna with web search, since October 19, 2026; before that GPT-4.1) and Gemini (Google, gemini-3.5-flash with Google Search). We ask through the API, in the language set for the project. Since November 5, 2026, Gemini answers at the default temperature Google recommends for this model (at temperature 0 before that). GPT-6 Luna takes no temperature, so its answers can differ between runs. We set its reasoning to the low level so that it answers within the time of a measurement. ChatGPT also gets the project’s country as an approximate location, and Gemini, whose API has no such setting, has been told it in its instructions since October 22, 2026.
  • Cadence: each engine once a week, and after the first change of the questions in a week also at the next data collection. A project can ask each question up to 3 times in one check, at least 4 hours apart. The brand then counts as named when it appears in more than half of the answers (a tie counts as absent), and its position is the median across them.
  • Three kinds of questions:
    • about the category, the way someone who does not know your brand would ask. Only these count toward the score;
    • about the brand (“what is X”, “X reviews”). They show what AI knows about you, but they do not count toward the score: a question that names the brand almost always gets it named back. In two of our projects such questions inflated the score 2 to 4 times (our measurement, October 2026) before we took them out;
    • about a differentiator: whether AI links the brand with the trait that sets it apart. That is a separate number next to the score, not part of it.
  • We recognise a question about the brand by its text: the name, aliases and domain, not only by its label.

How the questions are written

We write new sets in two steps. First a model describes the market: the category, the offer, the audience, competitors and the brand’s differentiators. Then a second step writes customer needs blind: it gets the category description without the brand’s name, site or differentiators, so the questions do not repeat the words the brand uses about itself. Each need is written in two different phrasings.

On request, up to 1/3 of the needs can come from real Google Search Console and Bing searches the brand’s site was shown for in the last 90 days. The model gets only searches that don’t name the brand, its aliases or its differentiators, and each such need keeps the search it came from. The search engine already shows the site for these searches, so such needs may raise the score. That is why the option is off by default and the limit is enforced in code, not left to the model.

An automatic check rejects questions that contain the brand’s name, an alias, its domain or a differentiator’s wording, and flags those that repeat words from the brand’s site or name a year or a country. A second model (Gemini) flags questions nobody would ask. We freeze the set before the first measurement and never pick questions by their results.

How we score

Each category question scores 0 to 100 points:

  • a brand mention: up to 75 points. 75 when the brand comes first; less when competitors are named before it (60, 45, at least 30);
  • a link to your site among the answer’s sources: 25 points.

The score is the average over all category questions, from 0 to 100%. We match the brand by its name and aliases as whole words, never as part of another word.

What the number is worth

  • Uncertainty range. The questions are a sample of what people might ask, so next to the score we show a 95% range (Wilson method). With 15 questions it runs from about ±13 to ±23 points, depending on the score. More questions narrow it; asking the same question twice does not.
  • Week-on-week change. We call it a change only when it passes a statistical test on pairs of the same questions (p < 0.05). Otherwise we say “no significant change”. Next to the latest score we show the average of the last four measurements.
  • Alerts. Engines answer a little differently every time: in our data from October 2026, 16 of 31 weekly “drops” were gone a week later. So we report a drop only when the loss holds for two measurements in a row and more questions lost than gained.

When the questions or the model change

We mark every change of the question set on the chart. A comparison across it shows no percentage change, because it would set answers to different questions against each other. We treat a change of the model that answers in ChatGPT the same way: we mark it on the ChatGPT chart and show no change in the score between measurements before and after it. We do not recompute past results.

What the engine searched for

For ChatGPT and Gemini we record the phrases the engine typed into the search engine, and for ChatGPT also the pages it looked at. That separates “AI did not find your site” from “it found it but did not cite it”.

How we judge the answers

When an answer names the brand, we judge it three more ways. Tone and recommendations are judged by gpt-6.1-sol, accuracy by gpt-6.1-sol; each judgment separately, once per question in each check. Since November 2, 2026, the judges use the model’s default temperature, as this model accepts no other (temperature 0 before that). These judgments are not part of the score; we show them next to it.

  • Tone. Positive when the answer speaks of the brand favourably; one favourable word or description is enough (“modern”, “the largest”, “known for…”), unless criticism balances it. Negative when criticism outweighs it. Neutral when the brand is only named or described in factual terms.
  • Recommendations. “Recommends” when the answer presents the brand as a good choice, including when it lists it among what is worth choosing or the best, even next to competitors. “Mentions” when it only names it. “Against” when it warns the user away. This judge also sees the question.
  • Accuracy. We compare the answer’s claims about the brand with the facts entered for the project and the text of the brand’s site. An answer is wrong only when one of its claims contradicts them; what they don’t cover is unverifiable, not wrong. An answer counts as wrong only when the same assistant also answered the same question wrongly the previous time it was judged. A first wrong answer awaits confirmation and is not counted in the error rate. An answer judged wrong is read once more by a second model from another maker (Gemini), against the same facts. If it does not find the answer wrong, the answer awaits confirmation, like a first wrong answer. If it does not reply, the first judgment stands. When the brand’s facts change, confirmation starts over: a verdict against the previous facts confirms nothing.

When we change these rules, we record the day of the change and do not compare judgments from before it with judgments after it.

What this measurement does not tell you

  • The API is not the app. The ChatGPT and Gemini apps can answer differently: they use other models, remember the conversation and know the user. The score is a measurement comparable over time, not a copy of what every customer sees.
  • The score leaves out AI Overviews in Google search results.
  • The score depends on the questions. That is why we keep one set as long as we can and mark every change to it.
  • A model makes the judgments. Tone, recommendations and accuracy are judged by a language model, so borderline cases go wrong. Accuracy is only as good as the facts entered for the project: an out-of-date fact makes a true answer come out wrong.
  • The score counts neither visits nor customers. It tells you whether AI mentions the brand and links to its site, not how many people visit afterwards. Those numbers are gathered separately, as our post How to measure traffic and customers from AI explains.