Back to Insights
AI Visibility Isn't a Number. It's a Range.
Ron Sielinski
Ron Sielinski
,
,
Chief Data Scientist
Chief Data Scientist
7 min read
7 min read
When a dashboard reports that a brand earned 17.3% of AI citations last month, we tend to take it at face value. The number is precise to a tenth of a percent. It came from software. It must be right.
The number may well be accurate. It's also just one of many numbers that the same measurement could have produced.
Consider a decision that a brand manager might face on any given Tuesday. She measures visibility for the topic of running gear on OpenAI's SearchGPT, using 200 prompts. Tom's Guide earns 9.5% of all citations, and Runner's World earns 6.0%. The conclusion seems obvious: Tom's Guide is the dominant source, ahead by three and a half points, so that's where her partnership budget should go.
Now add the 95% confidence intervals. Tom's Guide spans roughly 5.5% to 12.5%. Runner's World spans roughly 4.0% to 8.0%. The entire Runner's World interval sits inside the Tom's Guide interval.
The data can't tell these two sources apart. What looked like a clear winner is, statistically, a tie.
This example comes from our published research, and it's typical. Across the platforms and topics we studied, overlapping intervals were the norm for domains separated by less than five to seven percentage points of citation share. Most dashboards would have reported 9.5% and 6.0% as facts, when they were really just two draws from two distributions.
Same prompt, different answer
Traditional search results are deterministic. Run the same query twice, and you'll see the same ten blue links in the same order.
Generative search engines work differently. They sample from a probability distribution when they write an answer, and the retrieval step that selects sources can return different candidates each time. Ask the same question twice, and you'll get two different answers with two different sets of citations.
How different? We submitted the same prompts to the same platforms repeatedly and compared the domains they cited. The typical overlap between two runs of the same prompt was about 30% on Gemini, 40% on SearchGPT, and 50% on Perplexity. Even the most consistent of the three shares only half its sources from one run to the next.
We also wanted to rule out the obvious alternative explanation: maybe the cited pages were changing. So, during each collection, we scraped the cited pages and fingerprinted their content. The pages were overwhelmingly stable while the citations swung back and forth. In one test, with runs just ten minutes apart, top-ranked domains still traded places. The variability comes from the engines themselves.
That makes every visibility metric a sample statistic. Citation Share, Citation Coverage, Mention Coverage, and Sentiment are all estimates of some underlying rate, computed from a finite set of answers. Run the measurement again, and you'll get a different estimate. The only honest way to report one is with a range that says how far it could plausibly move.
Reading the range
A 95% confidence interval answers a practical question: Given the amount of data we collected, what range of values is consistent with what we observed?
Two brands can have the same point estimate and very different intervals. A Citation Share of 43% ± 2 points is a dependable reading. A Citation Share of 43% ± 11 points is a warning that the data isn't ready to support a decision. Same number, entirely different meaning.
For a simple proportion, the width of the interval follows from a familiar formula:
width ≈ 3.92 × √( p(1 − p) / n )
where p is the rate we're measuring and n is the number of observations. (The 3.92 is simply 2 × 1.96, which spans both tails of a 95% interval.)
Let's plug in some numbers. A brand that appears in 40% of answers, measured over 100 prompts, has an interval about 19 points wide, or ±9.6 points. In other words, the true rate could plausibly be anywhere from about 30% to 50%. Quadruple the sample to 400 prompts, and the interval narrows to ±4.8 points.
Notice what it took to cut the uncertainty in half: four times the data. Precision improves with the square root of the sample size, so doubling the number of prompts shrinks the interval by only about 29%. Past a certain point, more data buys very little, which is a tradeoff we'll return to in How Many Queries Do You Need to Measure AI Visibility?
In practice, the textbook formula understates the uncertainty, because AI answers aren't independent draws. A single answer that cites a domain five times contributes all five citations together or none at all. We account for that clustering with design-effect corrections and with bootstrap resampling, which recomputes each metric over thousands of resampled versions of the data. The principle doesn't change, though: Every figure comes with its range.
One bad day
Intervals do more than add error bars to a chart. Sometimes they're the only clue that a result is an outlier.
In one of our daily samples, National Geographic earned 3.2% of Gemini's citations for the topic of bird feeders, which would have ranked it among the top-cited sources. Across the nine days of the study, though, its average was 0.5%. On every other day, it was close to zero.
A 95% interval will miss the true value about one time in twenty. (That's what 95% means.) But the miss here was large. Anyone looking at that one sample would have concluded that National Geographic was a major source for the topic, and every subsequent sample would have proved them wrong. Repeated measurement and properly computed intervals are how we catch this. A single number gives us no way to know.
Before and after
Most of the questions people ask about AI visibility are comparative. Did our content changes improve visibility? Are we ahead of a competitor? Did Gemini change how it treats our category?
The intuitive approach is to check whether two intervals overlap. That's a reasonable first look, but it's conservative: Two intervals can overlap even when the difference between them is real.
The stronger approach is to put an interval directly on the difference. If we measured 8% share before a content change and 11% after, we compute the change (3 points) and its own confidence interval. If that interval excludes zero, the change is real, and sampling noise is ruled out as the explanation. If it includes zero, we can't yet distinguish a genuine improvement from a lucky draw.
This works best when both measurements use the same prompts. When every prompt is measured both before and after, prompt-to-prompt variation cancels out, much as it does in a crossover clinical trial, in which each patient receives both treatments. The paired design produces a tighter interval on the difference, and it can resolve changes that two separate intervals would leave ambiguous.
Without that discipline, it's easy to take credit for noise. On SearchGPT, a three-point gain in Citation Share falls within the typical width of a single domain's interval. A dashboard would show it as a win. The data doesn't support the claim.
Thinner slices, wider ranges
Analysts slice. They want Citation Share on Perplexity for expert-recommendation prompts only, or Sentiment for comparison questions alone. Every filter shrinks the sample underneath the view.
A filtered number can look every bit as precise as the top-line figure while resting on a fraction of the evidence. The remedy is to recompute the interval for every slice. When a slice is thin, its interval widens, and that wider interval is the warning: It tells us the segment can't support a conclusion yet. That's far better than presenting noise at higher resolution as if it were insight.
The number and the range
We should keep reporting numbers. We should just never report them alone.
Read the interval before the point estimate, because the interval tells you whether the number is ready to use. Treat brands with overlapping intervals as tied until a direct comparison says otherwise. Judge changes by the interval on the difference, never by the before and after figures alone. Expect filtered views to be less certain than the top line. And be skeptical of any visibility figure reported without a range. It may well be accurate, but there's no way to tell.
That's why IQRush reports every metric with a confidence interval, recomputes that interval for every slice you filter to, and sends the same prompts to every platform on every run, so that comparisons across platforms and over time are paired. The range arrives with the number, whether or not anyone asks for it.
That 17.3% from the beginning of this post? It's a perfectly good estimate. It just needs its range before anyone makes a decision with it.
References
In this series
Research papers
Sielinski, R. (2026). Quantifying Uncertainty in AI Visibility: A Statistical Framework for Generative Search Measurement. arXiv:2603.08924.
When a dashboard reports that a brand earned 17.3% of AI citations last month, we tend to take it at face value. The number is precise to a tenth of a percent. It came from software. It must be right.
The number may well be accurate. It's also just one of many numbers that the same measurement could have produced.
Consider a decision that a brand manager might face on any given Tuesday. She measures visibility for the topic of running gear on OpenAI's SearchGPT, using 200 prompts. Tom's Guide earns 9.5% of all citations, and Runner's World earns 6.0%. The conclusion seems obvious: Tom's Guide is the dominant source, ahead by three and a half points, so that's where her partnership budget should go.
Now add the 95% confidence intervals. Tom's Guide spans roughly 5.5% to 12.5%. Runner's World spans roughly 4.0% to 8.0%. The entire Runner's World interval sits inside the Tom's Guide interval.
The data can't tell these two sources apart. What looked like a clear winner is, statistically, a tie.
This example comes from our published research, and it's typical. Across the platforms and topics we studied, overlapping intervals were the norm for domains separated by less than five to seven percentage points of citation share. Most dashboards would have reported 9.5% and 6.0% as facts, when they were really just two draws from two distributions.
Same prompt, different answer
Traditional search results are deterministic. Run the same query twice, and you'll see the same ten blue links in the same order.
Generative search engines work differently. They sample from a probability distribution when they write an answer, and the retrieval step that selects sources can return different candidates each time. Ask the same question twice, and you'll get two different answers with two different sets of citations.
How different? We submitted the same prompts to the same platforms repeatedly and compared the domains they cited. The typical overlap between two runs of the same prompt was about 30% on Gemini, 40% on SearchGPT, and 50% on Perplexity. Even the most consistent of the three shares only half its sources from one run to the next.
We also wanted to rule out the obvious alternative explanation: maybe the cited pages were changing. So, during each collection, we scraped the cited pages and fingerprinted their content. The pages were overwhelmingly stable while the citations swung back and forth. In one test, with runs just ten minutes apart, top-ranked domains still traded places. The variability comes from the engines themselves.
That makes every visibility metric a sample statistic. Citation Share, Citation Coverage, Mention Coverage, and Sentiment are all estimates of some underlying rate, computed from a finite set of answers. Run the measurement again, and you'll get a different estimate. The only honest way to report one is with a range that says how far it could plausibly move.
Reading the range
A 95% confidence interval answers a practical question: Given the amount of data we collected, what range of values is consistent with what we observed?
Two brands can have the same point estimate and very different intervals. A Citation Share of 43% ± 2 points is a dependable reading. A Citation Share of 43% ± 11 points is a warning that the data isn't ready to support a decision. Same number, entirely different meaning.
For a simple proportion, the width of the interval follows from a familiar formula:
width ≈ 3.92 × √( p(1 − p) / n )
where p is the rate we're measuring and n is the number of observations. (The 3.92 is simply 2 × 1.96, which spans both tails of a 95% interval.)
Let's plug in some numbers. A brand that appears in 40% of answers, measured over 100 prompts, has an interval about 19 points wide, or ±9.6 points. In other words, the true rate could plausibly be anywhere from about 30% to 50%. Quadruple the sample to 400 prompts, and the interval narrows to ±4.8 points.
Notice what it took to cut the uncertainty in half: four times the data. Precision improves with the square root of the sample size, so doubling the number of prompts shrinks the interval by only about 29%. Past a certain point, more data buys very little, which is a tradeoff we'll return to in How Many Queries Do You Need to Measure AI Visibility?
In practice, the textbook formula understates the uncertainty, because AI answers aren't independent draws. A single answer that cites a domain five times contributes all five citations together or none at all. We account for that clustering with design-effect corrections and with bootstrap resampling, which recomputes each metric over thousands of resampled versions of the data. The principle doesn't change, though: Every figure comes with its range.
One bad day
Intervals do more than add error bars to a chart. Sometimes they're the only clue that a result is an outlier.
In one of our daily samples, National Geographic earned 3.2% of Gemini's citations for the topic of bird feeders, which would have ranked it among the top-cited sources. Across the nine days of the study, though, its average was 0.5%. On every other day, it was close to zero.
A 95% interval will miss the true value about one time in twenty. (That's what 95% means.) But the miss here was large. Anyone looking at that one sample would have concluded that National Geographic was a major source for the topic, and every subsequent sample would have proved them wrong. Repeated measurement and properly computed intervals are how we catch this. A single number gives us no way to know.
Before and after
Most of the questions people ask about AI visibility are comparative. Did our content changes improve visibility? Are we ahead of a competitor? Did Gemini change how it treats our category?
The intuitive approach is to check whether two intervals overlap. That's a reasonable first look, but it's conservative: Two intervals can overlap even when the difference between them is real.
The stronger approach is to put an interval directly on the difference. If we measured 8% share before a content change and 11% after, we compute the change (3 points) and its own confidence interval. If that interval excludes zero, the change is real, and sampling noise is ruled out as the explanation. If it includes zero, we can't yet distinguish a genuine improvement from a lucky draw.
This works best when both measurements use the same prompts. When every prompt is measured both before and after, prompt-to-prompt variation cancels out, much as it does in a crossover clinical trial, in which each patient receives both treatments. The paired design produces a tighter interval on the difference, and it can resolve changes that two separate intervals would leave ambiguous.
Without that discipline, it's easy to take credit for noise. On SearchGPT, a three-point gain in Citation Share falls within the typical width of a single domain's interval. A dashboard would show it as a win. The data doesn't support the claim.
Thinner slices, wider ranges
Analysts slice. They want Citation Share on Perplexity for expert-recommendation prompts only, or Sentiment for comparison questions alone. Every filter shrinks the sample underneath the view.
A filtered number can look every bit as precise as the top-line figure while resting on a fraction of the evidence. The remedy is to recompute the interval for every slice. When a slice is thin, its interval widens, and that wider interval is the warning: It tells us the segment can't support a conclusion yet. That's far better than presenting noise at higher resolution as if it were insight.
The number and the range
We should keep reporting numbers. We should just never report them alone.
Read the interval before the point estimate, because the interval tells you whether the number is ready to use. Treat brands with overlapping intervals as tied until a direct comparison says otherwise. Judge changes by the interval on the difference, never by the before and after figures alone. Expect filtered views to be less certain than the top line. And be skeptical of any visibility figure reported without a range. It may well be accurate, but there's no way to tell.
That's why IQRush reports every metric with a confidence interval, recomputes that interval for every slice you filter to, and sends the same prompts to every platform on every run, so that comparisons across platforms and over time are paired. The range arrives with the number, whether or not anyone asks for it.
That 17.3% from the beginning of this post? It's a perfectly good estimate. It just needs its range before anyone makes a decision with it.
References
In this series
Research papers
Sielinski, R. (2026). Quantifying Uncertainty in AI Visibility: A Statistical Framework for Generative Search Measurement. arXiv:2603.08924.
Frequently asked questions
Why do AI visibility metrics need a confidence interval?
Generative search engines sample when they write an answer, so the same prompt submitted twice cites different sources. In IQRush's research, the typical overlap between two runs of the same prompt was about 30% on Gemini, 40% on SearchGPT, and 50% on Perplexity. That makes every visibility metric a sample statistic, and a sample statistic is only honest when it comes with a range saying how far it could plausibly move.
How should I read a 95% confidence interval on a visibility number?
It answers a practical question: given the data collected, what range of values is consistent with what was observed? A Citation Share of 43% ± 2 points is a dependable reading. A Citation Share of 43% ± 11 points means the data is not yet ready to support a decision. Same number, entirely different meaning.
How many more prompts does it take to make a visibility interval narrower?
Precision improves with the square root of the sample size. A brand appearing in 40% of answers over 100 prompts has an interval of about ±9.6 points; quadrupling to 400 prompts narrows it to ±4.8. Cutting the uncertainty in half takes four times the data, so past a certain point more prompts buy very little.
How do I tell whether a change in AI visibility is real or just noise?
Put a confidence interval on the difference itself, not just on the before and after figures. If the interval on the change excludes zero, the change is real. If it includes zero, you cannot yet distinguish a genuine improvement from a lucky draw. Using the same prompts in both measurements (a paired design) tightens that interval considerably.
Why do filtered views of AI visibility data look less certain?
Every filter shrinks the sample underneath the view. A number for one platform and one prompt type can look as precise as the top-line figure while resting on a fraction of the evidence. IQRush recomputes the interval for every slice, so a thin segment shows a wider range, which is the warning that it cannot yet support a conclusion.