Stop asking where you rank in ChatGPT

The most common question we hear from founders about AI search is some version of "where do we rank in ChatGPT?" It's a reasonable question. For twenty years, search success had a position number attached. You were third for "organic dog treats," and next month you wanted to be second.

The trouble is that the number doesn't exist in AI answers, at least not in a form you can track. Asking for it leads people to buy dashboards that report a figure with false precision, and to make decisions on noise. This piece makes the case for dropping rank entirely and measuring something that holds still long enough to be useful.

The same question, a different answer every time

In late 2025, Rand Fishkin at SparkToro recruited 600 volunteers to run the same 12 prompts through ChatGPT, Claude and Google's AI tools. The prompts asked for recommendations: brands, products, service providers. Across 2,961 runs, the team compared the lists that came back.

The headline finding, from SparkToro's write-up: if you asked ChatGPT or Google's AI the same question 100 times, there was less than a 1 in 100 chance that any two answers would contain the same list of brands. For order, Fishkin writes, it was "more like 1 in 1,000 runs before you'd see two lists in the same order."

Think about what that means for a "rank." If you check today and you're third, the next person asking the same question will almost certainly see a different list, and very probably a different order. Your position isn't a fact about your brand. It's a roll of the dice.

What does hold still

The more useful finding is buried a little further down. While the order shuffled constantly, the set of brands that appeared at all was much more stable.

The clearest example from SparkToro's data was a cancer hospital. Asked for the best cancer care hospitals on the US West Coast, ChatGPT included City of Hope in 69 of 71 answers. It named City of Hope first in only 25 of them.

Show the data

Measure	Share of answers
Appeared in the answer (69 of 71)	97%
Named first (25 of 71)	35%

Source: SparkToro, January 2026 (percentages are our calculation from the reported counts)

If you tracked "rank," you'd see City of Hope bouncing around and conclude something was wrong. If you tracked "share of answers," you'd see a brand that shows up almost every time, which is the thing that actually matters to the hospital.

SparkToro saw similar patterns elsewhere. A digital agency, Smartsites, appeared in 85 of 95 Google AI answers to its prompt. And in a test with 142 prompts written by real people about headphones, the prompts themselves barely resembled each other (an average semantic similarity score of 0.081, where 1 would be identical). Even so, across 994 responses, Bose, Sony, Sennheiser and Apple each appeared in somewhere between 55% and 77% of answers.

Test	What changed between runs	What stayed stable
City of Hope, ChatGPT	Order of the list, other names included	Appeared in 69 of 71 answers
Smartsites, Google AI	Order and the rest of the list	Appeared in 85 of 95 answers
Headphones, 142 human-written prompts	The wording of every prompt	Four big brands each in 55% to 77% of 994 answers

Fishkin's own conclusion was blunt. Visibility percentage across many runs is a reasonable metric, he argues, but "any tool that gives a 'ranking position in AI' is full of baloney." We'd put it more politely, but we agree.

How many runs make a real number

Once you accept that answers vary, the next question is how many you need to look at before the percentage means anything. Here, Gumshoe's Nick Clark did the useful work of applying ordinary survey statistics.

The rule of thumb Gumshoe uses for the margin of error at 95% confidence is 0.98 divided by the square root of the number of observations. That gives these sample sizes.

Margin of error	Observations needed
±10 percentage points	97
±5 percentage points	385
±1 percentage point	9,604

Three hundred and eighty-five runs of one prompt sounds like a lot of copying and pasting. The useful trick in Gumshoe's analysis is that the observations don't have to come from a single prompt. If you're measuring the same underlying question ("which brands come to mind for high-protein snacks?"), runs of related prompts can be pooled.

Show the data

Related prompts	Runs per prompt
1 prompt	385
10 prompts	39
50 prompts	8
100 prompts	4

Source: Gumshoe, February 2026

Gumshoe's example: 10 prompts, each written seven ways, run on seven models, gives 490 observations in one pass. That's enough for about a ±5 point reading straight away, ±2.5 points after four passes and ±1 point after twenty.

A single screenshot of ChatGPT naming your brand tells you almost nothing. A few hundred answers tell you something real.

Why rank is so tempting anyway

It's worth being honest about why this habit is hard to drop. Rank is easy to explain in a board meeting. It moves, which feels like progress. And it's what the old dashboards showed, so it's what people are used to asking for.

There's also a commercial reason. Some AI visibility tools report a single "position" because customers ask for one. If a tool tells you your brand ranks 2.4 in ChatGPT for a category, ask how many runs that number is based on, which prompts and which models. If the answer is a handful, the number is noise dressed up as a metric.

The deeper problem is what rank does to decisions. A team chasing position will rewrite a page, check the next morning, see a better spot and declare success. Given SparkToro's numbers, that "improvement" is most likely the dice landing differently. You can spend months reacting to randomness.

What to measure instead

The better question isn't "where do we rank?" but "out of all the times someone asks an assistant the kind of question our customers ask, how often are we in the answer?" That's share of answers (some tools call it share of voice or visibility). It's stable enough to track, it maps to something real, and you can compare it with competitors.

Here's a version a small team can run without buying anything. Tip 12 has the basic setup; this adds the sample-size discipline.

1. Write 10 to 20 prompts your customers would actually type. Mix broad questions ("best gluten-free crackers") with specific ones ("crackers with no seed oils for a kid with a nut allergy"). Write each one two or three ways.
2. Fix the list. Use the same prompts every time. Changing prompts month to month makes the trend meaningless.
3. Run them across the assistants your customers use. For most CPG brands that's ChatGPT, Google's AI Mode and Gemini, and Alexa for Shopping on Amazon. Use fresh sessions with no history.
4. Record presence, not position. For each answer, note whether your brand appears and whether each of two or three named competitors does. Note whether the facts about you were right.
5. Pool and compare. Calculate the share of answers that mention you, overall and per assistant. Only treat a change as real if it's bigger than your margin of error.

Month	Answers checked	Mentioned us	Share of answers	Competitor A	Facts wrong
July	120	18	15%	41%	3
August	120	22	18%	39%	1
September	120	31	26%	40%	0

At 120 answers a month, the margin of error is roughly ±9 points by Gumshoe's formula, so the July to August move in the example above is noise. July to September starts to look like a real change. That's the kind of judgment this method forces, and it's the point.

The bigger shift

Dropping rank isn't only a measurement fix. It changes what you work on.

If you're chasing position, you obsess over one page and one prompt. If you're chasing share of answers, you start asking why an assistant would think of you at all across many different questions. The answers to that are the boring, durable things: clear product facts on your own site, consistency with your retailer listings, real reviews that mention specific uses, and mentions in the sources assistants trust. Those are also the things that move your visibility in a way that lasts.

The dice will keep rolling. You can't control which way they land on a given day. You can control how many of the faces have your name on them.

A note on method: the share-of-answers approach above is close to how XENA Intelligence scores products in SearchPanel, by asking assistants the kinds of questions shoppers ask and keeping the answers.