Your analytics cannot see the shortlist

Your analytics can tell you who arrived from ChatGPT. It cannot tell you whether ChatGPT mentioned you before the buyer chose a shortlist. If you lead marketing at a B2B company, that is the part worth checking.

This creates an awkward measurement problem.

You can have very little AI referral traffic and still be influencing buyers before they visit your website. You can also have some AI-sourced visits without being one of the companies buyers remember when the shortlist is formed.

The click is visible. The recommendation may not be.

The visit is often the second event

Imagine a buyer trying to solve a technical problem.

They ask an answer engine how to approach it. The response gives them a few companies to consider, or explains the kinds of tools and services they should look for. The buyer then speaks to two or three vendors, visits their websites, and books a meeting.

Your analytics may show the later visit. It will not necessarily show that your company was considered earlier, mentioned alongside alternatives, or ruled out before the website visit happened.

That makes AI referral traffic useful, but incomplete.

It tells you that someone arrived through an identifiable path. It does not tell you how often your company appears when a buyer asks a high-intent question, what the answer says you are good at, or whether the buyer heard your name before they searched for you directly.

This is why referral traffic is a lagging signal for AI visibility. It records one outcome after the answer has already shaped the buyer’s options.

Picture someone hearing about a supplier from a colleague, then visiting the supplier’s website later. The analytics report credits the visit. It does not capture the conversation that made the visit likely.

Answer engines create a similar blind spot, only with worse reporting and fewer people willing to admit they asked a robot for help.

The face-to-face objection misses a step

The most common pushback I hear is: “Our industry buys face-to-face, not from AI.”

Fair enough. In many B2B categories, the final decision still involves meetings, references, procurement, technical reviews, and a surprising amount of calendar negotiation.

But that does not mean AI has no influence.

The buyer may use an answer engine before the first meeting to understand the category, narrow the options, or work out which questions to ask vendors. By the time they speak to you, they may already have a shortlist and a rough view of what each company is known for.

Face-to-face buying still includes a shortlist stage. That stage may begin before the first meeting.

If you judge AI visibility only by visits from an answer engine, you are measuring the part that is easiest to see, not necessarily the part that affects the buying process first.

What to inspect instead

Start with the prompts your buyers might actually use.

Not branded prompts such as:

“Why is [your company] the best option for this problem?”

Those are useful for some checks, but they do not tell you whether the market understands you when your name is absent.

Use category and problem prompts instead. Ask how a buyer would solve the problem your product solves, without mentioning your brand.

For an initial check, choose one real buyer problem and use the same prompt across the answer engines. For example:

“How would you solve [high-intent buyer problem]?”

Do not add your company name. Do not quietly improve the prompt until it gives you the answer you hoped for. That is a very human form of quality assurance, but it is still cheating. Once you have run the initial check, you can expand it to more buyer problems.

Then record three things:

1. **Whether your company is named at all.**
2. **Which two or three alternatives are named instead.**
3. **The reason given for each recommendation.**

After the check, you can optionally note whether the reason matches the position you want buyers to remember.

That last point matters more than simple inclusion.

Being named as a low-cost option is not much help if you want to be known for technical depth. Being described as an implementation partner is useful if that is your commercial position, and confusing if you sell a product customers are meant to run themselves.

Visibility without the right meaning can create a different problem. Buyers may find you, but for a reason you did not choose.

The two-minute shortlist check

Here is the manual check I would run before making a larger decision about AI visibility.

1. Pick one real buyer problem

Choose a problem with commercial intent, not a broad topic.

Use the language a buyer might use when they are trying to decide what to do next. “How can we improve marketing?” is too broad. “How would you solve declining organic search for a technical B2B company?” is more useful.

You can also use a prompt around implementation, vendor selection, measurement, or a known operational pain. Start with one problem for this first check; expand to more problems once you have a baseline.

2. Open three answer engines

Check the same prompt in ChatGPT, Perplexity, and Gemini.

Use the same prompt in all three engines. You are checking whether the same pattern appears across them, not deciding which engine is best.

3. Ask the same question each time

Use this template:

“How would you solve [buyer problem]?”

Do not mention your company or your competitors. If the answer gives an instruction manual instead of a list of brands, record that too. It tells you that the answer engine is describing a solution without attaching your category to specific companies.

4. Record three things per result

Keep it simple:

• whether your company is named at all
• which two or three companies are named instead
• the reason given for each recommendation

You can put this in a small spreadsheet. One row per answer engine is enough for the initial check. Add an optional final column for whether the answer reflects your intended positioning.

This should take about two minutes if you resist the urge to turn it into a research project. You are looking for a first signal, not preparing evidence for a parliamentary inquiry.

How to read the result

Do not treat one answer as a verdict on your marketing.

Answer outputs can change. A company may appear for one problem and disappear for another. The wording of the prompt matters. The answer may list no companies at all.

So do not ask, “Did we show up once?” Look for the pattern across the buyer problems that matter.

You might find that:

• your company appears for the category, but not for the problems buyers pay you to solve
• competitors appear with clear reasons, while your company is absent
• your company is named, but for a position you no longer want to own
• the answers describe your category correctly but do not connect it to any company
• your presence is inconsistent across the three answer engines

Each result points to a different marketing problem.

Absence may mean your positioning is hard to associate with the problem. The wrong reason may mean your public evidence supports an outdated position. Competitor inclusion may show that their language is easier for answer engines, and buyers, to connect to a specific use case.

None of those conclusions comes from referral traffic alone.

What marketing leaders should measure

I would use AI referral traffic as downstream evidence, not as the whole budget test.

The more useful questions are:

• Do we appear for the category and high-intent problems that matter commercially?
• Are we being recommended alongside credible alternatives?
• What reason is given for recommending us?
• Does that reason match the position we want buyers to remember?
• Does the pattern hold over time?

This shifts the check from website visits to whether answer engines recommend you for the problems buyers actually need to solve.

Run the two-minute shortlist check for your own category and one high-intent buyer problem. Once you have a baseline, expand the check to more problems.

Record who gets recommended, what each company is recommended for, and whether your company appears at all. Save the answers and repeat the check over time.

The shortlist is where I would start looking. Your analytics probably cannot see it.