How to run a fair AI visibility test across answer engines

A café owner looking for a bookkeeper in Carlton is asking a different question from someone comparing accountants across Australia. Keep that distinction when you test AI visibility. Write down the exact customer questions, market and settings, then use them again when you check the results.

Save the full answers and the pages they cite. Over repeated checks, that record helps you see which businesses are recommended and which sources are used. The sections below show what to record and how to compare the answers without changing the question halfway through.

Choose questions your customers would ask

Choose a buying decision you care about: finding a local supplier, comparing tools or working out a budget. Write it as a customer would. If you want to learn whether the engine finds your business, leave your brand name out of the question. Keep questions about your brand in a separate group.

Use the market and language you want to test. A US result will not tell you whether customers in Australia see you. A country setting also cannot establish what every suburb sees, so keep a note of the setting you actually used.

Keep a short list of questions for regular checks. Add new questions to a separate list when you want to explore. If you keep changing the main list, you will not know whether a better score came from better visibility or easier questions.

Keep a record of the setup

Start with a fresh conversation and record any settings that might affect the result. If you have already discussed your company with the assistant, keep that answer out of the comparison. Apply the same rule to good and bad results, and record why an answer was excluded.

Keep product surfaces distinct. Google documents Search grounding for the Gemini API, including access to web information and citations. That does not make an API response interchangeable with an answer in a consumer interface. Record which method you actually tested.

Decide what you are counting

• Brand mention: the answer explicitly refers to the intended business, after resolving ambiguous names.
• Recommendation: the answer presents the business as a relevant option, rather than only discussing it.
• Citing answer: the answer includes at least one citation to your site.
• Cited page: a distinct URL used within an answer; count repeat passage links to the same page once for page coverage.
• Disclosed search: a query the provider actually exposes. No visible query is not proof that no retrieval happened.

Show each engine’s results separately. For example, if two of 20 completed answers cite your site, that is 10% for those questions. Three links in each of those two answers still means two citing answers. Show failed runs alongside the result so readers can see what was checked.

Read what the link supports

Open the source link and read the sentence it supports. It could point to a price, a definition, a comparison or your description of another company. Record whether the answer actually recommends you as well as whether it uses your page.

Look at your closest page next. Would a pricing example help? Is a comparison missing a useful detail? Our fan-out content guide shows how to turn those questions into edits. You can investigate what made a source useful without pretending to know exactly why an engine selected it.

Check again after the page goes live

Write down what you changed and when it went live. Then repeat the checks at similar intervals. Where possible, keep an unchanged topic for comparison. Changes to engines, competitor pages and indexing can affect the results too.

We review pages after seven, fourteen and twenty-eight days. Those are useful reminders to look again, not deadlines for a ranking change. At each review, check that the questions, engine coverage and collection setup still match.

Google’s guidance for AI features says the usual search fundamentals apply and that eligibility does not guarantee crawling, indexing or serving. Focus on useful, accessible content and internal links. A special file or extra posting frequency is not a guarantee of inclusion.

Separate visibility from business results

Track cited answers, referred visits, qualified enquiries and trial registrations as distinct outcomes. An engine can cite a page without sending a visitor, and a visitor can arrive through another route. For Google AI Overviews and AI Mode, Search Console reports their traffic within its overall Web search reporting, so a change in that total alone cannot identify the exact AI surface.

Keep the conclusion close to what you found: which questions cited you, which pages appeared and whether that happened again. If you are choosing software for the job, our AI visibility buyer’s guide covers what to look for.