Test whether an intent score helps your team choose what to read

Test a social listening intent score by reading a small, fixed sample of conversations and deciding which deserve your team's attention before you reveal the scores. Then compare your decisions with the tool's ordering. Count useful requests near the top, relevant requests lower down and the time needed to make a decision. A high score should help prioritise review; it does not establish a person's budget, permission to contact them or probability of becoming a customer.

This is a buyer-run acceptance test for a local business or agency choosing a listening tool. It is not a published accuracy benchmark, and the example numbers below are invented. The aim is to find a workflow that makes your own review time worthwhile.

Agree what a useful request means for this business

Write a short acceptance rule before looking at the feed. For example: “A public request for move-out cleaning in an area we serve, where the person is still choosing a provider and a relevant response is possible.” Confirm the service, location, availability and source rules with the person who will actually handle enquiries.

Keep three judgments separate. Intent asks whether the person appears to be considering a purchase. Fit asks whether your business can meet the need. Actionability asks whether there is an appropriate next step now. An urgent request can have strong intent and still be outside your service area. A useful research question may deserve a helpful answer without being a sales opportunity.

Reputably's score guide distinguishes buying intent from sentiment and describes scores as an aid to human judgment. An angry comment and a request for a supplier therefore need different decisions. For the specific case of dissatisfied customers, use our complaint qualification guide.

Choose a sample that can reveal both strengths and misses

Take a consecutive set of available conversations from a stated period, rather than selecting the most convincing examples. Record the business brief, sources, date range, keywords and filters. If volume is low, use everything available and state the small sample size. A quiet trial is not evidence that the wider market has no demand.

Include material below your usual priority cutoff where the tool exposes it. Reviewing only the highest scores can tell you how useful that shortlist is, but cannot show which good requests were left out. If the trial exposes only a filtered feed, label the exercise “shortlist review”; do not report recall across all monitored conversations.

For a manageable first exercise, a team might inspect 30 or 50 items. Those are practical workload choices, not statistically representative sample requirements. A narrow sample cannot establish accuracy across languages, platforms, clients or seasons. Keep each client's results separate until you understand the differences.

Open the original conversation when available and note any missing context. Remove duplicate copies of the same request from a buyer-count calculation using a consistent deduplication rule. Keep an inaccessible or ambiguous item in an “unresolved” group rather than guessing.

Label the conversations before revealing the scores

Use a private worksheet with the source link, your decision, the evidence for it and any unresolved question. Hide the tool's score while making the first decision so that a large number does not become the reason for accepting an item. If two people will operate the workflow, have them independently review part of the same sample, then resolve disagreements against the written rule.

These labels belong to the evaluation worksheet; they are not claims about built-in product stages. If location is unclear, use the original request and the service-location checks before assigning a fit judgment. Keep personal details out of shared reports unless you have an authorised reason to include them.

Compare the shortlist with your decisions

Now reveal the ordering. Choose the review capacity you can actually sustain: for instance, the first ten conversations in a daily review. If a tool offers a configurable cutoff, record it. If it does not, evaluate the top portion of the feed without implying that a threshold control exists.

Suppose an invented sample contains 30 conversations with resolved labels. Your team finds eight worth checking. Six of those appear in the top ten; the other two sit lower in the feed. The calculation is:

• Useful share of the shortlist: six out of ten, or 60%. Four shortlisted items did not meet this business's rule.
• Coverage within the reviewed sample: six out of eight, or 75%. Two useful items in that sample fell below the chosen cutoff.
• Review effort: the actual minutes spent resolving the shortlist, plus any time spent finding or checking missing context.

The 75% figure describes the labelled sample only. It says nothing about posts the tool never collected, private conversations or unavailable sources. Test collection coverage separately. Neither percentage is a close rate, and neither is a Reputably or competitor result.

Report unresolved items beside the calculation. If three more items could not be assessed, “30 resolved, three unresolved” is more useful than silently dropping them. A score also needs its definition: an 80 on one vendor's scale need not mean the same thing as an 80 elsewhere. Buska's buying-signals page, for example, describes scoring signals alongside ideal-customer-profile matching. Ask each vendor which judgments its displayed number combines.

Test one change, then use a fresh sample

Inspect the mistakes before changing settings. Repeated location mismatches suggest a business-brief or filtering issue; repeated informational questions may call for clearer intent criteria. Do not raise a cutoff just to make the accepted share look better: you may also discard worthwhile requests.

Change one supported setting at a time and record it. Then evaluate new conversations using the same acceptance rule. Repeatedly adjusting against the same examples can produce an attractive result that does not hold on the next day's feed. If the tool's ordering works well but the team cannot respond promptly, the next improvement may be ownership or review cadence rather than another filter.

For a vendor comparison, use the same business brief, period and review capacity, and record differences in accessible sources and collection timing. When the tools collect different conversations, separate collection differences from ordering quality. Do not combine the results into a universal accuracy ranking.

Use the exercise to decide whether Reputably fits

Reputably's lead-finding workflow is worth evaluating for local service teams and agencies that want buying-intent review, location context and follow-up alongside AI visibility and Google reviews. It prioritises available conversations for review; the score is not a probability of a sale. Collection follows a configured schedule, and source availability and response routes vary.

In a trial, use your real service brief and inspect the original conversations and the reasons for prioritisation. Record which items your team accepts, which require clarification and how long review takes. Use a private worksheet for this exercise; it does not assume a built-in benchmark, replay feature or adjustable scoring model.

The decision is whether the workflow surfaces enough relevant, timely conversations for your team to review and handle responsibly. Keep later enquiries and won work in a separate outcome record. Compare the available tools and their fit, then test the requirement that matters most to your business.