How to test social listening keywords and exclusions

Test listening keywords against a small set of conversations you would want to keep, reject or review before applying them to your daily feed. Include awkward examples that share the same words. Change one rule at a time and check what it removes as well as what it finds.

For a local repair business, “Need someone for a small repair job” could be a useful enquiry. “Looking for a weekend job” is probably an employment request. Excluding the word “job” would remove both. These are invented examples, but the decision is a practical one: a quieter feed is useful only if it still contains the conversations your business can help with.

Define the work you would accept

Start with a short service brief. Describe the work, the area you cover and any conditions that would make a request unsuitable. An illustrative repair brief might say: small residential repairs in a named service area, with the original post available for review; exclude recruitment and supplier advertising. An unknown location goes to review rather than being assumed local.

Keep customer language alongside your service names. A person might ask for someone to “fix a sticking door” without using the phrase “property maintenance”. A list made entirely from your website's category labels can miss how people describe the problem.

Vendor guides such as Hootsuite's lead-generation workflow and Buska's tool comparison emphasise buying signals and acting on relevant conversations. For your setup, turn that broad principle into examples with explicit reasons to keep or reject them. Their product claims and rankings are not comparative accuracy evidence.

Build a test set with difficult examples

You can begin with invented examples, clearly labelled as such. Add appropriately handled real examples from your own review process later. Keep any private customer material within your authorised systems; you do not need to upload it to a new tool to write a useful brief.

The table below is a fictional test set for the repair business. “Review” means more context is needed before calling it a lead.

Add examples of place-name ambiguity, recommendations made by someone else, repeat shares of one request and services you do not provide. The point is to expose predictable mistakes before making a rule broad enough to affect the whole feed.

Separate matching from qualification

A keyword match gives you a conversation to inspect. It does not establish buying intent, location, availability or permission to promote your business. Keep those decisions separate in your test notes.

Organise candidate terms into three groups: the service or problem, language suggesting someone wants help, and location context. Treat these as a planning structure, not a universal search command. A tool may offer exact phrases, Boolean operators, semantic matching, dedicated location settings or a combination. Ask how each input is interpreted before pasting in a complex expression.

For example, requiring both “repair” and “recommend” could miss “Who can fix my door?” Requiring a suburb name in every post could miss a relevant request where the location appears only in the thread. On the other hand, assuming every member of a local group lives nearby can include people outside your service area. Put uncertain cases into review.

If you need to establish which conversations a tool can access in the first place, use the separate source-coverage test. Keyword tuning cannot recover material outside the product's collection scope.

Test exclusions against the examples you want to keep

Before adding an exclusion, write its intended purpose. “Remove employment requests” is a clearer requirement than “exclude job”. Consider a narrower phrase or a review rule if a single-word exclusion also removes customer requests.

Run every proposed exclusion against the keep examples, including natural variations. Check “small job”, “after work”, “repair quote” and other phrases relevant to your business. A term that often appears in unwanted posts can still carry useful meaning in a genuine enquiry.

Do not assume the tool supports a specific operator or automatic exclusion workflow. Demonstrate the rule in the available interface. Where the product lacks the required control, keep the distinction in the manual review brief and assess whether that workload is acceptable.

Compare one change using a fixed review sample

Save the original settings and a dated copy of the test brief. Change one term or rule, then compare the same examples before and after. Record the intended decision, actual decision and reason for each disagreement. This makes an improvement inspectable rather than relying on the feed feeling cleaner.

If a tool cannot replay old conversations, keep that limitation explicit. A hand-checked example exercise tests your rule design; a later live sample tests the whole workflow under different conditions. Different conversations, source availability and collection times prevent a clean claim that the keyword change alone caused the result.

Useful review measures include relevant conversations among those inspected, known good examples incorrectly removed, duplicate requests and review time. You cannot calculate how many relevant conversations across the entire internet were missed from a feed of collected matches. Nor does a high relevance rate establish enquiries or revenue; track those separately using the lead outcome worksheet.

Take a short acceptance brief into the trial

Give the person reviewing results the service brief, example set, current settings and reason for the latest change. Agree what happens to unclear location, closed requests and posts where promotion is unwelcome. Keep the original conversation link so decisions can be checked.

Reputably's social listening workflow helps local teams and agencies review buying-intent conversations with location context and a follow-up process. Use the trial to inspect actual results against your brief. This testing method does not imply a built-in replay engine, a particular Boolean syntax or guaranteed access to every conversation.

Bring a few examples of work you would take and a few near misses to your Reputably trial. For the wider purchase decision, compare listening tools by the workflow and sources you need. Keep the settings that save review time while preserving the opportunities you can genuinely help with.