Compare independent sources, their definitions and their evidence before calling a view a consensus. Several articles repeating one study still represent one source. Show disagreements and uncertainty, then explain which decision the evidence supports.
Three summaries of the same press release are still one source.
Use a consensus report to find what the evidence supports, where it conflicts and what remains unknown. Agreement between models is not a substitute for agreement between independent sources.
Step 1: Write the decision as a question
Use a bounded question: “Which checkout issue should we investigate first?” Include the market, period and decision deadline.
State what would change the decision. That keeps business intelligence tied to an action instead of an expanding reading list.
Step 2: Build a source register
Collect original documentation, observed records or research with an inspectable method. Record the URL, publisher, date, scope and relevant passage.
An AI search tool can help locate material: OpenAI web search and Claude web search document source citations. Open those sources yourself. A citation is a pointer, not a completed verification.
Step 3: Trace independence before counting agreement
Ask where each claim originated. If several articles cite the same survey, mark them as one evidence family.
Compare definitions, samples and dates before treating findings as compatible. Do not average values that describe different populations or use different denominators.
Step 4: Classify each finding
| State | Meaning | Action |
|---|---|---|
| Agreement | Comparable independent evidence supports the claim | Use it within that scope |
| Contradiction | Comparable evidence points in different directions | Investigate method or context |
| Unknown | Evidence is missing or not comparable | Name the next check |
These labels describe the evidence collected, not universal certainty. A documented minority finding stays in the report.
Step 5: Challenge the draft
Have a reviewer or model check the draft against the source register. Ask it to identify specific errors and unsupported conclusions.
Audit this brief using only the supplied sources. For each claim, identify its evidence family, supporting passage and limits. Flag copied evidence, incompatible definitions and conclusions stronger than the records. Do not add facts. List the smallest checks that could change the recommendation.
Step 6: Deliver one usable page
Use this structure:
- Decision: question and scope
- Supported: finding, evidence and limits
- Disputed: competing findings and their methods
- Unknown: missing evidence and next check
- Action: recommendation, owner and review date
Remove any recommendation that cannot survive its own source register. Keep uncertainty visible; the brief should make the decision clearer, not merely sound unanimous. For a marketing decision, use the channel planning guide to compare the options against budget and team capacity.
A worked evidence-family check
The following evidence is wholly fictional, showing how to reason rather than claiming real research results. The business question is whether a retailer should test clearer delivery costs before changing its checkout design.
| Record | Evidence family | Finding | Limit |
|---|---|---|---|
| Survey A | One shopper survey | 40 of 100 respondents say unexpected delivery costs concern them | Self-reported concern; not measured abandonment |
| Articles B and C | Both repeat Survey A | Repeat the same 40-of-100 figure | No independent observations added |
| Store log D | Retailer's own coded support requests | Six of 20 checkout-related requests ask about delivery charges | Support contacts are not all shoppers |
| Experiment E | A different retailer's small test | Eight purchases in one group and ten in the other | Different population; insufficient detail to claim a reliable effect |
A finished one-page decision could say:
Decision: test whether clearer delivery information addresses a known customer question before redesigning the checkout.
Supported: the fictional survey and the store's separate support log both identify concern about delivery costs, within their different populations. Three publications do not turn the survey into three sources.
Not established: how much abandonment the issue causes, or whether the small experiment generalises to this store. The two purchase counts alone do not establish a meaningful improvement.
Next action: verify the current delivery explanation, fix any factual omission, and design a proportionate test with a defined outcome and adequate observation period. The commerce owner records the decision and review date.
That recommendation is narrower than “transparent shipping always increases conversion”. It is also something the business can act on.
In the source register, record original study, population, period, measure, finding and limitation. A useful conclusion might be that the survey suggests a question worth testing locally, while the experiment offers a possible method. That is weaker than a universal claim, but more actionable. Keep the original URLs and relevant passages with the decision so another person can challenge it.
Keep the evidence family with the decision
Use one row per claim and original evidence family. If two articles repeat one study, give them the same family ID. Preserve the relevant passage, population, limitation and next check; do not count model agreement as another source.
Download the evidence-family worksheet. Open the CSV in your spreadsheet application, keep the specimen labelled, and add your own rows. Pending observations are work to complete, not failed or successful results.
Sources
- OpenAI web searchdevelopers.openai.com
- Claude web searchplatform.claude.com
