Turn a research question into an evidence table that survives disagreement

Start with a question that can reject an answer

A useful research question sets a boundary. Write the decision it informs, the population or situation it covers, the period that matters, and what would count as contrary evidence. “Which workflow is best?” invites a pile of opinions. “For a two-person support queue, which documented workflow prevents two people from replying to the same message?” gives the researcher something observable to collect.

Then freeze that wording at the top of the working sheet. If the question changes, record a new version rather than quietly bending the evidence to fit. The Cochrane Handbook’s data-collection guidance is written for systematic reviews, but its practical discipline transfers well: use a structured, piloted form; keep the form tied to the question; and record enough unambiguous source data that another person can check it. The companion PDF extraction quality check is the right companion when the source is difficult to copy reliably.

Give every source its own row

Do not begin with a narrative document. Begin with one row per source claim. A source can occupy several rows when it supports several distinct claims. That makes disagreement visible instead of blending it away.

The PRISMA 2020 checklist is also aimed at formal reviews, not ordinary office research. Its reporting emphasis is still useful: disclose why the work was done, where the material came from, what was selected, and how the result was produced. A lightweight team does not need to imitate an academic review. It does need enough structure to reconstruct its own path.

Treat disagreement as a result

When two rows conflict, do not average them into mush. First check whether they actually describe the same population, date, definition, and outcome. One source may measure reported time saved while another measures elapsed task time. One may describe a feature as launched while another documents only a preview. Add a “difference” note that identifies the mismatched dimension.

If the conflict remains, retain both rows and write a resolution rule before choosing: prefer the primary specification for feature behavior, the original study for measured outcomes, and the newer source only when the underlying fact can genuinely change. Mark the conclusion “unresolved” when the rule does not settle it. Cochrane explicitly recommends defining how discrepancies will be resolved and preserving the original extractions as well as the consensus record. That is a better model than overwriting the losing interpretation.

Make the handoff testable

Before synthesis, ask a teammate to reproduce two claims using only the table. They should be able to open the source, find the cited location, understand the scope, and see why a conflicting row was retained or resolved. A hypothetical example: a freelancer comparing two export workflows records one vendor’s documentation as “attachments included” and a second source as “linked files excluded.” The useful conclusion is conditional, not “exports include attachments.”

1. Freeze the research question and version it.
2. Pilot five source rows before collecting the whole set.
3. Separate quoted evidence, interpretation, and resolution status.
4. Keep disagreements in the table until a written rule resolves them.
5. Have another reader reproduce at least two claims before drafting.

Only after that check should the team write prose. The related archive notes on research-summary error measurement and citation-context labels are useful reminders that a polished synthesis is not the evidence record itself.