Field Guide
What makes clinical evidence hold up.
A plain-language guide to the criteria behind a rigorous read: the dimensions that get checked, the reporting guidelines that define good practice for each study type, how to read a CONSORT diagram, and how discrepancies and non-reproducible results actually surface. Every standard here links to its primary source.
Eight dimensions, evaluated separately.
These are the axes a careful reviewer, and our engine, work through. They are graded independently so a reporting gap never gets confused with a design flaw.
Scientific premise
Is the question grounded in the prior literature, and does the rationale actually motivate the design? A weak premise puts every result downstream on a shaky foundation.
Study design & rigor
Randomization, blinding, controls, and sample size appropriate to the claim. The safeguards that keep bias out are the ones a reader should be able to find, not infer.
Biological variables
Whether sex, age, strain, genotype, and other key variables are accounted for in both design and analysis, rather than averaged away or left unreported.
Ethical approvals
IRB or IACUC approval, informed consent, and trial or study registration where the work requires it. Absence here is often an omission, not misconduct, but it still has to be checkable.
Key resources
Antibodies, cell lines, reagents, organisms, and software identified precisely enough to be reused: catalog numbers, RRIDs, versions. Unidentified resources cannot be reproduced.
Statistical analysis
Tests appropriate to the data, assumptions met, multiplicity handled, and the reported numbers internally consistent with the counts and intervals around them.
Data & code availability
Underlying data and analysis code accessible through working links, with enough detail that an independent group could rerun the pipeline and land in the same place.
Reporting transparency
The relevant reporting checklist followed, and methods complete enough to evaluate and repeat. Transparency is what makes every dimension above verifiable in the first place.
Reporting guidelines, by study type.
Good reporting is not a matter of taste. For most study designs there is a community checklist that defines what a complete report contains. Match the guideline to the design, then check the paper against it.
Consolidated Standards of Reporting Trials — the 25-item checklist and flow diagram for reporting an RCT.
Strengthening the Reporting of Observational Studies in Epidemiology — cohort, case-control, and cross-sectional.
Preferred Reporting Items for Systematic Reviews and Meta-Analyses, including the study-selection flow diagram.
Animal Research: Reporting of In Vivo Experiments — the 2.0 essential-10 for preclinical work.
Standard Protocol Items: Recommendations for Interventional Trials — what a trial protocol should specify up front.
Standards for Reporting of Diagnostic Accuracy Studies.
Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis.
CAse REport guidelines for a complete, transparent single-case write-up.
The full catalog, with many more designs, lives at the EQUATOR Network, which indexes over 500 reporting guidelines.
How to read a CONSORT flow diagram.
For a randomized trial, the flow diagram is the single most information-dense object in the paper. It tracks every participant from screening to analysis, and it is where dropouts and exclusions have to be accounted for.
- Enrollment
Assessed for eligibility
Excluded: did not meet criteria, declined, other reasons
- Allocation
Randomized, then allocated to each arm
Received allocated intervention vs. did not, with reasons
- Follow-up
Followed over the study period
Lost to follow-up or discontinued, with reasons, per arm
- Analysis
Analyzed for the primary outcome
Excluded from analysis, with reasons, per arm
A worked template of this diagram lives on the CONSORT site.
The numbers have to reconcile.
A CONSORT diagram is an accounting statement for people. Read it as one:
- Every participant enrolled should appear again at allocation, follow-up, and analysis, or be explained by a stated exclusion.
- Losses to follow-up are expected. Losses without a reason, or that differ sharply between arms, are the ones that move a conclusion.
- The count analyzed should match the analysis in the tables. When it is smaller, ask which participants dropped out, and whether that was decided in advance.
The same logic reads a PRISMA diagram for a systematic review: records identified, screened, excluded with reasons, and finally included have to reconcile top to bottom.
How discrepancies and non-reproducible results surface.
Most defects are not hidden. They are visible to anyone who cross-checks the parts of a paper against each other, against the protocol, and against the sources it cites. These are the checks that do it.
- Recomputing the statistics
Reported test statistics, degrees of freedom, and p-values imply one another. Recomputing them (a statcheck-style pass) shows when a p-value cannot follow from the numbers beside it.
The tell: A p-value that is inconsistent with its own test statistic.
- Internal consistency
Denominators, group counts, table totals, and confidence intervals all have to agree with each other and with the text. Numbers that do not add up are the most common, most checkable defect.
The tell: Arm sizes in a table that do not sum to the reported total.
- Protocol-to-publication concordance
The registered protocol and analysis plan set the primary outcome and analysis in advance. Comparing them to the paper reveals outcomes that were switched, added, or quietly dropped.
The tell: A primary endpoint in the registry that is not the one reported.
- The participant flow
A CONSORT diagram is an accounting statement for people. When enrolled, allocated, and analyzed counts diverge without stated reasons, the missing participants can carry the effect.
The tell: Dropouts that appear at analysis but never in the flow diagram.
- Citation and link integrity
Claims lean on references and on declared data and code. Resolving each citation to its source, and probing each data or code link for liveness, separates a real pointer from a dead or mismatched one.
The tell: A “data available on request” statement with no working repository.
- Reproducibility of the pipeline
Where data and code are posted, the question becomes whether they actually regenerate the reported figures. Content-level checks probe whether the shared materials match the paper.
The tell: Posted code that references files absent from the deposit.
Registries worth cross-checking.
A registration made before a trial begins is the reference point against which the published result is judged. These are the public registries a report should point to, and where an independent reader can verify what was promised.
Primary sources.
This guide is a map. These are the standard bodies and tools that maintain the territory. All external, all authoritative.
A searchable index of 500+ reporting guidelines, the umbrella for everything above.
ICMJE RecommendationsThe editorial standard for conduct, reporting, and authorship in medical journals.
NIH Rigor & ReproducibilityThe funder framing of premise, design, biological variables, and resource authentication.
statcheckThe open tool that recomputes reported statistics, one of the checks described above.
Retraction WatchA running record of what happens when problems surface after publication.
From guide to practice
Adcurare runs these checks, at scale, on real evidence.
Every finding traces back to its source, graded on the eight dimensions above and against the guideline that fits the design. See how the method works, or watch it read an asset you bring.