One of the most quietly damaging habits in health and medical research is the casual equation of 'statistically non-significant' with 'no effect.' This methodological error has shaped clinical guidelines, discouraged promising interventions, and given false reassurance about genuine risks — making this PNAS methodological paper directly relevant to how anyone reads health research.
The core argument centers on a fundamental logical error: a p-value at or above 0.05 tells researchers only that their data failed to clear a conventional significance threshold, not that an effect is absent. The paper outlines the formal framework of equivalence testing — particularly two one-sided tests (TOST) — as a principled alternative. Rather than asking 'is this effect significant?', equivalence testing pre-specifies a smallest effect size of interest (SESOI) and tests whether the observed effect falls within that 'equivalence zone.' Only when confidence intervals fall entirely within those pre-defined bounds can researchers legitimately conclude practical equivalence. The paper appears to synthesize existing statistical methodology while calling for broader adoption, particularly in disciplines where null results carry clinical weight.
This contribution arrives in the context of a decade-long replication crisis that has exposed p-value misuse as a structural problem, not an occasional oversight. Statisticians including Lakens, Wellek, and others have been advancing equivalence and non-inferiority testing for years, but uptake in mainstream biomedical literature remains uneven. The practical implication for health-conscious readers is interpretive: when a study reports that a supplement, dietary change, or intervention 'had no effect,' the critical question is whether equivalence was formally tested or whether researchers simply failed to reject a null hypothesis. The distinction is not semantic — it separates genuine evidence of no effect from the absence of evidence. This paper does not introduce new data but could be paradigm-reinforcing for researchers and meaningfully improve how null findings are communicated to clinicians and the public.