No Surprise Here: Experts Value Clinical Relevance Over Statistical Relevance

By Abby Proch, executive editor, Clinical Leader

An exercise to raise a debatable, even contemptuous, question about clinical research to a group of experts quickly showed they shared a single POV. Rather than yielding a respectful debate, everyone lined up on one side of the issue.

But rather than scrapping the idea, I decided there’s value in exploring the unity brought on by the question: “Agree or disagree? Statistical significance is preferable to clinical relevance.”

Everyone Disagrees

Of all the respondents to this prompt, not a single one said they’d prefer statistical significance over clinical relevance. Below, hear from a myriad of experts why:

Bhargav Raman, MD, MBA, principal, Sagen Consulting

“Statistical significance is not preferable to clinical relevance; rather, it is a mandatory prerequisite. A p-value below 0.05 simply suggests a low probability that an observed effect is due to chance. As we know, if you run 20 random studies, one is likely to turn up “significant” by pure chance. This is where clinical relevance becomes essential — it serves as the critical lens to determine if a statistically significant finding translates into a meaningful real-world benefit for patients.

From my experience in specialty pharmaceutical management from the payor side and now at Sagen Consulting advising provider groups, payors, employers, and hospital groups, I’ve seen the immense pressure from all sides — pharma, biotech, patient advocates, and even regulators — to conflate statistical significance with clinical utility. This is often done by overextending results to contexts for which a study was not designed, frequently without valid evidence supporting that extension.

A perfect example lies in oncology, where surrogate endpoints like progression-free survival (PFS) are often used to predict quality of life or overall survival (OS) benefits. These claims are frequently made without evidence. For instance, the FDA granted accelerated approval to a $24,000-per-month drug called Blenrep (belantamab mafodotin-blmf) for multiple myeloma based on response rate, but the indication was later withdrawn after confirmatory trials failed to show a survival benefit.

When the bar for approval is lowered to endpoints that are statistically significant but not clinically relevant, investment shifts toward gaming the system rather than funding research that helps patients live longer, better lives. The two concepts must work in tandem.”

Katherine W. Perry, MD, division chief, nephrology, medical director, chronic hemodialysis; program director, peritoneal dialysis, Phoenix Children’s

“Statistical significance is a valuable tool in research, but it can be overemphasized in large studies where even small clinically irrelevant differences can meet a p-value below 0.05. For example, blood pressure differences of just a few points can be labeled as significant just because the sample size was large. But from a clinical perspective, that difference may not be as meaningful.

In my own research in pediatric nephrology, we often work with small data sets. Because the sample size is small, we might not hit statistical significance, but that does not mean the findings aren’t important. Sometimes the story the data tells is biologically and clinically meaningful, even if the p-value doesn’t meet an arbitrary threshold.

What matters more to me is the effect size and how it’s communicated. This can be through confidence intervals, the number needed to treat, or other measures that reflect real-world impact. Confidence intervals are an essential tool in research because they give clinicians a sense of the magnitude or uncertainty of an effect.

Prioritizing statistical significance over clinical relevance can mislead clinicians. We need to focus on effect size, confidence intervals, and, most importantly, patient-reported outcomes along with p-values.”

Susanne Mitschke, healthcare entrepreneur and clinical research expert

“Clinical relevance is more important than statistical significance. Statistical significance tells us whether an observed effect is likely due to chance, but it does not tell us whether the effect actually matters to patients. A study can easily reach p < 0.05 with a large enough sample size, even if the difference is trivial in real life. In other words, statistical significance can be manipulated, but clinical relevance cannot.

Clinical relevance gets to the heart of why we run trials in the first place: to improve patient outcomes in ways that are meaningful and felt. Whether it’s improved sleep, reduced pain, or better quality of life, the ultimate measure of success is whether the intervention makes a tangible difference to people’s lives.

This distinction is especially important for patient retention. If patients do not feel or experience a meaningful benefit, they are less likely to stay on a drug, supplement, or therapeutic product long term, regardless of what the p-value says.

As an industry, we need to prioritize effect sizes and patient-reported outcomes, not just statistical thresholds. Clinical relevance ensures that research translates into real-world impact, which is ultimately what builds both scientific credibility and patient trust.”

Lew Bender, CEO, Intensity Therapeutics

"The FDA will likely reject the statistical significance of data in an endpoint that lacks clinical relevance or meaning. In my 33 years of experience dealing with the FDA, when meeting the agency, the clinical reviewers will insist on an endpoint that is clinically relevant and highly meaningful. Such an endpoint can often be a proven surrogate endpoint that has a strong correlation with the clinical gold standard. Soft endpoints will be harder for the agency to accept than gold standards. For example, in cancer, an endpoint of progression-free survival is weaker than overall survival. A further point of influence on the importance of statistics versus relevance is whether the study design includes a control arm and whether the control arm is a placebo or an active agent (and the choice of the active agent).”

Akshaya Srikanth Bhagavathula, Ph.D., FACE, associate professor of epidemiology, North Dakota State University

“Statistical significance is not always better than clinical relevance. A p-value is an indicator of how probable the observed data is under a null hypothesis; it is not a measure of the patient's benefit. Clinical trials that are proud of having a p < 0.05 without taking effect size or patient-reported outcomes into account may lead to results that are right in terms of statistics but they have no practical meaning.
Clinical relevance is the question of whether the effect changes the length of people's lives, their functions, or their feelings. As an illustration, a cancer drug trial may demonstrate a statistically significant survival improvement of three weeks, but with no quality of life. From a statistical lens, the trial achieved its goal. From a patient's perspective, it failed.

Moreover, the problem is deepened by dependence on averages. In a “significant” pooled effect, it may be that the benefits for some groups are minimized or even that the effects of these groups get worse, while the rest of the population derive good from it. Significance alone cannot differentiate between heterogeneity and justice.

Thus, the future trials must concentrate on estimation more: effect sizes with confidence intervals, number-needed-to-treat, and patient-reported outcomes. Furthermore, Bayesian models and real-world evidence are sources of richer context. The use of statistical significance can still be a checkpoint; however, it should never be more than a clinical meaning. Research is not done to get significant p-values; rather, it is done to provide evidence that has value for patients and that guides practice.”

Bob Discordia, cofounder, president, and CEO, EQUULUS Therapeutics

“I would disagree with the statement that statistical significance is preferable to clinical relevance, but the reason is nuanced. Statistical significance (commonly defined as p < 0.05) only indicates that an observed effect is unlikely due to chance, but it does not address whether the effect is meaningful for patients, improves outcomes, or warrants a change in practice. A study can achieve statistical significance yet provide only a trivial benefit, while clinically important effects may sometimes go undetected in underpowered studies. For this reason, the importance of interpreting p-values in the context of effect size, confidence intervals, and patient-reported outcomes cannot be overstated. In short, statistical significance is necessary but insufficient without context.”