Home  |  About us  |  Editorial board  |  Ahead of print  | Current issue  |  Archives  |  Submit article  |  Instructions |  Search  |   Subscribe  |  Advertise  |  Contacts  |  Login 
  Users Online: 598Home Print this page Email this page Small font sizeDefault font sizeIncrease font size  

 Table of Contents      
Year : 2015  |  Volume : 6  |  Issue : 3  |  Page : 169-170

Common pitfalls in statistical analysis: Clinical versus statistical significance

1 Department of Anaesthesiology, Division of Thoracic Surgery, Tata Memorial Centre, Parel, Mumbai, Maharashtra, India
2 Department of Surgical Oncology, Division of Thoracic Surgery, Tata Memorial Centre, Parel, Mumbai, Maharashtra, India
3 International Drug Development Institute, Hasselt University, Belgium

Date of Web Publication6-Jul-2015

Correspondence Address:
Priya Ranganathan
Department of Anaesthesiology, Tata Memorial Centre, Ernest Borges Road, Parel, Mumbai - 400 012, Maharashtra
Login to access the Email id

Source of Support: None, Conflict of Interest: None

DOI: 10.4103/2229-3485.159943

Rights and Permissions

In clinical research, study results, which are statistically significant are often interpreted as being clinically important. While statistical significance indicates the reliability of the study results, clinical significance reflects its impact on clinical practice. The third article in this series exploring pitfalls in statistical analysis clarifies the importance of differentiating between statistical significance and clinical significance.

Keywords: Biostatistics, confidence intervals, data interpretation, statistical

How to cite this article:
Ranganathan P, Pramesh C S, Buyse M. Common pitfalls in statistical analysis: Clinical versus statistical significance. Perspect Clin Res 2015;6:169-70

How to cite this URL:
Ranganathan P, Pramesh C S, Buyse M. Common pitfalls in statistical analysis: Clinical versus statistical significance. Perspect Clin Res [serial online] 2015 [cited 2021 Apr 21];6:169-70. Available from: https://www.picronline.org/text.asp?2015/6/3/169/159943

One of the common problems faced by readers (and authors!) of medical articles is in the interpretation of the word "significance." The term "statistical significance" is often misinterpreted as a "clinically important" result. The confusion stems from the fact that many people equate "significance" with its literal meaning of "importance," whereas in statistics, it has a far more restrictive connotation. This article explains the idea of the statistical significance and differentiates it from clinical relevance or importance, which is an entirely different concept. In the previous article, in this series, we looked at different ways of expressing statistical significance ("P0" values versus confidence intervals). [1] Measures of statistical significance quantify the probability of a study's results being due to chance. Clinical significance, on the other hand, refers to the magnitude of the actual treatment effect (i.e., the difference between the intervention and control groups, also known as the "treatment effect size"), which will determine whether the results of the trial are likely to impact current medical practice. The "P" value, frequently used to measure statistical significance, is the probability that the study results are due to chance rather than to a real treatment effect. The conventional cut off for the "P0" value to be considered statistically significant is of 0.05 (or 5%). What a P < 0.05 implies is that the possibility of the results in a study being due to chance is <5%.

In clinical practice, the "clinical significance" of a result is dependent on its implications on existing practice-treatment effect size being one of the most important factors that drives treatment decisions. LeFort suggests that the clinical significance should reflect "the extent of change, whether the change makes a real difference to subject lives, how long the effects last, consumer acceptability, cost-effectiveness, and ease of implementation". [2] While there are established, traditionally accepted values for statistical significance testing, this is lacking for evaluating clinical significance. [3] More often than not, it is the judgment of the clinician (and the patient) which decides whether a result is clinically significant or not.

Statistical significance is heavily dependent on the study's sample size; with large sample sizes, even small treatment effects (which are clinically inconsequential) can appear statistically significant; therefore, the reader has to interpret carefully whether this "significance" is clinically meaningful. A study published in the Journal of Clinical Oncology compared overall survival in 569 patients with advanced pancreatic cancer who were randomised to receive erlotinib plus gemcitabine versus gemcitabine alone. [4] Median survival was found to be "significantly" prolonged in the erlotinib/gemcitabine arm (6.24 months vs. 5.91 months, P = 0.038). The P = 0.038 means that there is only a 3.8% chance that this observed difference between the groups occurred by chance (which is less than the traditional cut-off of 5%) and therefore, statistically significant. In this example, the clinical relevance of this "positive" study is the "treatment effect" or difference in median survival between 6.24 and 5.91 months - a mere 10 days, which most oncologists would agree is a clinically irrelevant "improvement" in outcomes, especially when considering the added toxicity and costs involved with the combination.

Most journals now endorse the use of the CONSORT statement for reporting of parallel-group randomized trials, which emphasizes the need for reporting of the estimated effect size and its precision (such as 95% confidence interval) for each primary and secondary outcome. [5] Readers should bear in mind that interpretation of study results should take into account the clinical significance by looking at the actual treatment effect (with confidence intervals) and should not just be based on "P" values and statistical significance.

   References Top

Ranganathan P, Pramesh CS, Buyse M. Common pitfalls in statistical analysis: "P" values, statistical significance and confidence intervals. Perspect Clin Res 2015;6:116-7.  Back to cited text no. 1
[PUBMED]  Medknow Journal  
LeFort SM. The statistical versus clinical significance debate. Image J Nurs Sch 1993;25:57-62.  Back to cited text no. 2
Fethney J. Statistical and clinical significance, and how to use confidence intervals to help interpret both. Aust Crit Care 2010;23:93-7.  Back to cited text no. 3
Moore MJ, Goldstein D, Hamm J, Figer A, Hecht JR, Gallinger S, et al. Erlotinib plus gemcitabine compared with gemcitabine alone in patients with advanced pancreatic cancer: A phase III trial of the National Cancer Institute of Canada Clinical Trials Group. J Clin Oncol 2007;25:1960-6.  Back to cited text no. 4
Schulz KF, Altman DG, Moher D, CONSORT Group. CONSORT 2010 statement: Updated guidelines for reporting parallel group randomized trials. Ann Intern Med 2010;152:726-32.  Back to cited text no. 5

This article has been cited by
1 Factors determining nurses’ knowledge of evidence-based pressure ulcer prevention practices in Finland: a correlational cross-sectional study
Heidi Parisod,Arja Holopainen,Marita Koivunen,Pauli Puukka,Elina Haavisto
Scandinavian Journal of Caring Sciences. 2021;
[Pubmed] | [DOI]
2 Effect of acute lower respiratory tract infection on pulmonary artery pressure in children with post-tricuspid left-to-right shunt
Sakshi Sachdeva,Shyam S. Kothari,Saurabh K. Gupta,Sivasubramanian Ramakrishnan,Anita Saxena
Cardiology in the Young. 2021; : 1
[Pubmed] | [DOI]
3 p value variability and subgroup testing
Graham Horgan
European Journal of Nutrition. 2021;
[Pubmed] | [DOI]
4 Assessing responsiveness of the EQ-5D-3L, the Oxford Hip Score, and the Oxford Knee Score in the NHS patient-reported outcome measures
Sujin Kang
Journal of Orthopaedic Surgery and Research. 2021; 16(1)
[Pubmed] | [DOI]
5 The Effect of a High-Intensity PrO2Fit Inspiratory Muscle Training Intervention on Physiological and Psychological Health in Adults with Bronchiectasis: A Mixed-Methods Study
Jessica L. McCreery,Kelly A. Mackintosh,Rebekah Mills-Bennett,Melitta A. McNarry
International Journal of Environmental Research and Public Health. 2021; 18(6): 3051
[Pubmed] | [DOI]
6 The predictors of sepsis-related acute kidney injury
Wen-Ling Lee,Fa-Kung Lee,Peng-Hui Wang
Journal of the Chinese Medical Association. 2021; 84(3): 243
[Pubmed] | [DOI]
7 Effect of an E-Learning Module on Personal Protective Equipment Proficiency Among Prehospital Personnel: Web-Based Randomized Controlled Trial
Laurent Suppan,Mohamed Abbas,Loric Stuby,Philippe Cottet,Robert Larribau,Eric Golay,Anne Iten,Stephan Harbarth,Birgit Gartner,Mélanie Suppan
Journal of Medical Internet Research. 2020; 22(8): e21265
[Pubmed] | [DOI]
8 Anatomical consideration for optimal position of orthodontic miniscrews in the maxilla: a CBCT appraisal
Maha S. Al Amri,Hanadi M. Sabban,Doaa H. Alsaggaf,Fahad F. Alsulaimani,Ghassan A. Al-Turki,Mohammad S. Al-Zahrani,Khalid H. Zawawi
Annals of Saudi Medicine. 2020; 40(4): 330
[Pubmed] | [DOI]

Relation of Corneal Astigmatism with Various Corneal Image Quality Parameters in a Large Cohort of Naïve Corneas

Mohamed Omar Yousif,Rania Serag Elkitkat,Noha Abdelsadek Alaarag,Abdelrhman Shams,Hesham Mohamed Gharieb
Clinical Ophthalmology. 2020; Volume 14: 2203
[Pubmed] | [DOI]
10 Impact of macronutrient supplements on later growth of children born preterm or small for gestational age: A systematic review and meta-analysis of randomised and quasirandomised controlled trials
Luling Lin,Emma Amissah,Gregory D. Gamble,Caroline A. Crowther,Jane E. Harding,Lars Åke Persson
PLOS Medicine. 2020; 17(5): e1003122
[Pubmed] | [DOI]
11 The Relationships Between Tooth-Supported Fixed Dental Prostheses and Restorations and the Periodontium
Carlo Ercoli,Dennis Tarnow,Carlo E. Poggio,Alexandra Tsigarida,Marco Ferrari,Jack G. Caton,Konstantinos Chochlidakis
Journal of Prosthodontics. 2020;
[Pubmed] | [DOI]
12 Comparison of the Effects of Cognitive Behavioral Therapy and Behavioral Treatment on Obesity Treatment by Patient Subtypes: A Systematic Review and Meta-analysis
Jin-Young Cha,Seo-Young Kim,In-Soo Shin,Young-Bae Park,Young-Woo Lim
Journal of Korean Medicine for Obesity Research. 2020; 20(2): 178
[Pubmed] | [DOI]
13 Deprivation matters: understanding associations between neighbourhood deprivation, unhealthy food outlets, unhealthy dietary behaviours and child body size using structural equation modelling
Victoria Egli,Matthew Hobbs,Jordan Carlson,Niamh Donnellan,Lisa Mackay,Daniel Exeter,Karen Villanueva,Caryn Zinn,Melody Smith
Journal of Epidemiology and Community Health. 2020; 74(5): 460
[Pubmed] | [DOI]
14 Testing rimegepant for migraine—time to revise the trial design?
Sara Gasparini,Claudia Torino,Damiano Branca,Edoardo Ferlazzo,Umberto Aguglia
The Lancet. 2020; 395(10241): 1901
[Pubmed] | [DOI]
15 Artificial Intelligence in Cardiology
Dipti Itchhaporia
Trends in Cardiovascular Medicine. 2020;
[Pubmed] | [DOI]
16 Adequate statistical analyses and inferences of Randomized Clinical Trial
Andrea Batista de Sousa Canheta,Annelisa Silva e Alves de Carvalho Santos,Jacqueline Danesio de Souza,Erika Aparecida Silveira
Clinical Nutrition. 2020;
[Pubmed] | [DOI]
17 Comparison of Biochemical Test Results between Portable and Conventional Laboratory Analyzers in Chickens
Fernando Ruiz-Jimenez,Erika Gruber,Maria Correa,Rocio Crespo
Poultry Science. 2020;
[Pubmed] | [DOI]
18 Psychological Treatments in Adult ADHD: A Systematic Review
Tim Fullen,Sarah L Jones,Lisa Marie Emerson,Marios Adamou
Journal of Psychopathology and Behavioral Assessment. 2020;
[Pubmed] | [DOI]
19 Statistical data presentation: a primer for rheumatology researchers
Durga Prasanna Misra,Olena Zimba,Armen Yuri Gasparyan
Rheumatology International. 2020;
[Pubmed] | [DOI]
20 Considering Evidence From Studies With Findings Beyond the Sacred Cow of p  < .05 for Best Practice
Audra Hanners,Bernadette Melnyk,Jinhong Guo
Worldviews on Evidence-Based Nursing. 2020;
[Pubmed] | [DOI]
21 Global trends of antimicrobial susceptibility to ceftaroline and ceftazidime–avibactam: a surveillance study from the ATLAS program (2012–2016)
Hui Zhang,Yingchun Xu,Peiyao Jia,Ying Zhu,Ge Zhang,Jingjia Zhang,Simeng Duan,Wei Kang,Tong Wang,Ran Jing,Jingwei Cheng,Yali Liu,Qiwen Yang
Antimicrobial Resistance & Infection Control. 2020; 9(1)
[Pubmed] | [DOI]
22 Role of Homeopathy in Epidemics: Paving Way for a Plausible Solution in the Management of COVID-19/SARS-CoV-2 Infection
Abhijit Dutta
Homœopathic Links. 2020; 33(04): 297
[Pubmed] | [DOI]
23 Metered-dose inhalers versus nebulization for the delivery of albuterol for acute exacerbations of wheezing or asthma in children: A systematic review with meta-analysis
Laura Payares-Salamanca,Sandra Contreras-Arrieta,Victor Florez-García,Alexander Barrios-Sanjuanelo,Ivan Stand-Niño,Carlos E. Rodriguez-Martinez
Pediatric Pulmonology. 2020;
[Pubmed] | [DOI]
24 The effects of acculturation on neuropsychological test performance: A systematic literature review
Yi Wen Tan,Gerald H. Burgess,Robin J. Green
The Clinical Neuropsychologist. 2020; : 1
[Pubmed] | [DOI]
25 Does the implementation of a care pathway for patients with hip or knee osteoarthritis lead to fewer diagnostic imaging and referrals by general practitioners? A pre-post-implementation study of claims data
Esther H. A. van den Bogaart,Mariëlle E. A. L. Kroese,Marieke D. Spreeuwenberg,Ramon P. G. Ottenheijm,Patrick Deckers,Dirk Ruwaard
BMC Family Practice. 2019; 20(1)
[Pubmed] | [DOI]
26 Ultrasound and Clinical Preoperative Characteristics for Discrimination Between Ovarian Metastatic Colorectal Cancer and Primary Ovarian Cancer: A Case-Control Study
Maciej Stukan,Juan Luis Alcazar,Jacek Gebicki,Elizabeth Epstein,Marcin Liro,Alexandra Sufliarska,Sebastian Szubert,Stefano Guerriero,Elena Ioana Braicu,Mariusz Szajewski,Malgorzata Pietrzak-Stukan,Daniela Fischerova
Diagnostics. 2019; 9(4): 210
[Pubmed] | [DOI]
27 Effects of kinesio taping alone versus sham taping in individuals with musculoskeletal conditions after intervention for at least one week: a systematic review and meta-analysis
Robinson Ramírez-Vélez,Ignacio Hormazábal-Aguayo,Mikel Izquierdo,Katherine González-Ruíz,Jorge Enrique Correa-Bautista,Antonio García-Hermoso
Physiotherapy. 2019;
[Pubmed] | [DOI]
28 Evidence Based Medicine V; how to use in clinical practice
Magdalena Fossum,Kathrine Herbst,Martin Kaefer,Luke Harper,Marco Castagnetti,Goedele Beckers,Nicolas Kalfa,Darius Bagli
Journal of Pediatric Urology. 2019;
[Pubmed] | [DOI]
29 Regulatory-Accepted Drug Development Tools Are Needed to Accelerate Innovative CNS Disease Treatments
Stephen P. Arneric,Volker D. Kern,Diane T. Stephenson
Biochemical Pharmacology. 2018;
[Pubmed] | [DOI]
30 Dental caries and developmental defects of enamel in individuals with chronic kidney disease: Systematic review and meta-analysis
Francisco Ivison Rodrigues Limeira,Monica Yamauti,Allyson Nogueira Moreira,Tuélita Marques Galdino,Cláudia Silami de Magalhães,Lucas Guimarães Abreu
Oral Diseases. 2018;
[Pubmed] | [DOI]
31 Statistical analysis of patient-reported outcome data in randomised controlled trials of locally advanced and metastatic breast cancer: a systematic review
Madeline Pe,Lien Dorme,Corneel Coens,Ethan Basch,Melanie Calvert,Alicyn Campbell,Charles Cleeland,Kim Cocks,Laurence Collette,Linda Dirven,Amylou C Dueck,Nancy Devlin,Hans-Henning Flechtner,Carolyn Gotay,Ingolf Griebsch,Mogens Groenvold,Madeleine King,Michael Koller,Daniel C Malone,Francesca Martinelli,Sandra A Mitchell,Jammbe Z Musoro,Kathy Oliver,Elisabeth Piault-Louis,Martine Piccart,Francisco L Pimentel,Chantal Quinten,Jaap C Reijneveld,Jeff Sloan,Galina Velikova,Andrew Bottomley
The Lancet Oncology. 2018; 19(9): e459
[Pubmed] | [DOI]
32 Association of Systemic Inflammatory and Anti-inflammatory Responses with Adverse Outcomes in Acute Pancreatitis: Preliminary Results of an Ongoing Study
Deepesh Sharma,Aparna Jakkampudi,Ratnakar Reddy,Panyala Balakumar Reddy,Aasish Patil,H. V. V. Murthy,G. Venkat Rao,D. Nageshwar Reddy,Rupjyoti Talukdar
Digestive Diseases and Sciences. 2017;
[Pubmed] | [DOI]
33 Effect of dentin hypersensitivity treatment on oral health related quality of life — A systematic review and meta-analysis
Dhelfeson Willya Douglas-de-Oliveira,Glayson Pereira Vitor,Juliana Oliveira Silveira,Carolina Castro Martins,Fernando Oliveira Costa,Luís Otávio Miranda Cota
Journal of Dentistry. 2017;
[Pubmed] | [DOI]
34 Acute effects of Kinesio taping on pain, disability and back extensor muscle endurance in patients with low back pain caused by magnetic resonance imaging-confirmed lumbar disc degeneration
Nai-Jen Chang,Willy Chou,Pei-Chi Hsiao,Wen-Dien Chang,Yi-Ming Lo
Journal of Back and Musculoskeletal Rehabilitation. 2017; : 1
[Pubmed] | [DOI]
35 Curricular Interventions to Promote Self-care in Prelicensure Nursing Students
Pamela F. Ashcraft,Susan L. Gatto
Nurse Educator. 2017; : 1
[Pubmed] | [DOI]
36 Reporting the results of meta-analyses: A plea for incorporating clinical relevance referring to an example
Ronald H.M.A. Bartels,Roland D. Donk,Wim I.M. Verhagen,Allard J.F. Hosman,André L.M. Verbeek
The Spine Journal. 2017;
[Pubmed] | [DOI]
37 Are the effects of Kinesio Taping clinically meaningful in patients with acute low back pain?
Amanda Costa Araújo,Leonardo Oliveira Pena Costa
Clinical Rehabilitation. 2016; 30(11): 1136
[Pubmed] | [DOI]


    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
    Access Statistics
    Email Alert *
    Add to My List *
* Registration required (free)  

  In this article

 Article Access Statistics
    PDF Downloaded4040    
    Comments [Add]    
    Cited by others 37    

Recommend this journal