Home  |  About us  |  Editorial board  |  Current issue  |  Archives  |  Submit article  |  Instructions |  Search  |   Subscribe  |  Advertise  |  Contacts  |  Login 
  Users Online: 195Home Print this page Email this page Small font sizeDefault font sizeIncrease font size  

 Table of Contents      
STATISTICS
Year : 2015  |  Volume : 6  |  Issue : 3  |  Page : 169-170

Common pitfalls in statistical analysis: Clinical versus statistical significance


1 Department of Anaesthesiology, Division of Thoracic Surgery, Tata Memorial Centre, Parel, Mumbai, Maharashtra, India
2 Department of Surgical Oncology, Division of Thoracic Surgery, Tata Memorial Centre, Parel, Mumbai, Maharashtra, India
3 International Drug Development Institute, Hasselt University, Belgium

Date of Web Publication6-Jul-2015

Correspondence Address:
Priya Ranganathan
Department of Anaesthesiology, Tata Memorial Centre, Ernest Borges Road, Parel, Mumbai - 400 012, Maharashtra
India
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/2229-3485.159943

Rights and Permissions
   Abstract 

In clinical research, study results, which are statistically significant are often interpreted as being clinically important. While statistical significance indicates the reliability of the study results, clinical significance reflects its impact on clinical practice. The third article in this series exploring pitfalls in statistical analysis clarifies the importance of differentiating between statistical significance and clinical significance.

Keywords: Biostatistics, confidence intervals, data interpretation, statistical


How to cite this article:
Ranganathan P, Pramesh C S, Buyse M. Common pitfalls in statistical analysis: Clinical versus statistical significance. Perspect Clin Res 2015;6:169-70

How to cite this URL:
Ranganathan P, Pramesh C S, Buyse M. Common pitfalls in statistical analysis: Clinical versus statistical significance. Perspect Clin Res [serial online] 2015 [cited 2018 Jan 22];6:169-70. Available from: http://www.picronline.org/text.asp?2015/6/3/169/159943

One of the common problems faced by readers (and authors!) of medical articles is in the interpretation of the word "significance." The term "statistical significance" is often misinterpreted as a "clinically important" result. The confusion stems from the fact that many people equate "significance" with its literal meaning of "importance," whereas in statistics, it has a far more restrictive connotation. This article explains the idea of the statistical significance and differentiates it from clinical relevance or importance, which is an entirely different concept. In the previous article, in this series, we looked at different ways of expressing statistical significance ("P0" values versus confidence intervals). [1] Measures of statistical significance quantify the probability of a study's results being due to chance. Clinical significance, on the other hand, refers to the magnitude of the actual treatment effect (i.e., the difference between the intervention and control groups, also known as the "treatment effect size"), which will determine whether the results of the trial are likely to impact current medical practice. The "P" value, frequently used to measure statistical significance, is the probability that the study results are due to chance rather than to a real treatment effect. The conventional cut off for the "P0" value to be considered statistically significant is of 0.05 (or 5%). What a P < 0.05 implies is that the possibility of the results in a study being due to chance is <5%.

In clinical practice, the "clinical significance" of a result is dependent on its implications on existing practice-treatment effect size being one of the most important factors that drives treatment decisions. LeFort suggests that the clinical significance should reflect "the extent of change, whether the change makes a real difference to subject lives, how long the effects last, consumer acceptability, cost-effectiveness, and ease of implementation". [2] While there are established, traditionally accepted values for statistical significance testing, this is lacking for evaluating clinical significance. [3] More often than not, it is the judgment of the clinician (and the patient) which decides whether a result is clinically significant or not.

Statistical significance is heavily dependent on the study's sample size; with large sample sizes, even small treatment effects (which are clinically inconsequential) can appear statistically significant; therefore, the reader has to interpret carefully whether this "significance" is clinically meaningful. A study published in the Journal of Clinical Oncology compared overall survival in 569 patients with advanced pancreatic cancer who were randomised to receive erlotinib plus gemcitabine versus gemcitabine alone. [4] Median survival was found to be "significantly" prolonged in the erlotinib/gemcitabine arm (6.24 months vs. 5.91 months, P = 0.038). The P = 0.038 means that there is only a 3.8% chance that this observed difference between the groups occurred by chance (which is less than the traditional cut-off of 5%) and therefore, statistically significant. In this example, the clinical relevance of this "positive" study is the "treatment effect" or difference in median survival between 6.24 and 5.91 months - a mere 10 days, which most oncologists would agree is a clinically irrelevant "improvement" in outcomes, especially when considering the added toxicity and costs involved with the combination.

Most journals now endorse the use of the CONSORT statement for reporting of parallel-group randomized trials, which emphasizes the need for reporting of the estimated effect size and its precision (such as 95% confidence interval) for each primary and secondary outcome. [5] Readers should bear in mind that interpretation of study results should take into account the clinical significance by looking at the actual treatment effect (with confidence intervals) and should not just be based on "P" values and statistical significance.

 
   References Top

1.
Ranganathan P, Pramesh CS, Buyse M. Common pitfalls in statistical analysis: "P" values, statistical significance and confidence intervals. Perspect Clin Res 2015;6:116-7.  Back to cited text no. 1
[PUBMED]  Medknow Journal  
2.
LeFort SM. The statistical versus clinical significance debate. Image J Nurs Sch 1993;25:57-62.  Back to cited text no. 2
    
3.
Fethney J. Statistical and clinical significance, and how to use confidence intervals to help interpret both. Aust Crit Care 2010;23:93-7.  Back to cited text no. 3
    
4.
Moore MJ, Goldstein D, Hamm J, Figer A, Hecht JR, Gallinger S, et al. Erlotinib plus gemcitabine compared with gemcitabine alone in patients with advanced pancreatic cancer: A phase III trial of the National Cancer Institute of Canada Clinical Trials Group. J Clin Oncol 2007;25:1960-6.  Back to cited text no. 4
    
5.
Schulz KF, Altman DG, Moher D, CONSORT Group. CONSORT 2010 statement: Updated guidelines for reporting parallel group randomized trials. Ann Intern Med 2010;152:726-32.  Back to cited text no. 5
[PUBMED]    



This article has been cited by
1 Association of Systemic Inflammatory and Anti-inflammatory Responses with Adverse Outcomes in Acute Pancreatitis: Preliminary Results of an Ongoing Study
Deepesh Sharma,Aparna Jakkampudi,Ratnakar Reddy,Panyala Balakumar Reddy,Aasish Patil,H. V. V. Murthy,G. Venkat Rao,D. Nageshwar Reddy,Rupjyoti Talukdar
Digestive Diseases and Sciences. 2017;
[Pubmed] | [DOI]
2 Effect of dentin hypersensitivity treatment on oral health related quality of life — A systematic review and meta-analysis
Dhelfeson Willya Douglas-de-Oliveira,Glayson Pereira Vitor,Juliana Oliveira Silveira,Carolina Castro Martins,Fernando Oliveira Costa,Luís Otávio Miranda Cota
Journal of Dentistry. 2017;
[Pubmed] | [DOI]
3 Reporting the results of meta-analyses: A plea for incorporating clinical relevance referring to an example
Ronald H.M.A. Bartels,Roland D. Donk,Wim I.M. Verhagen,Allard J.F. Hosman,André L.M. Verbeek
The Spine Journal. 2017;
[Pubmed] | [DOI]
4 Acute effects of Kinesio taping on pain, disability and back extensor muscle endurance in patients with low back pain caused by magnetic resonance imaging-confirmed lumbar disc degeneration
Nai-Jen Chang,Willy Chou,Pei-Chi Hsiao,Wen-Dien Chang,Yi-Ming Lo
Journal of Back and Musculoskeletal Rehabilitation. 2017; : 1
[Pubmed] | [DOI]
5 Are the effects of Kinesio Taping clinically meaningful in patients with acute low back pain?
Amanda Costa Araújo,Leonardo Oliveira Pena Costa
Clinical Rehabilitation. 2016; 30(11): 1136
[Pubmed] | [DOI]



 

Top
  
 
  Search
 
    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
    Access Statistics
    Email Alert *
    Add to My List *
* Registration required (free)  

 
  In this article
    Abstract
    References

 Article Access Statistics
    Viewed3980    
    Printed68    
    Emailed0    
    PDF Downloaded862    
    Comments [Add]    
    Cited by others 5    

Recommend this journal