Home  |  About us  |  Editorial board  |  Ahead of print  | Current issue  |  Archives  |  Submit article  |  Instructions |  Search  |   Subscribe  |  Advertise  |  Contacts  |  Login 
  Users Online: 1317Home Print this page Email this page Small font sizeDefault font sizeIncrease font size  

 Table of Contents      
ORIGINAL ARTICLE
Year : 2016  |  Volume : 7  |  Issue : 3  |  Page : 128-131

Assessment of adherence to the statistical components of consolidated standards of reporting trials statement for quality of reports on randomized controlled trials from five pharmacology journals


1 Department of Pharmacology, Topiwala National Medical College and BYL Nair Charitable Hospital, Mumbai Central, Mumbai, Maharashtra, India
2 Department of Pharmacology, Seth G. S. Medical College and KEM Hospital, Parel, Mumbai, Maharashtra, India

Date of Web Publication28-Jun-2016

Correspondence Address:
Sachin Satpute
Department of Pharmacology, Topiwala National Medical College and BYL Nair Charitable Hospital, Dr. A L Nair Road, Mumbai Central, Mumbai - 400 008, Maharashtra
India
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/2229-3485.184816

Rights and Permissions
   Abstract 

Background: The Consolidated Standards of Reporting Trials (CONSORT) statement is a device to standardize reporting and improve the quality of controlled trials. However, little attention is paid to the statistical components in the CONSORT checklist. The present study evaluates the randomized controlled trials [RCTs] published in five high impact pharmacology journals with respect to its statistical methods. Methods: Randomized Controlled Trials [RCTs] published in the years 2013 & 2014 in five pharmacology journals with high impact factor, The Journal of Clinical Pharmacology (JCP), British Journal of Clinical Pharmacology (BJCP), European Journal of Clinical Pharmacology (EJCP), Journal of Pharmacology & Pharmacotherapeutics (JPP) and Indian Journal of Pharmacology (IJP) were assessed for adherence to the statistical components of CONSORT statement. Results: Of the 174 RCTs analysed, 103 described the method of sample size calculation. Of the five journals, maximum reports in JCP (34/50) and minimum in IJP (13/31) adhered to the CONSORT checklist [item 7a-sample size calculation]. Most reports mentioned the statistical methods used for analysis of data. (171/174) as per the checklist [item 12=statistical methods used]. Analysis of variance (ANOVA) was the most commonly used test (88/174). The software used for statistical analysis was mentioned in 111 RCTs and SPSS was used more frequently (58/111). The exact p value was stated in 108 reports. Certain errors in statistical analysis were also noted (40/174). Conclusion: These findings show inconsistencies and non- adherence to the statistical components of the CONSORT statement especially with respect to sample size calculation. Special attention must be paid to the statistical accuracy of the reports.

Keywords: Consolidated standards of reporting trials, randomized controlled trials, sample size, statistical component


How to cite this article:
Satpute S, Mehta M, Bhete S, Kurle D. Assessment of adherence to the statistical components of consolidated standards of reporting trials statement for quality of reports on randomized controlled trials from five pharmacology journals. Perspect Clin Res 2016;7:128-31

How to cite this URL:
Satpute S, Mehta M, Bhete S, Kurle D. Assessment of adherence to the statistical components of consolidated standards of reporting trials statement for quality of reports on randomized controlled trials from five pharmacology journals. Perspect Clin Res [serial online] 2016 [cited 2019 Sep 19];7:128-31. Available from: http://www.picronline.org/text.asp?2016/7/3/128/184816


   Introduction Top


Reports of randomized controlled trials (RCTs) are the "gold standard" by which decisions about treatment efficacy and effectiveness are made by health professionals and others. [1] Since RCTs, more than any other methodology, can have a powerful and immediate impact on patient care, the published report should be of the highest possible standards. [2] Ideally, accurate and complete reporting concerning the design, conduct, analysis, and generalizability of the trial should be conveyed. [3] Adequate reporting is essential because the readers need and deserve to know the quality of methods to make informed judgments regarding the validity of a clinical trial. [1],[2],[3],[4] Evidence indicates that reports of low-quality RCTs, compared with reports of higher-quality ones, overestimate the effectiveness of interventions by about 30% across a variety of health care conditions. [5],[6],[7] The scientific world worries, of course, that sloppy reporting reflects sloppy methods, and that with sloppy methods come biased results. [8]

A flurry of activity to address the defect and standardize reporting of RCTs culminated in the most prominent guideline, the consolidated standards of reporting trials (CONSORT) in 1996 which has been revised twice, in 2001 and 2010. [6],[7],[8] The objective of the CONSORT is to provide a guideline for authors to standardize and improve the reporting of trials. [9] Preliminary appraisals suggest that the use of CONSORT items is associated with improvements in the quality of reports published. [10] However, little attention is paid to the statistical components in the CONSORT checklist. Adherence to the CONSORT requires continual appraisal for improving the precision and transparency of published trials. The aim of this study was to assess the adherence to the statistical components of CONSORT 2010 statement for quality of reports on RCTs from five pharmacology journals.


   Methods Top


Data sources

We selected five pharmacology journals in descending order of impact factor as per the Thomas Reuters impact factor list as follows: The Journal of Clinical Pharmacology (JCP), British Journal of Clinical Pharmacology (BJCP), European Journal of Clinical Pharmacology (EJCP), Journal of Pharmacology and Pharmacotherapeutics (JPP), and Indian Journal of Pharmacology (IJP).

We conducted a MEDLINE/PubMed search to identify all RCTs published between January 2013 and December 2014 with the following search strategy: "Journal of Clinical Pharmacology" (Jour) OR "British Journal of Clinical Pharmacology" (Jour) OR "European Journal of Clinical Pharmacology" (Jour) OR "Journal of Pharmacology and Pharmacotherapeutics" (Jour) OR "Indian Journal of Pharmacology" (Jour) AND RCT (ptyp) AND MEDLINE (sb) AND ("2013/01/01 [PDAT]": "2014/12/31[PDAT]").

Study selection

RCTs of preventive and therapeutic interventions were selected. We included reports in which the allocation of participants to interventions was described as random, randomly allocated, randomized or randomization. Other study designs such as observational studies, economic analyses on RCTs, quasi-randomized trials, cluster randomized trials, diagnostic or screening tests, follow-up studies of previously reported RCTs, editorials, reviews, case reports, and letters were excluded.

Data extraction

The reviewers underwent training in evaluating RCTs using the CONSORT 2010 statement with special reference to its statistical components. One reviewer extracted data from all included papers. A second reviewer independently checked a random sample of 30% of the papers. Discrepancies were resolved where possible by discussion, and the sample results were compared with the full results using Kappa scores.

The following items were included in the checklist: Name, issue and volume of the journal, method of sample size calculation (item 7a of CONSORT checklist), alpha and beta error in calculation, percentage of drop-outs considered for sample size determination, statistical methods used to compare primary and secondary outcomes (item 12a of CONSORT checklist), statistical software used for analysis, and use of exact P value.

Data analysis

Data for descriptive statistics were described as frequencies and percentages. The data were analyzed using Microsoft Excel version 2013.


   Results Top


Among the 174 RCTs included for the study, 28.7% (50/174) were published in JCP followed by 23% (40/174) from BJCP, 19.5% (34/174) from EJCP, 17.8% (31/174) from IJP, and 10.9% (19/174) from JPP.

Of the 174 reports in the study, 103 (59.2%) adhered to item 7a of the CONSORT checklist and reported the method of sample size determination. A journal-wise distribution is given in [Table 1].
Table 1: Journal-wise distribution of reports mentioning method of sample size determination


Click here to view


The method used for sample size determination was most commonly based on the primary outcome of the study in question 72.8% (75/103) followed by previous studies 21.4% (22/103) and very few reports 0.05% (6/103) used feasibility as a method. A detailed distribution is given in [Table 2].
Table 2: Methods of sample size determination used in individual journals


Click here to view


The assumption of alpha and beta error was mentioned in 40% (70/174) reports. The alpha error was assumed at 5% in all the reports. Out of these 70 reports, the beta error was assumed as 80% in 48 of 70 reports and 90% in 22 of 70 reports.

28.7% (50/174) reports mentioned the drop-out rate for calculation of sample size. Out of these 50 reports, the drop-out rate was assumed to be 25% in 23 reports and 20% in 27 reports.

Most reports 98.3% (171/174) mentioned the test used for statistical analysis. Two articles in IJP and one article in BJCP did not mention the test used for statistical analysis in the methodology section in spite of using a test for statistical analysis.

The most commonly used tests for statistical analysis have been outlined in [Table 3].
Table 3: Tests used for statistical analysis


Click here to view


Analysis of variance

The software used for statistical analysis was mentioned in 63.8% (111/174) reports. Out of these 111 reports, Statistical Package for the Social Sciences (SPSS) was used most frequently 52.3% (58/111) followed by same-area sales 27.9% (31/111) then GraphPad 17.1% (19/111) and others 2.7% (3/111).

The actual P value was reported in 62.0% (108/174) reports.


   Discussion Top


In this study, we have observed that the majority of RCTs reported in high impact factor pharmacology journals did not adhere completely to the statistical components of CONSORT guidelines. A study by Ayatolahi et al.[11],[12] reported that 6.2% reports mentioned the method of sample size calculation. In a study done by Nojomi et al. [7] only 30% of evaluated articles reported the method of sample size calculation. The method of sample size determination (item 7a of CONSORT checklist) was mentioned in 59.2% reports in our study. This indicated an increase in adherence to the CONSORT checklist of RCTs in our study compared to the previous studies. To the best of our knowledge, there are no studies evaluating only the statistical components of the CONSORT statement. The method of sample size calculation was most commonly based on the primary outcome of the study followed by previous studies and feasibility. Most RCTs should mention the drop-out rate for the calculation of sample size. However, in our study we found that only 28.7% articles reported the drop-out rate.

The reporting of statistical methods used to compare groups for primary and secondary outcomes was 98.2%. Similar results (96.3%) were obtained in a study done by Nojomi et al.[7] The most common test used for statistical analysis was analysis of variance followed by Mann-Whitney U-test and then Student's t-test and others. None of the studies have reported the most commonly used software for statistical analysis. In our study, SPSS Software was used most frequently.

We have assessed RCTs published only over the last 1-year from five pharmacology journals. We reviewed RCTs published in high impact factor pharmacology journals that have universal acceptance in the pharmacology research community. To the best of our knowledge, no studies have specifically evaluated the adherence to the statistical components of the CONSORT statement.


   Conclusions Top


Thus, the finding from this study shows that the quality of reports on RCTs from five pharmacology journals did not adhere completely to the statistical components of the CONSORT 2010 statement. CONSORT is an evolving guideline not considered to be an absolute standard. [13] However, improvements in the quality of RCT reports can be expected by adherence to the existing standards and guidelines.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

 
   References Top

1.
Moher D, Jones A, Lepage L; CONSORT Group (Consolidated Standards for Reporting of Trials). Use of the CONSORT statement and quality of reports of randomized trials: A comparative before-and-after evaluation. JAMA 2001;285:1992-5.  Back to cited text no. 1
[PUBMED]    
2.
Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA 1996;276:637-9.  Back to cited text no. 2
    
3.
Ghimire S, Kyung E, Kang W, Kim E. Assessment of adherence to the CONSORT statement for quality of reports on randomized controlled trial abstracts from four high-impact general medical journals. Trials 2012;13:77.  Back to cited text no. 3
    
4.
Sánchez-Thorin JC, Cortés MC, Montenegro M, Villate N. The quality of reporting of randomized clinical trials published in ophthalmology. Ophthalmology 2001;108:410-5.  Back to cited text no. 4
    
5.
Schulz KF, Chalmers I, Grimes DA, Altman DG. Assessing the quality of randomization from reports of controlled trials published in obstetrics and gynecology journals. JAMA 1994;272:125-8.  Back to cited text no. 5
    
6.
Moher D, Schulz KF, Altman DG. The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet 2001;357:1191-4.  Back to cited text no. 6
    
7.
Nojomi M, Ramezani M, Ghafari-Anvar A. Quality of reports on randomized controlled trials published in Iranian journals: Application of the new version of consolidated standards of reporting trials (CONSORT). Arch Iran Med 2013;16:20-2.  Back to cited text no. 7
    
8.
Schulz KF. Randomised trials, human nature, and reporting guidelines. Lancet 1996;348:596-8.  Back to cited text no. 8
    
9.
Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: Updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol 2010;63:e1-37.  Back to cited text no. 9
    
10.
Plint AC, Moher D, Morrison A, Schulz K, Altman DG, Hill C, et al. Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review. Med J Aust 2006;185:263-7.  Back to cited text no. 10
    
11.
Uetani K, Nakayama T, Ikai H, Yonemoto N, Moher D. Quality of reports on randomized controlled trials conducted in Japan: Evaluation of adherence to the CONSORT statement. Intern Med 2009;48:307-13.  Back to cited text no. 11
    
12.
Ayatolahi SM, Jafari P, Ghaem H. An evaluation of the quality of published clinical trials in Iranian medical journals during 2001-2004. J Babol Univ Med Sci 2005;7:64-70.  Back to cited text no. 12
    
13.
Schulz KF, Altman DG, Moher D; CONSORT Group. CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials. Trials 2010;11:32.  Back to cited text no. 13
    



 
 
    Tables

  [Table 1], [Table 2], [Table 3]


This article has been cited by
1 Use of publication procedures to improve research integrity by addiction journals
Dennis M. Gorman
Addiction. 2019;
[Pubmed] | [DOI]
2 A Systems Approach to Understanding and Improving Research Integrity
Dennis M. Gorman,Amber D. Elkins,Mark Lawley
Science and Engineering Ethics. 2017;
[Pubmed] | [DOI]
3 Bewertung der methodischen Qualität publizierter RCTs in der Kataraktchirurgie
C. Baulig,F. Krummenauer,S. Knippschild
Der Ophthalmologe. 2017;
[Pubmed] | [DOI]



 

Top
  
 
  Search
 
    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
    Access Statistics
    Email Alert *
    Add to My List *
* Registration required (free)  

 
  In this article
    Abstract
   Introduction
   Methods
   Results
   Discussion
   Conclusions
    References
    Article Tables

 Article Access Statistics
    Viewed1647    
    Printed27    
    Emailed0    
    PDF Downloaded197    
    Comments [Add]    
    Cited by others 3    

Recommend this journal