|
 |
| ORIGINAL ARTICLE |
|
| Year : 2016 | Volume
: 7
| Issue : 3 | Page : 128-131 |
|
Assessment of adherence to the statistical components of consolidated standards of reporting trials statement for quality of reports on randomized controlled trials from five pharmacology journals
Sachin Satpute1, Manthan Mehta1, Sandeep Bhete1, Dnyneshwar Kurle2
1 Department of Pharmacology, Topiwala National Medical College and BYL Nair Charitable Hospital, Mumbai Central, Mumbai, Maharashtra, India
2 Department of Pharmacology, Seth G. S. Medical College and KEM Hospital, Parel, Mumbai, Maharashtra, India
| Date of Web Publication | 28-Jun-2016 |
Correspondence Address:
Sachin Satpute
Department of Pharmacology, Topiwala National Medical College and BYL Nair Charitable Hospital, Dr. A L Nair Road, Mumbai Central, Mumbai - 400 008, Maharashtra
India
 Source of Support: None, Conflict of Interest: None  | Check |
DOI: 10.4103/2229-3485.184816
 Abstract | | |
Background: The Consolidated Standards of Reporting Trials (CONSORT) statement is a device to standardize reporting and improve the quality of controlled trials. However, little attention is paid to the statistical components in the CONSORT checklist. The present study evaluates the randomized controlled trials [RCTs] published in five high impact pharmacology journals with respect to its statistical methods. Methods: Randomized Controlled Trials [RCTs] published in the years 2013 & 2014 in five pharmacology journals with high impact factor, The Journal of Clinical Pharmacology (JCP), British Journal of Clinical Pharmacology (BJCP), European Journal of Clinical Pharmacology (EJCP), Journal of Pharmacology & Pharmacotherapeutics (JPP) and Indian Journal of Pharmacology (IJP) were assessed for adherence to the statistical components of CONSORT statement. Results: Of the 174 RCTs analysed, 103 described the method of sample size calculation. Of the five journals, maximum reports in JCP (34/50) and minimum in IJP (13/31) adhered to the CONSORT checklist [item 7a-sample size calculation]. Most reports mentioned the statistical methods used for analysis of data. (171/174) as per the checklist [item 12=statistical methods used]. Analysis of variance (ANOVA) was the most commonly used test (88/174). The software used for statistical analysis was mentioned in 111 RCTs and SPSS was used more frequently (58/111). The exact p value was stated in 108 reports. Certain errors in statistical analysis were also noted (40/174). Conclusion: These findings show inconsistencies and non- adherence to the statistical components of the CONSORT statement especially with respect to sample size calculation. Special attention must be paid to the statistical accuracy of the reports.
Keywords: Consolidated standards of reporting trials, randomized controlled trials, sample size, statistical component
How to cite this article:
Satpute S, Mehta M, Bhete S, Kurle D. Assessment of adherence to the statistical components of consolidated standards of reporting trials statement for quality of reports on randomized controlled trials from five pharmacology journals. Perspect Clin Res 2016;7:128-31 |
How to cite this URL:
Satpute S, Mehta M, Bhete S, Kurle D. Assessment of adherence to the statistical components of consolidated standards of reporting trials statement for quality of reports on randomized controlled trials from five pharmacology journals. Perspect Clin Res [serial online] 2016 [cited 2021 Jul 27];7:128-31. Available from: https://www.picronline.org/text.asp?2016/7/3/128/184816 |
| Introduction | |  |
Reports of randomized controlled trials (RCTs) are the "gold standard" by which decisions about treatment efficacy and effectiveness are made by health professionals and others. [1] Since RCTs, more than any other methodology, can have a powerful and immediate impact on patient care, the published report should be of the highest possible standards. [2] Ideally, accurate and complete reporting concerning the design, conduct, analysis, and generalizability of the trial should be conveyed. [3] Adequate reporting is essential because the readers need and deserve to know the quality of methods to make informed judgments regarding the validity of a clinical trial. [1],[2],[3],[4] Evidence indicates that reports of low-quality RCTs, compared with reports of higher-quality ones, overestimate the effectiveness of interventions by about 30% across a variety of health care conditions. [5],[6],[7] The scientific world worries, of course, that sloppy reporting reflects sloppy methods, and that with sloppy methods come biased results. [8]
A flurry of activity to address the defect and standardize reporting of RCTs culminated in the most prominent guideline, the consolidated standards of reporting trials (CONSORT) in 1996 which has been revised twice, in 2001 and 2010. [6],[7],[8] The objective of the CONSORT is to provide a guideline for authors to standardize and improve the reporting of trials. [9] Preliminary appraisals suggest that the use of CONSORT items is associated with improvements in the quality of reports published. [10] However, little attention is paid to the statistical components in the CONSORT checklist. Adherence to the CONSORT requires continual appraisal for improving the precision and transparency of published trials. The aim of this study was to assess the adherence to the statistical components of CONSORT 2010 statement for quality of reports on RCTs from five pharmacology journals.
| Methods | | |
Data sources
We selected five pharmacology journals in descending order of impact factor as per the Thomas Reuters impact factor list as follows: The Journal of Clinical Pharmacology (JCP), British Journal of Clinical Pharmacology (BJCP), European Journal of Clinical Pharmacology (EJCP), Journal of Pharmacology and Pharmacotherapeutics (JPP), and Indian Journal of Pharmacology (IJP).
We conducted a MEDLINE/PubMed search to identify all RCTs published between January 2013 and December 2014 with the following search strategy: "Journal of Clinical Pharmacology" (Jour) OR "British Journal of Clinical Pharmacology" (Jour) OR "European Journal of Clinical Pharmacology" (Jour) OR "Journal of Pharmacology and Pharmacotherapeutics" (Jour) OR "Indian Journal of Pharmacology" (Jour) AND RCT (ptyp) AND MEDLINE (sb) AND ("2013/01/01 [PDAT]": "2014/12/31[PDAT]").
Study selection
RCTs of preventive and therapeutic interventions were selected. We included reports in which the allocation of participants to interventions was described as random, randomly allocated, randomized or randomization. Other study designs such as observational studies, economic analyses on RCTs, quasi-randomized trials, cluster randomized trials, diagnostic or screening tests, follow-up studies of previously reported RCTs, editorials, reviews, case reports, and letters were excluded.
Data extraction
The reviewers underwent training in evaluating RCTs using the CONSORT 2010 statement with special reference to its statistical components. One reviewer extracted data from all included papers. A second reviewer independently checked a random sample of 30% of the papers. Discrepancies were resolved where possible by discussion, and the sample results were compared with the full results using Kappa scores.
The following items were included in the checklist: Name, issue and volume of the journal, method of sample size calculation (item 7a of CONSORT checklist), alpha and beta error in calculation, percentage of drop-outs considered for sample size determination, statistical methods used to compare primary and secondary outcomes (item 12a of CONSORT checklist), statistical software used for analysis, and use of exact P value.
Data analysis
Data for descriptive statistics were described as frequencies and percentages. The data were analyzed using Microsoft Excel version 2013.
| Results | | |
Among the 174 RCTs included for the study, 28.7% (50/174) were published in JCP followed by 23% (40/174) from BJCP, 19.5% (34/174) from EJCP, 17.8% (31/174) from IJP, and 10.9% (19/174) from JPP.
Of the 174 reports in the study, 103 (59.2%) adhered to item 7a of the CONSORT checklist and reported the method of sample size determination. A journal-wise distribution is given in [Table 1]. | Table 1: Journal-wise distribution of reports mentioning method of sample size determination
Click here to view |
The method used for sample size determination was most commonly based on the primary outcome of the study in question 72.8% (75/103) followed by previous studies 21.4% (22/103) and very few reports 0.05% (6/103) used feasibility as a method. A detailed distribution is given in [Table 2]. | Table 2: Methods of sample size determination used in individual journals
Click here to view |
The assumption of alpha and beta error was mentioned in 40% (70/174) reports. The alpha error was assumed at 5% in all the reports. Out of these 70 reports, the beta error was assumed as 80% in 48 of 70 reports and 90% in 22 of 70 reports.
28.7% (50/174) reports mentioned the drop-out rate for calculation of sample size. Out of these 50 reports, the drop-out rate was assumed to be 25% in 23 reports and 20% in 27 reports.
Most reports 98.3% (171/174) mentioned the test used for statistical analysis. Two articles in IJP and one article in BJCP did not mention the test used for statistical analysis in the methodology section in spite of using a test for statistical analysis.
The most commonly used tests for statistical analysis have been outlined in [Table 3].
Analysis of variance
The software used for statistical analysis was mentioned in 63.8% (111/174) reports. Out of these 111 reports, Statistical Package for the Social Sciences (SPSS) was used most frequently 52.3% (58/111) followed by same-area sales 27.9% (31/111) then GraphPad 17.1% (19/111) and others 2.7% (3/111).
The actual P value was reported in 62.0% (108/174) reports.
| Discussion | | |
In this study, we have observed that the majority of RCTs reported in high impact factor pharmacology journals did not adhere completely to the statistical components of CONSORT guidelines. A study by Ayatolahi et al.[11],[12] reported that 6.2% reports mentioned the method of sample size calculation. In a study done by Nojomi et al. [7] only 30% of evaluated articles reported the method of sample size calculation. The method of sample size determination (item 7a of CONSORT checklist) was mentioned in 59.2% reports in our study. This indicated an increase in adherence to the CONSORT checklist of RCTs in our study compared to the previous studies. To the best of our knowledge, there are no studies evaluating only the statistical components of the CONSORT statement. The method of sample size calculation was most commonly based on the primary outcome of the study followed by previous studies and feasibility. Most RCTs should mention the drop-out rate for the calculation of sample size. However, in our study we found that only 28.7% articles reported the drop-out rate.
The reporting of statistical methods used to compare groups for primary and secondary outcomes was 98.2%. Similar results (96.3%) were obtained in a study done by Nojomi et al.[7] The most common test used for statistical analysis was analysis of variance followed by Mann-Whitney U-test and then Student's t-test and others. None of the studies have reported the most commonly used software for statistical analysis. In our study, SPSS Software was used most frequently.
We have assessed RCTs published only over the last 1-year from five pharmacology journals. We reviewed RCTs published in high impact factor pharmacology journals that have universal acceptance in the pharmacology research community. To the best of our knowledge, no studies have specifically evaluated the adherence to the statistical components of the CONSORT statement.
| Conclusions | | |
Thus, the finding from this study shows that the quality of reports on RCTs from five pharmacology journals did not adhere completely to the statistical components of the CONSORT 2010 statement. CONSORT is an evolving guideline not considered to be an absolute standard. [13] However, improvements in the quality of RCT reports can be expected by adherence to the existing standards and guidelines.
Financial support and sponsorship
Nil.
Conflicts of interest
There are no conflicts of interest.
| References | | |
| 1. | Moher D, Jones A, Lepage L; CONSORT Group (Consolidated Standards for Reporting of Trials). Use of the CONSORT statement and quality of reports of randomized trials: A comparative before-and-after evaluation. JAMA 2001;285:1992-5.  [ PUBMED] |
| 2. | Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA 1996;276:637-9. |
| 3. | Ghimire S, Kyung E, Kang W, Kim E. Assessment of adherence to the CONSORT statement for quality of reports on randomized controlled trial abstracts from four high-impact general medical journals. Trials 2012;13:77. |
| 4. | Sánchez-Thorin JC, Cortés MC, Montenegro M, Villate N. The quality of reporting of randomized clinical trials published in ophthalmology. Ophthalmology 2001;108:410-5. |
| 5. | Schulz KF, Chalmers I, Grimes DA, Altman DG. Assessing the quality of randomization from reports of controlled trials published in obstetrics and gynecology journals. JAMA 1994;272:125-8. |
| 6. | Moher D, Schulz KF, Altman DG. The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet 2001;357:1191-4. |
| 7. | Nojomi M, Ramezani M, Ghafari-Anvar A. Quality of reports on randomized controlled trials published in Iranian journals: Application of the new version of consolidated standards of reporting trials (CONSORT). Arch Iran Med 2013;16:20-2. |
| 8. | Schulz KF. Randomised trials, human nature, and reporting guidelines. Lancet 1996;348:596-8. |
| 9. | Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: Updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol 2010;63:e1-37. |
| 10. | Plint AC, Moher D, Morrison A, Schulz K, Altman DG, Hill C, et al. Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review. Med J Aust 2006;185:263-7. |
| 11. | Uetani K, Nakayama T, Ikai H, Yonemoto N, Moher D. Quality of reports on randomized controlled trials conducted in Japan: Evaluation of adherence to the CONSORT statement. Intern Med 2009;48:307-13. |
| 12. | Ayatolahi SM, Jafari P, Ghaem H. An evaluation of the quality of published clinical trials in Iranian medical journals during 2001-2004. J Babol Univ Med Sci 2005;7:64-70. |
| 13. | Schulz KF, Altman DG, Moher D; CONSORT Group. CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials. Trials 2010;11:32. |
[Table 1], [Table 2], [Table 3]
| This article has been cited by | | 1 |
Reporting of statistical sample size calculations in publications of trials on age-related macular degeneration, glaucoma and cataract |
|
| Sabrina Tulka,Stephanie Knippschild,Sina Funck,Isabelle Goetjes,Yasmin Uluk,Christine Baulig,Gianni Virgili | | PLOS ONE. 2021; 16(6): e0252640 | | [Pubmed] | [DOI] | | | 2 |
Randomized controlled trials in non-pharmacological rehabilitation research: a scoping review of the reporting of sample size calculation, randomization procedure, and statistical analyses |
|
| Mohammadreza AMIRI,Dinesh KUMBHARE | | European Journal of Physical and Rehabilitation Medicine. 2021; 56(6) | | [Pubmed] | [DOI] | | | 3 |
Evaluation of reporting of CONSORT flow diagrams in randomized controlled trials in a national and international pharmacology journal |
|
| AsifMoeed Shaikh,ManthanM Mehta,VihangV Chawan | | Perspectives in Clinical Research. 2019; 10(4): 168 | | [Pubmed] | [DOI] | | | 4 |
Use of publication procedures to improve research integrity by addiction journals |
|
| Dennis M. Gorman | | Addiction. 2019; | | [Pubmed] | [DOI] | | | 5 |
A Systems Approach to Understanding and Improving Research Integrity |
|
| Dennis M. Gorman,Amber D. Elkins,Mark Lawley | | Science and Engineering Ethics. 2017; | | [Pubmed] | [DOI] | | | 6 |
Bewertung der methodischen Qualität publizierter RCTs in der Kataraktchirurgie |
|
| C. Baulig,F. Krummenauer,S. Knippschild | | Der Ophthalmologe. 2017; | | [Pubmed] | [DOI] | |
|
 |
|
|
|
