|
 |
| EDITORIAL |
|
| Year : 2019 | Volume
: 10
| Issue : 2 | Page : 49-50 |
|
Reporting of randomized controlled trials: Will it ever improve?
Nithya J Gogtay
Department of Clinical Pharmacology, Seth GS Medical College and KEM Hospital, Mumbai, Maharashtra, India
| Date of Web Publication | 4-Apr-2019 |
Correspondence Address:
Dr. Nithya J Gogtay
Department of Clinical Pharmacology, Seth GS Medical College and KEM Hospital, Mumbai - 400 012, Maharashtra
India
 Source of Support: None, Conflict of Interest: None  | Check |
DOI: 10.4103/picr.PICR_11_19
How to cite this article:
Gogtay NJ. Reporting of randomized controlled trials: Will it ever improve?. Perspect Clin Res 2019;10:49-50 |
The quest for delivering the best patient care that we can is one that will never end. There are several facets to this delivery, one of which is using the best available evidence. Randomized Controlled Trials (RCTs) dominate this landscape and feature high in the evidence-based medicine hierarchy and quality reporting of these studies is an ethical imperative. The Consolidated Standards of Reporting Trials (CONSORT) to address quality and transparency of reporting was first published in 1996,[1] and its current avatar is the 2010 statement.[2] The statement also has multiple extensions, some of which include noninferiority, equivalence and cluster designs, Chinese herbal medicine formulas, and extension to randomized pilot and feasibility trials among others.[3]
The measure of the success of a guideline is the extent of adherence to it. Editors of several biomedical journals including Core Clinical Journals such as the JAMA, Lancet, New England Journal of Medicine are signatories to the CONSORT. Despite this, literature is replete with papers that report inadequate and/or poor adherence to the CONSORT guidelines.[4],[5] The problem, however, is not unique to CONSORT. Poor adherence is also seen with other guidelines such as the Preferred Reporting Items for Systematic Reviews and Meta-Analyses, STrengthening the Reporting of OBservational studies in Epidemiology, Quality of Reporting of Meta-Analyses, and STAndards for Reporting of Diagnostic accuracy.[6]
Goenka et al. in this issue of the journal evaluated the quality of RCTs published in select Indian Medical Journals and the extent of adherence to the CONSORT 2010 checklist.[7] After a search of PubMed and Google Scholar, 7 of 53 journals were chosen that had published a RCT in 2017. For the 25-item checklist, they allocated 1 mark for complete information and 0 for no information and found a mean compliance score of 13.7 ± 2.66 (57%). While the finding in itself is not a problem or even unanticipated, the methodology used by the authors has several limitations.
Beginning with a missing search strategy, the use of a binary scoring system of presence (1 point) or absence (0 point) would mean that partially present information was scored 0, leading to an underestimate of the compliance. For example, if the word randomization was mentioned by the authors and they also actually followed it, but simply failed to mention the method used for random number allocation; this would still be marked 0 and hence underestimate the overall adherence. Furthermore, a binary scoring system will inherently underestimate the adherence as each of the IMRaD headings in the checklist has multiple subitems that need to be present for a full score of 1. Trial designs (in Methods) for instance, has two components: description of the design itself and description of any changes to it. The absence of the latter would lead to a zero score.
The second problem is that almost three quarters of the papers came from the Indian Journal of Anesthesia with some contribution from the Journal of Obstetrics and Gynecology and the Indian Journal of Ophthalmology. The two latter journals when combined contributed about 15% of the articles. The data are, thus, largely driven by a single journal with negligible contribution from the other journals listed. This grossly limits the generalizability of the findings. The third and fourth problems lie in the discussion. The comparison of the present paper to another published a year earlier makes little sense, given that the 2016 paper pertains to CONSORT adherence in RCTs done in patients with multiple sclerosis. The fourth is the lack of trend analysis, which has been mentioned by the authors. Whether the journals studied endorsed or at least made a mention of adherence to the CONSORT checklist on their website is also unclear.
The limitations aside, it would be useful to look at the genesis of the problem. Indian biomedical journals, unlike say a Core Clinical Journal like the New England Journal of Medicine, are often run by a single editor with a skeletal editorial team. This editor is almost always full-time faculty member, usually at a government medical college and works pro bono in his/her available free time. Many Indian journals are also society-run journals often struggling to maintain acceptable turnaround times, bring out issues in a timely manner, and work ceaselessly toward getting quality articles. Much of this is done with limited or no finances. The best bet to publish a quality RCT, thus, is to rely on the peer reviewer who in turn needs to be knowledgeable and recommend CONSORT adherence. Beyond the recommendation, the onus lies with the author to ensure adherence in the revised manuscript and the editor's oversight of the changes made. All of this is assuming that the methodology used was actually in order and it is just the writing that was flawed. CONSORT adherence in the face of flawed methodology itself is another matter, more difficult to handle by the editor. There does exist evidence in literature of use of flawed methodology leading to erroneous conclusions.[8]
Can we move beyond mere documentation of adherence failures? World literature is now seeing similar problems with the CONSORT extensions.[9] The focus with reporting of RCTs should thus shift to actionable areas for intervention. The identification of deficiencies in the reporting of methods and results is the strength of the paper by Goenka et al.[7] These sections have 17 and 10 subitems, respectively. Figures 2 and 3 in Goenka's paper point toward poor reporting of trial designs, randomization, recruitment, and numbers analyzed as some of the problem areas. Similar findings have been reported recently from journals published outside the country.[10] Focus on these in the future RCTs submitted to these journals would be expected to have the greatest impact on adherence. Similarly, mentioning the CONSORT checklist and guidelines both on the journal website as well as through training workshops by the journal will help improve awareness among authors.
Enforcement of guidelines such as the CONSORT either through a rigorous peer review and editorial process or by being a signatory to it though will be a much tougher proposition. While enforcement is a viable option for leading, high impact factor biomedical journals, Indian journals and their editors on the other hand would be faced with a tough balancing act. While they would need to rigidly enforce these guidelines, it may come at the cost of losing already meager submissions and driving away authors who may not necessarily submit well-written, good-quality RCTs. The question of enforcement, thus, is not one that can be easily answered and will require interactions between multiple stakeholders over time to bring about an attitudinal shift. Finally, if we wish to improve clinical care, health services, and patient outcomes, we need to move beyond RCTs [11] and their reporting to developing new ways of evaluating and using evidence.
 References | |  |
| 1. | Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA 1996;276:637-9.  |
| 2. | |
| 3. | |
| 4. | Stevanovic A, Schmitz S, Rossaint R, Schürholz T, Coburn M. CONSORT item reporting quality in the top ten ranked journals of critical care medicine in 2011: A retrospective analysis. PLoS One 2015;10:e0128061. |
| 5. | Lee SY, Teoh PJ, Camm CF, Agha RA. Compliance of randomized controlled trials in trauma surgery with the CONSORT statement. J Trauma Acute Care Surg 2013;75:562-72. |
| 6. | Jin Y, Sanger N, Shams I, Luo C, Shahid H, Li G, et al. Does the medical literature remain inadequately described despite having reporting guidelines for 21 years? A systematic review of reviews: An update. J Multidiscip Healthc 2018;11:495-510. |
| 7. | Goenka L, Rajendran S, Arumugam K, Rani J, George M. The assessment of the quality of randomized controlled trials published in Indian medical journals. Perspect Clin Res 2018;10:79-83. |
| 8. | Thombs BD, Kwakkenbos L, Levis AW, Benedetti A. Addressing overestimation of the prevalence of depression based on self-report screening questionnaires. CMAJ 2018;190:E44-9. |
| 9. | Chan CL, Leyrat C, Eldridge SM. Quality of reporting of pilot and feasibility cluster randomised trials: A systematic review. BMJ Open 2017;7:e016970. |
| 10. | Susvirkar A, Gada P, Figer B, Thaker S, Thatte UM, Gogtay NJ. An assessment of the compliance of randomized controlled trials published in two high impact journals with the CONSORT statement. Natl Med J India 2018;31:79-82. [ PUBMED] [Full text] |
| 11. | Frieden TR. Evidence for health decision making – Beyond randomized, controlled trials. N Engl J Med 2017;377:465-75. |
|
