|Year : 2018 | Volume
| Issue : 3 | Page : 123-126
An analysis of completeness and quality of adverse drug reaction reports at an adverse drug reaction monitoring centre in Western India
Manali Mangesh Mahajan, Urmila Mukund Thatte, Nithya Jaideep Gogtay, Siddharth Deshpande
Department of Clinical Pharmacology, Seth GS Medical College and KEM Hospital, Mumbai, Maharashtra, India
|Date of Web Publication||12-Jul-2018|
Dr. Urmila Mukund Thatte
Department of Clinical Pharmacology, Seth GS Medical College and KEM Hospital, Mumbai - 400 012, Maharashtra
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Purpose/Aim: The Adverse Drug Reaction [ADR] form is the source document for the Pharmacovigilance Programme of India [PvPI] and captures information first hand from the patient. The raw data from it then gets converted into an individual case safety report [ICSR] after entry into Vigiflow. The National Coordinating Centre [NCC] uses an instrument to assess quality of these ICSRs. We carried out the present study to assess whether the same instrument with minor modifications could be used to check the quality of ADR forms at our centre.
Materials and Methods: ADR reports of three months from three consecutive years were selected randomly. The ADR form [18 fields] was matched with the NCC instrument [14 fields] as the latter is made from the former. A perfect ICSR would score 1. Three fields in the NCC instrument - case narrative, compliance with standard operating procedures [SOPs] and free text [5 components] were modified, while the rest were retained. Zero was given to the first two fields. In the third field, we retained only 3/5 components and changed the last two components [sender and reporter comments] to dechallenge and rechallenge while keeping the total score the same.
Results: A total of 1008 ADR reports were analyzed. We found an overall completeness score of approximately 80% with the lowest completeness score being for the year 2015. The mandatory fields had close to 100% scores.
Conclusion: The NCC instrument was found well suited to evaluate quality and completeness of ADR forms.
Keywords: ADR form, completeness score, India, instrument, pharmacovigilance, quality assessment
|How to cite this article:|
Mahajan MM, Thatte UM, Gogtay NJ, Deshpande S. An analysis of completeness and quality of adverse drug reaction reports at an adverse drug reaction monitoring centre in Western India. Perspect Clin Res 2018;9:123-6
|How to cite this URL:|
Mahajan MM, Thatte UM, Gogtay NJ, Deshpande S. An analysis of completeness and quality of adverse drug reaction reports at an adverse drug reaction monitoring centre in Western India. Perspect Clin Res [serial online] 2018 [cited 2019 Nov 22];9:123-6. Available from: http://www.picronline.org/text.asp?2018/9/3/123/236479
| Introduction|| |
The Pharmacovigilance Programme of India (PvPI) was launched in 2010 to ensure the safety of medicines, and the Indian Pharmacopoeia Commission functions as the National Coordinating Centre (NCC) for the PvPI under the aegis of Ministry of Health and Family Welfare, Government of India. Currently, over 200 adverse drug reactions (ADRs) and ADR monitoring Centres (AMCs) in the country are recognized to monitor and report ADRs. All centres upload ADR reports into VigiFlow™ which is the World Health Organization-Uppsala Monitoring Centre's (WHO-UMC) web-based system to collate ADRs worldwide.
It is the ADR form which serves as the source document and primarily captures information firsthand from the patient. This raw data then get secondarily converted into an individual case safety report (ICSR) after the data entry into VigiFlow, and these ICSRs are now evaluated by the NCC for quality and signal generation. In 2016, the NCC published an in- house method developed by them for evaluating the quality of ICSRs. This method gives a weighted score to each field, and a final completeness score is generated for each ICSR using a multiplicative model.,
If the information filled in the ADR form is complete, it hastens the process of signal generation. Hence, a quality check at the level of the source document, which being the very first step in the process of signal generation, will prove to be beneficial at not only the level of the AMC but also on a national and worldwide level. Hence, the present study was carried out with the objective of evaluating completeness and the quality of ADR reports using the NCC instrument for the same. Since the scoring system used for ICSRs was modified to be applicable to ADR forms, reliability of the modified scoring system was also assessed to see if the findings can be replicated.
| Materials and Methods|| |
This study was deemed exempt from review by the Institutional Ethics Committee and granted a waiver.
Selection of adverse drug reaction reports
ADR reports were selected from three consecutive years (2014, 2015, and 2016). In each of these 3 years, 3 months were randomly selected using a random number table giving a total of 9 months for evaluation and analysis.
The adverse drug reaction form used by Adverse Drug Reaction Monitoring Centre
The suspected ADR reporting form used by the AMC has four elements (patient Information [n = 4 fields], suspected adverse reaction [n = 3 fields], suspected medication [n = 8 fields], and reporter information [n = 2 fields]) that are captured in 17 fields. Among these fields, fields (n = 6 fields) of patient initials and age at onset of reaction, reaction term(s), date of onset of reaction, suspected medication(s): name and details, and reporter information are mandatory. An ADR report is considered valid only when all six mandatory fields are completely filled.
Modifications made in the National Coordinating Centre instrument for scoring adverse drug reaction form
We made minor modifications (given below) to the validated, and published weighted scoring system developed by the NCC to evaluate the ICSR [Table 1]. Three fields in the NCC instrument, those of case narrative, compliance with the NCC standard operating procedures (SOPs), and free text, are absent in the ADR form. Hence, case narrative and compliance with NCC SOPs were scored zero by us, whereas the free-text field was modified. The free-text field in the NCC instrument consists of test procedure, relevant medical history, additional drug information, sender's comments, and reporter's comments. Each of these five components is given 0.2 points (0.2 × 5 = 1). We retained the first three components, but as the last two of sender's comments and reporter's comments are absent in the ADR form, we replaced them with dechallenge and rechallenge information, thereby, still giving one as the total score for this field. This field of free-text data in the NCC instrument corresponds to field of “Others 1” in the ADR form.
|Table 1: Scoring instrument (making minor modifications in National Coordinating Centre instrument) used to evaluate the completeness of adverse drug reaction reporting form|
Click here to view
Assessment of inter-rater reliability using intraclass correlation coefficient
Thirty ADR reports were assessed independently by three different raters who had at least 3 years of training in the discipline using the scoring instrument [Table 1]. Intraclass correlation coefficient was obtained using SPSS version 20 based on a mean rating (number of raters = 3), absolute agreement, and two-way mixed-effects model.
Calculation of completeness score
Completeness score calculated on the basis of the formula given below: completeness score: πn(i=1)(1-wi) + (wi* fi). Where I indicates, the field included in the score, wi is the field weight, and fi is the field score.
Quantitative data (scores) were expressed as median and range. Normality of completeness scores was assessed using the Kolmogorov–Smirnov test. Between-year difference was assessed using Kruskal–Wallis test followed by the Dunn's post hoc test. All analyses were done at 5% significance using IBM SPSS Statistics for Windows, Version 20.0. Armonk, NY and Microsoft Excel 2013.
| Results|| |
A total of 1008 ADR reports were analyzed. Of these, 102 (2014), 568 (2015), and 338 (2016) were from each of the years, respectively.
Assessment of inter-rater reliability using intraclass correlation coefficient
A total of thirty reports were assessed for reliability. Intraclass correlation coefficient (95% of confidence interval) was 0.97 (0.939–0.986).
The overall median (range) completeness score for all 1008 reports was 0.86 (0.28–0.9). The yearly median (range) completeness scores were 0.86 (0.81–0.86) (2014), 0.85 (0.28–0.9) (2015), and 0.86 (0.46–0.9) (2016), respectively. A significant difference was seen between the score for 2015 relative to the other 2 years (P< 0.001).
Score of mandatory fields (n = 6) in the adverse drug reaction form
The overall median score (range) for only mandatory fields for all years was 1 (0–1). The mandatory fields were filled in all the 1008 forms (100%).
Score of nonmandatory fields (n = 8) in the adverse drug reaction form
The overall median score (range) for only nonmandatory fields for all years was 1 (0–1). When individual fields were analyzed separately, a statistically significant difference was seen in the fields of report title (between all years), action taken (2016 vs. the other 2 years), indication (between all years), causality assessment (2016 vs. the other 2 years), and others 1 (2016 vs. the other 2 years). The two remaining fields (seriousness and gender) did not show a significant difference.
| Discussion|| |
Evaluation of the quality of an ADR form involves a multi-dimensional analysis of several criteria that include causality, clinical relevance, ability to add new knowledge, and completeness. Several tools have been developed and used to assess the quality of ADR reports worldwide. These include EudraVigilance feedback report by the European Medicines Agency, vigiGrade completeness score developed by UMC, clinical documentation tool developed by the Netherlands Pharmacovigilance Centre, and the quality of ADR reports algorithm developed in Italy.
In the present study, we used a prior published, weighted scoring method making minor modifications to assess the completeness of the ADRs. The instrument developed in the country by the NCC was chosen as we felt it would be best suited for the evaluation of completeness of the country's ADR data. We found the scoring system to have excellent reliability as even the lower limit of 95% confidence intervals was above 0.9. It was also easy to use, thus meeting the study objective.
We found an overall completeness score of approximately 80% with the lowest completeness score being for the year 2015. While we are unable to explain this particular drop for a single year only, scores can be interpreted in the context of the person reporting them. Reporting at our center is largely done by residents (medical practitioners) in training. Public hospitals such as ours have a low doctor to patient ratio. Thus, patient care remains the primary prerogative of the reporter with filling of the ADR forms taking a backseat.
The mandatory fields had close to 100% scores. This is likely to be a reflection of several awareness and skill development programs that have been undertaken to promote quality ADR reporting by the PvPI since its inception. The overall score of approximately 80% is a result of the much lower scores in the nonmandatory fields. Two nonmandatory fields such as report title and indication were uniformly deficient in all years. Of these, the report title does not exist in the current version of the ADR form. In the absence of a report title, no ADR form can be transcribed into VigiFlow, and this is clearly stated in the SOPs of each AMC. Thus, the report title gets written by patient safety officer at the AMC based on information present in the ADR form. The ADR form is currently undergoing revision, and we recommend that a new mandatory field of the report title be created, so that this is then not missed by reporter. The deficiency seen in the field of indication is likely to stem from the fact that ours being a tertiary referral center, patients are put on multiple medications, and the reporter may find it tedious to fill the indication for each and every drug prescribed.
Our study is limited by the fact that the modifications that we have made in the NCC instrument have been used at our center alone, and this study needs to be done at other AMC centers in the country to see if the findings can be replicated.
| Conclusion|| |
Good quality reporting increases the potential for signal generation, thereby enabling necessary regulatory actions. While challenges such as doctor–patient ratio are difficult to address, attitudes toward reporting, and quality of reporting can be easily addressed by educational and training interventions. It has been shown that even a small 15-minute lecture video demonstration can significantly improve the quality of ADR reports. Other measures to improve the quality used world over include verbal, written, and telephonic reminders. Feedback to the reporter in the form of case discussions journal clubs, regular newsletters would also go a longway in improving both the number and the quality of reports submitted to the AMCs.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Kalaiselvan V, Sharma S, Singh GN. Adverse reactions to contrast media: An analysis of spontaneous reports in the database of the pharmacovigilance program of India. Drug Saf 2014;37:703-10.
Bergvall T, Norén GN, Lindquist M. VigiGrade: A tool to identify well-documented individual case reports and highlight systematic data quality issues. Drug Saf 2014;37:65-77.
Kalaiselvan V, Kumar R, Kaur I. Best practices for improving the quality of individual case safety reports in pharmacovigilance. Ther Innov Regul Sci 2016;50:464-71.
Vivekanandan K, Rishi K, Prasad T, Arunabh T, Singh GN. Status of documentation grading and completeness score for Indian individual case safety reports. Indian J Pharmacol 2015;47:325-7.
] [Full text]
Tuccori M, Giustarini G, Blandizzi C, Capogrosso-Sansone A, Rossi M, Gori G, et al
. Quality of Adverse Drug Reaction (QADRA) reports: An algorithm to appraise the efficiency of spontaneous reporting systems in pharmacovigilance. J Public Health 2013;21:365-72.
Bajpai V. The challenges confronting public hospitals in India, their origins, and possible solutions. Adv Public Health 2014;1:27-35.
Rosebraugh CJ, Tsong Y, Zhou F, Chen M, Mackey AC, Flowers C, et al.
Improving the quality of adverse drug reaction reporting by 4th
year medical students. Pharmacoepidemiol Drug Saf 2003;12:97-101.
Molokhia M, Tanna S, Bell D. Improving reporting of adverse drug reactions: Systematic review. Clin Epidemiol 2009;1:75-92.