|Ahead of print publication
A comparative study to evaluate quality of data documentation between investigator-initiated and pharmaceutical industry-sponsored studies
Brinal H Figer, Keyur P Sapra, Nithya J Gogtay, Urmila M Thatte
Department of Clinical Pharmacology, Seth Gordhandas Sunderdas Medical College and KEM Hospital, Mumbai, Maharashtra, India
Urmila M Thatte,
Department of Clinical Pharmacology, First Floor, New Multistorey Building, KEM Hospital, Parel, Mumbai - 400 012, Maharashtra
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Purpose: In pharmaceutical industry funded clinical studies (PIS), there is rigorous monitoring to ensure adequate and accurate data documentation. In comparison, the investigator-initiated studies (IIS) often lack in resources and may not follow such quality checks. At present, very limited data on the existing deficiencies in documentation for IIS are available. Hence, the present study assessed data quality in IIS relative to those funded by the industry to identify and address issues in data documentation.
Materials and Methods: We evaluated records of 1276 participants in 13 studies (5 – industry sponsored and 8 – investigator initiated) conducted during 2009–2015 using a prevalidated checklist. The percentage total scores for overall documentation and general trial-related and patient-specific documents were calculated. The percentage total scores within the patient-specific documents were also calculated and compared. Between-group score analysis was done by Student's t-test using GraphPad InStat version 5.0.
Results: The mean (standard deviation [SD]) percentage total score for the IIS was 80.96 (13.26) and that for PIS was 98.77 (1.84) (P = 0.01). For IIS, the total percentage scores ranged from 63% to 100% while it was above 95% for all PIS. For general trial-related documents, the mean (SD) percentage total score for IIS was 90.39 (13.26) while that for PIS was 97.38 (0.92) (P > 0.05). In the patient-specific documents, IIS scored 80.52 [14.41] versus 98.95 (1.98) for PIS (P = 0.016). The scores for IIS versus PIS (70.22 [21.6] and 99.36 [1.43]) within patient-specific documents were significant only for admission criteria (P = 0.016).
Conclusion: Quality of IIS needs to be addressed by greater oversight and periodic quality control assessments.
Keywords: Data documentation, internal monitoring, investigator-initiated studies, quality
|How to cite this URL:|
Figer BH, Sapra KP, Gogtay NJ, Thatte UM. A comparative study to evaluate quality of data documentation between investigator-initiated and pharmaceutical industry-sponsored studies. Perspect Clin Res [Epub ahead of print] [cited 2019 Sep 22]. Available from: http://www.picronline.org/preprintarticle.asp?id=257950
| Introduction|| |
The International Conference on Harmonization–Good Clinical Practices (ICH-GCP) recommends generation of credible data in clinical trials. However, despite this, lack of reliable, accurate, and adequate source documentation is a common inspection finding at clinical trial sites., Systemic deficiencies in documentation may lead health authorities to ask for exclusion of such data from final analyses and regulatory submissions. Identification of these deficiencies is largely seen in pharmaceutical industry-funded clinical studies (PIS) which follow a system of rigorous monitoring and audits. In comparison, investigator-initiated studies (IIS) lack in resources and may not have similar quality checks. IIS are, however, equally important for the reason that they reassess safety and effectiveness of already approved drugs and explore the use of marketed drugs for new indications in different dosage regimens among others.
Studies in literature have emphasized the need to improve methodological and reporting quality of both PIS and IIS., An exhaustive literature search revealed scant evidence regarding comparison of data integrity of IIS vis-a-vis PIS. Hence, we conducted the present study with the objective of comparing the quality and completeness of documentation in IIS versus PIS done at a tertiary referral center.
| Materials and Methods|| |
The Institutional Ethics Committee (IEC) approval was taken before the study was conducted (EC/OA-08/2015). A consent waiver was obtained for the study from the IEC as this study was a retrospective audit. Confidentiality was maintained using unique identifiers.
Study selection criteria
The archival ledger of the department was retrieved and studies completed between the years 2009 and 2015 were considered for assessment. No studies were excluded.
Type of studies assessed and sample size
A total of n = 13 studies (5 – PIS and 8 – IIS) formed the study sample.
Pharmaceutical industry-sponsored studies
These were five in number and all five were interventional in nature. One was a Phase I and four were Phase II/III studies.
These were eight in number – three were interventional in nature and the remaining five were observational. Of the interventional studies, one was a bioequivalence study and two were pharmacokinetic evaluations.
A prevalidated and published checklist was used to assess quality and completeness of the documentation. We assessed general trial-related and patient-specific documents. For general trial-related documents, the following elements/subitems were assessed: presence of protocol, ethics committee correspondence, various study logs, and patient safety data. For patient-specific documents, we assessed inpatient information (source documents including outpatient and inpatient papers of the hospital), informed consent documents, source documents, case record forms (CRFs), and protocol and safety parameters.
Each study document was scored independently by authors BF and KS. These were verified by the senior authors NG and UT. Disagreements related to scoring were resolved through consensus.
Each element/subitem in the checklist for both general and patient-specific documents was scored out of a total of “1.” When documents were present but were incomplete, a score of “0.5” was assigned. Zero was the score given when the document was missing.
Due to the variation in applicable subitems for the different studies, the total possible score (TPS) for each study was different, and thus, percentage scores were calculated. For example, the subitem “investigator brochure” was not applicable to some IIS studies.
The number of patients for each study varied and the TPS for each subitem also differed for each study. The TPS for each subitem was given as TPS = (score given for the subitem present, incomplete, or absent) × (number of participants). For example, if a study had 22 participants, for the subitem “approved version of the informed consent,” the maximum possible score was 22. If the approved version of informed consent was present for all participants the total score would be 22. Similarly, total percentage scores were calculated for all the subitems. When marks were deducted, the reasons were noted.
The overall documentation for the studies was also assessed. This was done by adding the scores obtained for the general and patient-specific documents.
Outcomes of interest
These included (a) total scores for overall documentation, (b) general trial-related documents, and (c) patient-specific documents.
These were individual elements within patient-specific documents such as admission criteria (inpatient history sheet and eligibility criteria), informed consent, CRF, source documentation (laboratory reports, patient history sheets, and details of vital parameters), and protocol and safety parameters (number of protocol deviations reported, documentation of serious adverse events (SAEs), and AEs). Furthermore, documentation deficiencies were collated.
The demographic data for the number of participants in the studies were expressed in absolute numbers and percentage. Scores were assessed for normality using the Kolmogorov–Smirnov test. The percentage total scores were calculated and expressed as either mean (standard deviation [SD]) or median (range) based on whether scores were normally distributed or otherwise.
The percentage scores between the IIS and PIS were assessed using Mann–Whitney U-test or unpaired Student's t-test based on the normality of the score. All the statistical analysis was performed using Graphpad Instat version 5.0 (Graphpad Software, La Jolla, California, USA, 2007) at 5% significance.
| Results|| |
In the 13 studies, a total of n = 1276 participants were enrolled. Of these, 959/1276 (75%) were enrolled in IIS and 317/1276 (25%) in PIS.
Overall percentage total scores [PTS]
The mean (SD) percentage total score for the IIS was 80.96 (13.26) and that for PIS was 98.77 (1.84) (P = 0.01). For IIS, the total percentage scores ranged from 63% to 100% while it was above 95% for all PIS. For general trial-related documents, the mean (SD) percentage total score for IIS was 90.39 (13.26) while that for PIS was 97.38 (0.92) NS. In the patient-specific documents, IIS scored 80.52 (14.41) versus 98.95 (1.98) for PIS (P = 0.016). The details of the scores are given in [Table 1] and [Table 2].
|Table 2: Summary of percentage total score obtained: general trial-related and patient-specific documents|
Click here to view
Scores for individual elements within patient-specific documents
The scores for IIS versus PIS were 70.22 (21.6) and 99.36 (1.43) for admission criteria (P = 0.016); 85.26 (12.33) and 97.17 (6.31) for informed consent (P = 0.07); 93.87 (9.03) and 91.27 (19.17) for source documents (P = 0.74); and 60.34 (27.97) versus 90.81 (18.77) for CRF (P = 0.06), respectively [Figure 1]. All analysis done using unpaired t test.
|Figure 1: Scores obtained within the patient-specific documents. *Denotes significance using unpaired t test|
Click here to view
Among IIS, 7/8 did not have all the amended versions of the protocol filed in the Trial Master File, the “final approved IEC version” of the protocol stamp was missing in 4/8 studies, reference ranges were not filed in 3/8 studies, and study logs (for example, screening, enrollment, and investigational product administration log) were missing in 7/8 studies. None of these deficiencies were seen in PIS.
| Discussion|| |
The present study compared the quality of data documentation in IIS versus PIS and found that the overall quality of documentation was better in PIS (percentage total score: 98.77 vs. 80.96) relative to IIS, and this difference was statistically significant (P = 0.01). This was driven by the difference in the scores in the patient-specific documents between the two types of studies while scores for the general trial-related documents did not differ significantly.
The difference in overall quality of documentation between IIS and PIS could be attributed to the fact that IIS do not undergo the rigorous monitoring that PIS routinely undergo. The quality of documentation as measured by the percentage total score for general trial-related documents did not vary significantly between the PIS and IIS (97.38% vs. 90.39%). This observation is similar to a study by Patwardhan et al. who evaluated the quality and completeness of data between one industry-sponsored and one investigator-initiated studies. The reason why there are no differences in the general trial documents is likely to be due to the fact that most of these documents such as study protocol, informed consent documents, letters of correspondence with the ethics committees, CRFs, study staff's curriculum vitae, and GCP certificates are maintained as an integral part of the IEC submissions and are driven by IEC standard operating procedures (SOPs), thus mandating compliance.
The quality of documentation for patient-specific documents was better in PIS relative to IIS (98.95 vs. 80.96%; P = 0.02). This has serious implications as the key mandates of GCP are data integrity and patient protections. Our results highlight the need to improve documentation and monitoring for IIS. In addition, investigator sites must have and implement site-specific SOPs to ensure better patient documentation and GCP compliance. Another way to do this is to allocate study-specific monitors even in IIS further emphasizing the need for fund allocation for this activity. IEC monitoring these studies regularly also would help ensure compliance and improve quality of documentation.
Within the patient-specific documents, we found that the scores were higher for PIS relative to IIS regardless of the nature of the document (admission criteria, informed consent, source documents, and CRF), though it was significant only for admission criteria (70.2% for IIS vs. 99.36% for PIS; P = 0.02). The score was also low (60.34% for IIS as against 90.3% for PIS) for CRFs but not statistically significant (P = 0.07).
The low scores seen for informed consent documents are a matter of concern since informed consent is an important document to protect the participant's safety and rights and is an index of the autonomy. Literature shows that deficiencies related to patient-specific documentation are seen in IIS as well as PIS. A review of the warning letters issued by the US Food and Drug Administration (FDA) to the investigators between the years 2005 and 2010 found that inconsistencies in documentation of informed consent were the third most common reason behind issuing of warning letters. Similarly, source documentation errors were listed among the top ten findings by the European medicines Agency. Another study that analyzed the warning letters issued to the clinical trial investigators by the US FDA found that violations related to informed consent and CRF documentation were among the main reasons for the issue of these letters.
Our study is limited by the fact that it is restricted to research conducted at a single center and thus may have limited generalizability. Furthermore, there is a heterogeneity in the studies evaluated, especially among the IIS, where the 5/8 of the studies were observational in nature whereas all PIS were interventional.
The ICH-GCP recommends “accurate capture, storage and reporting of data” for all documents. However, compliance with these recommendations requires significant investment in terms of resources. The IIS quite often lack adequate workforce to carry out intense reviews and arduous monitoring of the trials. However, despite this, the documentation deficiencies must be addressed as accurate and diligent data collection along with storage and reporting could impact analysis and interpretation of the results.
| Conclusion|| |
Maintaining GCP is essentially an attitude, and therefore, regardless of the type of study, adequate steps must be taken to ensure quality data capture and documentation.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Bhatt A. Quality of clinical trials: A moving target. Perspect Clin Res 2011;2:124-8. [Full text]
Bargaje C. Good documentation practice in clinical research. Perspect Clin Res 2011;2:59-63.
] [Full text]
Patwardhan S, Gogtay N, Thatte U, Pramesh CS. Quality and completeness of data documentation in an investigator-initiated trial versus an industry-sponsored trial. Indian J Med Ethics 2014;11:19-24.
Siegfried N, Clarke M, Volmink J, Van der Merwe L. African HIV/AIDS trials are more likely to report adequate allocation concealment and random generation than North American trials. PLoS One 2008;3:e3491.
Gogtay NJ, Doshi BM, Kannan S, Thatte U. A study of warning letters issued to clinical investigators and institutional review boards by the United States food and drug administration. Indian J Med Ethics 2011;8:211-4.
Bellary S, Krishnankutty B, Latha MS. Basics of case report form designing in clinical research. Perspect Clin Res 2014;5:159-66.
] [Full text]
Shetty YC, Saiyed AA. Analysis of warning letters issued by the US food and drug administration to clinical investigators, institutional review boards and sponsors: A retrospective study. J Med Ethics 2015;41:398-403.
Chan AW, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: Review of publications and survey of authors. BMJ 2005;330:753.
[Table 1], [Table 2]