|Year : 2019 | Volume
| Issue : 2 | Page : 57-61
Risk-based monitoring: Review of the current perceptions and toward effective implementation
Veena Shridhar Jaguste
CEO, Medicines n We, Clinical Research and Healthcare Consultant, Mumbai, Maharashtra, India
|Date of Web Publication||4-Apr-2019|
Dr. Veena Shridhar Jaguste
A/113, Oshiwara Industrial Centre, New Link Road, Goregaon West, Mumbai - 400 104, Maharashtra
Source of Support: None, Conflict of Interest: None
| Abstract|| |
The United States Food and Drug Administration issued a guidance to industry in August 2013 on risk-based approach to monitoring. This prompted industry (sponsors and contract research organizations) to brainstorm, conceptualize, and implement risk-based monitoring (RBM) in their clinical studies and programs. The acceptance and implementation across the organizations have been variable in terms of pace and methodology. Published literature, commentaries, and views through Internet search were reviewed to understand the perceptions about RBM of different key stakeholders whose function has been significantly impacted, as these highlight ground-level challenges while implementing this major change. Some solutions are proposed to address these perceptions and challenges, as sooner than later RBM will become a way of life, given that recent ICH E6, revision 2 (November 2016) includes RBM in the document. Conceptual clarity, change management, skillset, and capacity building will be the key areas of focus to make RBM successful.
Keywords: Capacity building, change management, clinical research associates, clinical sites, data managers, medical monitors, perceptions, risk-based monitoring, skillset
|How to cite this article:|
Jaguste VS. Risk-based monitoring: Review of the current perceptions and toward effective implementation. Perspect Clin Res 2019;10:57-61
| Introduction|| |
Toward the end of the 20th and beginning of the 21st century, regulatory authorities across the world, including the United States Food and Drug Administration (US FDA), faced challenges while assessing the submissions. The number of clinical trials and their complexities increased dramatically, and trials became truly global involving several countries and hundreds of sites. The challenge was to get the assurance of effective oversight, data integrity and protection of safety, rights, and well-being of the trial subjects. The impediments were large geographic dispersion of the study/ies, variable investigator experience, site infrastructure, differences in standard of care, and treatment preferences. The traditional oversight method of regular on-site visits by the clinical research associate (CRA) was becoming quite cumbersome, time, resource, and cost intensive.
Around the same time, technological advances offered several advantages. Technology platforms were now available to remotely collect and collate and visualize large data, including tracking the site performance. Enhancement of statistical assessment techniques enabled identification of trends, risks, and outliers early on. This led to thinking of exploring effective, alternate methods of oversight. The US FDA, in their guidance document, proposed one such approach risk-based monitoring (RBM).
The two fundamental principles of study oversight remain unchanged, in fact, are reemphasized in the guidance. These are protection of participants' safety, well-being, rights and assurance of high quality, and accurate data. The three factors the guidance highlights are quality planning for risk identification, prevention, and mitigation (quality by design) at the study planning stage with ongoing review and revisions, customized, flexible monitoring plan with appropriate mix of central, remote, and onsite monitoring, and real-time remote centralized review and management of the data.
The US FDA guidance on RBM was published in August 2013. Till then, pharma-sponsored studies followed a standard approach for oversight – regular (at predefined frequency) on-site monitoring visits by the CRA with 100% source document verification (SDV). FDA, however, acknowledged that studies sponsored by academic centers, cooperative groups, and government agencies have been using less frequent and less intense on-site monitoring and the data from these studies have been accepted by the agencies and practitioners paving the way to alternate approaches of oversight.
Thus, the real push to adopt RBM at industry level started post-August 2013. The implementation approaches have been variable across the organizations. Regardless, the change has been disruptive impacting several roles in the operations chain.
More than 4 years after the US FDA guidance, I reviewed the published literature studies, opinions, and commentaries by doing Internet search, using search terms such as RBM methodology, RBM challenges, RBM impact on different stakeholders, RBM tools, and their assessments. I summarize below the perception of CRAs, investigators, and sites about RBM and changing the role of data managers and medical monitors in the new model. These functions (among others) are involved in day-to-day operationalization and monitoring of the clinical studies.
| Clinical Research Associates' Perception About Risk-Based Monitoring|| |
In 2014, The German Federal Association of Contract Research Organizations (CROs) (BVMA), supported by the European CRO Federation, conducted an e-mail-based survey among CRAs in Europe (mostly monitoring in Germany), to assess their understanding of and experience related to working in RBM model. The survey was repeated in 2016 (Internet-based online survey) to understand how knowledge and practical experience with RBM have changed the perceptions over a period of time. Christina et al. compared the results of both the surveys.
A total of 180 and 231 CRAs participated in the survey in 2014 and 2016, respectively. The respondents had average 3 years or more of monitoring experience. Thirty-six percent of respondents in 2014 and 41% in 2016 had experience of working in the RBM model (number of studies monitored in the RBM model were between 1 and 5). Only CRAs with experience in RBM were asked to respond to specific questions related to RBM. These questions were focused on four key areas – working efficiency, effective site contact, data quality, and patient safety. The responses/concerns are listed in [Table 1].
|Table 1: Clinical research associate perceptions/experience about risk-based monitoring|
Click here to view
| Changing Role of Data Managers With Risk-Based Monitoring Implementation|| |
With the adoption and implementation of RBM, the role of data management function has undergone a paradigm shift. Data management is expected to play a bigger role right from the protocol development stage. This is because RBM relies heavily on real-time data review and analysis to identify the trends, risks in the study with reduced on-site oversight. Thus, data management inputs are critical at protocol and monitoring plan development stage as well as during the study, as they can highlight critical data points, potential risk areas, and suggest mitigation plans. Poststudy completion, they help map and document the lessons learned.
Other parallel advances/changes (some highlighted below) also have made the job of data managers more challenging. They now handle not only much higher volume but also diverse data from different sources during the drug development. For example, electronic clinical outcome assessments, real-world data collected through smartphones and wearable devices, from social media, and electronic health/medical records. The new data areas, including unstructured data, are being managed outside primary electronic data capture system that collects data from the sites on electronic case report forms (CRFs). These present additional integration and technological challenges to the data management group.
| Challenges Faced by the Medical Monitors|| |
Medical monitors have a critical role to ensure focus on the safety of the trial subjects and identify important safety signals. During recent informal poll, as many as 37% expressed that their greatest fear was the risk of missing safety signal. Although not specific to RBM, most issues they raised are also applicable to RBM model. Currently, the way most data collected on CRF is usually optimized for the capture of source data from the site; however, the design is not optimized for the safety analysis. The data is present in different formats and exists many times in silos. Despite the best attempts, it may be unclean or erroneous at times. Thus, it may be challenging to quickly spot meaningful trends and outliers without assistance from technology team and statisticians. With RBM, this is complicated by the fact that data management is generally outsourced, and the team may be in different continents, while the medical monitors work from the United States of America or European Union making real-time coordination difficult. The review process is generally tracked through Excel-based applications, e-mails, and sticky notes, etc., by different stakeholders making it complicated to track what has changed over a period.
| Perceptions About Risk-Based Monitoring and Challenges Faced by the Sites|| |
The clinical study is conducted and data being generated at the sites, thus their understanding of RBM is pivotal to the success of the model. A survey was conducted in ten countries (six emerging and four developed) about the perception and understanding of RBM in the mind of site staff. Out of 3000 site team members, the survey was forwarded to, 595 (around 20%) responded. Out of 595 responders, 289 responders were not familiar with RBM and were excluded from the analysis. Of the remaining responses, 100 responses from emerging nations and 137 from developed countries were complete and further analyzed. The survey was carried out between July and September 2014. Out of 237 who had participated in RBM (more from the developed countries), only 19% from developing countries (Argentina, Brazil, China, India, Russia, and South Africa) and 27% from developed countries (The United States, the United Kingdom, Australia and Germany) claimed to have good conceptual knowledge of RBM. All participants had perception that RBM reduces overall cost of the study conduct. When asked if RBM may be more effective in addressing data quality, safety, and finding fraud/fabrications, researchers from emerging countries (mean 6.6 out of 98 respondents) were more confident than their counterparts in developed countries (mean 5.8 out of 131 respondents, P = 0.01).
Commentaries have been written about perceived burden about sites with RBM implementation. One such commentary by Bois Bob  mentions of perception of increased work burden on the sites, imposition of unrealistic data entry timelines, and reduced quality control activities by the CRAs.
Number of investigators I spoke to during the implementation of RBM in my earlier organization had the fear of missing their own genuine mistake/s due to lack of quality control by the CRA. The sites also mentioned the struggle implementing RBM for different sponsors, as the approach, methodology, and tools used might be quite different.
In a systemic survey, the authors reviewed 91 potential RBM tools of which 24 were eligible for inclusion of assessment. One of the conclusions was that RBM tools for clinical trials are relatively new, their features, use vary widely, and they continue to evolve. It is, therefore, difficult to define the “best” tool. Thus, it is not surprising that the sites struggle.
All study sites face a perennial problem of hiring and retaining good quality clinical research coordinators (CRCs) with backup available at all the time, as adequacy of funds cannot be assumed/assured continuously. Frequent CRC turnover and attrition can adversely impact RBM implementation, as RBM requires site team to have high scientific and technical skills with adequate training, self-reliance as CRA no longer frequently visits the sites to support CRCs.
| Discussion|| |
Why these perceptions?
RBM is potentially an effective alternative to traditional monitoring and recently included in ICH E6 revision. It is gathering momentum and may soon become “business as usual.” Why then there are still concerns, less confidence in the minds of those who operationalize this concept? One obvious reason could be that these are growing-up pangs as model evolves. However, the surveys, experiences shared, and opinion pieces point toward some gaps that need to be addressed.
The site survey showed that despite the participation in the study using RBM, many admitted lack of conceptual knowledge about RBM. This could be because many sponsors started implementing RBM with reduced on-site visits and SDV with real-time-enhanced data review, rather than following quality-by-design approach at the protocol inception stage. They trained the project managers, CRAs, and sites on tactical and technical aspects of implementation leaving a potential gap in understanding the “why” or concept. While training may have mentioned about enhanced real-time remote data review to ensure quality and patient safety, the key interpretation by sites seems to view RBM as a major cost-cutting initiative by the sponsors with reduced CRA visits to the site. Leave aside the sites, in a recent survey presented at Summit for Clinical Trial Operations Executives conference, the responders (mainly sponsors and CROs) still ranked “reduction of monitoring cost” as one of the top three reasons for adopting RBM. Thus, there seems to be a gap in conveying the concept in a convincing manner to all stakeholders that needs to be addressed.
The sites also seem to have the perception of sponsor shifting the administrative and other burdens, passing the risk to them, possibly because they were left to themselves to address the ground-level difficulties.
For CRAs, it is a significant shift in the way they have been monitoring. They not only need to understand the concept well but also learn new skills/enhance existing ones such as master the new technology tools, develop enhanced analytical abilities, work remotely with site teams, and yet give them adequate support. The learning curve has been quite steep. Their concerns about missing critical safety information, inability to ascertain process adherence at the site, adequate training to new site staff, and the assurance of adequate PI oversight need to be looked into and addressed.
Similarly, the data managers also need to understand the new/enhanced responsibilities the model has created for them and acquire the necessary skills. Medical monitors need to get assurance and confidence that they can detect the important signals with relative ease through enhanced analytics and seamlessly coordinate with different functions/stakeholders to get required inputs. Complexities of navigating through new evolving systems, time required, and coordination with different functions spread across the globe are probably leaving them feeling uncertain about their outputs and interpretations.
Many experts have written and suggested solutions, several CROs and technology companies are working actively on effective implementation solutions – mainly technology driven. TransCelerate Biopharma Inc, a nonprofit organization (www.transceleratebiopharmainc.com), had RBM as one of their five initial initiatives started in 2012 to create effective solutions of study oversight using this concept. They have done/are doing a lot of work in this area, and their Risk Assessment and Categorization Tool (RACT) is widely recognized and utilized.
As organizations move toward finding uniform standardized processes and tools, I would like to highlight few key and practical areas of focus toward the successful implementation of RBM.
- Change management: For a disruptive change, systematic plan to take all impacted roles through change management cycle is critical. While most organizations do this continued assessment, taking ground-level feedback, appropriate modifications to the approach, and additional timely support need to be provided. Introspection by sponsors and CROs is required if the sites were truly supported for the change management and given conceptual clarity when RBM was rolled out. While it may seem a daunting task, sponsor organizations need to think of ways they can additionally support the sites through this transition
- Training and scaling up: Several roles within the sponsor/CROs must function differently in the RBM model and need to acquire new/enhanced skill sets in several areas. The site teams also need to enhance their scientific, technical knowledge, and skills. The assumption that all these stakeholders, with technical training, will immediately develop required skills may not have worked with all. Investing time, efforts, and resources will help in making current teams more skilled, confident, and help in capacity building, critical for the success of the model
- Providing assurances with data: Going forward, using ongoing analysis of the large data with advanced techniques, sponsors should work toward objectively demonstrating how the RBM helps in prevention, identification, and early mitigation of risks to a study, program, and results in better quality and better patient protection. That will alley the apprehensions that have been expressed in the surveys by key stakeholders
- Choosing the right mix of on-site and remote monitoring: The US FDA guidance  states that “No single approach to monitoring is appropriate or necessary for every clinical trial. Ordinarily, the risk-based plan developed by the sponsor would include a mix of centralized and on-site monitoring.” While reduced on-site monitoring may have several advantages, it is important for the CRA to maintain connection with the site – keep the “A” or “Association” intact. Their feedback on the effectiveness of approach and flexibility to modify the monitoring plan appropriately (collaborative vs. top-down approach) can augment the quality conduct of the studies
- Trial participants: While RBM would drive technology-based efficiencies, we should continue to focus on minimizing the burden of the study participants. According to a survey of 3150 clinical trial participants by the center for information and study, over 20% mentioned that they find the experience of trial participation quite stressful. What they liked least was distant location of the study center and traveling, time-consuming nature of study visits, and undergoing several cumbersome procedures during the visits. Ensuring patient comfort is critical to compliance during the study (minimum missed visits, procedures, adherence to IP, and overall treatment plan) and will result in meaningful, high-quality data. RACT tool by Transcelerate includes patient burden in Section 3.3 that should be included in quality by design plan. This will inspire more patients to participate in the studies that eventually may bring the pathbreaking medicines sooner to the patients and better health care for all.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Christina K, Rollinger Y, Sigmund M, Kunert V, Breuer B. RBM – An Update of Experiences Among European CRAs. Applied Clinical Trials; 20 October, 2017.
Wilkinson M, Getz K. A Critical Juncture for Clinical Trial Data Management. Clinical Leader; 02 November, 2017.
Masha H. Medical Monitors Fear this Most. Clinical Leader; 19 September, 2017.
Kumar PP, Dalal J, Jadhav MP. Risk-based monitoring (RBM): A global study focusing on perception and merits among clinical investigational sites. Contemp Clin Trials Commun 2016;4:155-60.
Bois B. Risk-Based Monitoring: What Does it Mean for Clinical Study Sites? Applied Clinical Trials; 14 October, 2016.
Hurley C, Shiely F, Power J, Clarke M, Eustace JA, Flanagan E, et al.
Risk-based monitoring (RBM) tools for clinical trials: A systematic review. Contemp Clin Trials 2016;51:15-27.
ICH E6 Revision. Integrated Addendum to ICH E6 (R1): Guideline for Good Clinical Practice E6(R2); November 2016.
Sfera D. Remote Monitoring vs. Risk-Based Monitoring. Clinical Leader; 26 April, 2017.
Schroeder J. Patient Centricity: Making Clinical Trials More Effective by Keeping Focus on the Patients. Clinical Leader; 18 December, 2017.