Implementing Risk-Based Monitoring: 6 Lessons From The Trenches
By Laurie Halloran and Megan Brickley, Halloran Consulting Group
The initial FDA guidance for industry on taking a risk-based approach to monitoring was released in August of 2013.1 This was the FDA’s effort to bring industry a more effective means of executing what was previously an incredibly laborious and expensive process that yielded very little in the way of better data quality in return for a lot of expense.2 At a public presentation on the topic by one of the authors in June of that year, the room was packed with people displaying varying levels of excitement, apprehension, and skepticism.
Now, putting the theory of risk-based monitoring (RBM) into practical application has become an increasing focus within companies, especially as adoption of the new ICH E6 (R2) standards has pushed sponsors to consider what should change in their approach to monitoring. While risk management as a concept has always played a role in the clinical trial space, there is increased pressure to identify and manage trial-related risks more systematically.
It is accepted that the traditional process of site monitoring does not really result in higher-quality data, but there has been little practical guidance on implementing RBM. This vacuum has left many of our colleagues who are clinical development leaders unaware of what to expect when deciding to move forward with a process.
So, what are some insights from the trenches that haven’t already been covered on the topic?
During a recent in-depth discussion with some of our close industry colleagues, a few themes arose around their experiences with RBM implementation that we believe have value. The following sections discuss six important considerations when making the decision to utilize RBM as part of your operational growth strategies:
1. You will not save money … initially.
Let’s start with this: If your decision to implement RBM is strictly for the cost savings, you should reconsider. In fact, application of RBM will actually cost you money initially, due to increased identification of risk — which will demand increased oversight. The driving force behind implementing RBM should be the improvements to data integrity and increased protection for human subjects. There have been companies that have seen cost savings over time, but not without financial hurdles along the way.
2. Engage the experts — and ask the tough questions.
There is clearly a lack of expertise in this arena, and the leading RBM vendors have emerged with both analytics modules and expanded service offerings to assist sponsors with administration and oversight of their risk-based trials. As with any vendor, it is important to consider the needs of your company and the capabilities of the vendor. Several technology vendors employ similar metrics using industry standard models and analytic approaches. Some vendors may also provide services such as key risk indicator analysis and more in-depth project analysis, depending on your budget. Having even a small amount of insight from these technology vendors could allow sponsors to begin building their RBM programs.
A disconnect between technology vendors and sponsor companies is that they don’t typically have professional service units whose sole business objective is to get the technology fully implemented — including getting clinical development teams up to speed. To clarify, someone who has “been there and done that” in the RBM implementation space should be available to troubleshoot until the motor is fully humming and everyone understands the critical details involved in designing, planning, operationalizing, and reporting.
If you are at vendor identification and selection, be careful about fully understanding the engagement in the beginning, middle, and end. Ask questions about the discovery process for the vendor, as well as what support services they have to ensure the implementation has reached a level whereby the product is now part of the company’s workflow.
3. Fully understand the multiyear business commitment involved.
One smaller pharma company in our discussion started with a pilot program where it developed an in-house approach to designing protocols that incorporated the RBM process, identified key risk indicators, then collected and measured them on several studies before expanding further. It is important to realize that performing RBM at a larger scale requires access to data and analytics that identify risks easily — and it is critical that the right skills, tools, and processes are in place to guide teams to build a portfolio of proven success. This can certainly be a great starting point, but be cautious of the undertaking it may require as your process evolves.
An example of this is to evaluate each stage of the effort so that an effective RBM solution will be part of a closed-loop quality by design (QbD) approach that supports adaptive monitoring through action and feedback mechanisms so historic site performance data is captured for future site selection decisions.
4. Learn how to develop and sell the business case for RBM and how to manage up.
A large and potentially difficult piece of creating a companywide culture around RBM is obtaining leadership buy-in. Leadership involvement and backing of the process will be crucial, as an up-front investment is needed. In order to secure that endorsement, your company executives need to understand the longer-term benefits that will be gained with the up-front investment.
Some of our clients have not had years of practice in developing an executive-level business case, and supporting these efforts will help to get the project off the ground. Because there is so little public data around the return on investment (ROI) on implementing RBM, there is a dearth of guidance on the business case for it. Few CROs will provide a compelling business case for it because their margins are not yet tied to successful trial outcomes, but rather to hours and tasks accomplished. Identifying some support to develop a business case, either through your peer network or a consultant who has done multiple implementations, will enable you to start the process. You will need to continue to collect and present data on the value of RBM, even after implementation, to highlight sustainable efficiency and value to your executive team.
5. Recognize the disconnect on planning for risks.
An important part of implementing RBM is getting your team to start thinking about risk as a holistic entity rather than a situational issue. Take the time to educate colleagues about the benefits of considering risk beforehand and how it adds another level of quality and preparedness to your operation. Working with your teams in planning meetings and helping them learn how to assess risk can both allow you to build quality into your programs and help the teams to identify what the mind-set is. If your team is not adept at the concepts and theories of risk identification and management, consider bringing in a program to train them.
Application of RBM should allow for a shift in focus from the smaller details of a clinical operation to a larger picture of quality, which allows for more effective utilization of resources. But all of our colleagues agree: looking at the big picture is sometimes hard for detail-oriented trial managers!
The Risk Assessment and Categorization Tool (RACT) was originally developed by TransCelerate to identify and assess risks.3 This is an excellent starting point to build a framework to align critical data/processes and specify the levels of involvement for site monitoring. Key risk indicators (KRIs), therapeutic and study-specific risk templates, and mitigation steps come later. The RACT was designed to identify how and by which functions the risks will be managed; document risk mitigations; categorize individual risks as high, medium, or low; and provide guidance to determine the baseline of monitoring activities, including suggestions for source document verification (SDV).
Taking a risk-based approach to your operation will promote more data-driven action and thus more efficiency. But learning how to assess risk holistically and being comfortable with building in quality parameters is the key to beginning to see the savings in cost and rework. Companies utilizing RBM should be able to demonstrate better audit results and identify site performance issues much sooner than before. Not every potential risk that is causing issues will be identified through this process, but in thinking about risk and developing mitigation strategies, companies may be able to soften the blow and react more quickly and efficiently when issues do arise.
6. Let go of things that don’t work anymore. (Be a champion of being lean and mean!)
Implementation of RBM will not completely change the way clinical operations are run, but it will help to eliminate some of the more inefficient aspects. Sponsors that have implemented an RBM process were able to decrease the amount of source data verification performed - which is a costly practice and generally does not provide valuable information. One big area of discussion really worth digging into is how the selection of SDV parameters should work in practice. Selecting a good site and overseeing it will always be a key factor in the success of your clinical operation, but RBM can assist in building a monitoring plan that is tailored to the needs of your sites. Eventually, you should be able to adjust the level of site management based on past performance and other indicators (i.e., data quality from the site, ease of contract negotiation, level of compliance difficulty) gathered through RBM. RBM will not replace monitoring completely, but it could allow for more centralized monitoring, which can shift focus and efforts to areas that need it the most.
There is no “right” way to implement risk-based monitoring, but that shouldn’t be viewed as a deterrent. As with any process, what works for some might not always work for others. There can be a benefit to navigating this gray area, working out the kinks, and creating a process customized to your program.
References:
- https://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM269919.pdf
- https://www.clinicalleader.com/doc/the-foundation-of-risk-based-monitoring-success-0001
- Position Paper: Risk-Based Monitoring Methodology. TransCelerate Biopharma Inc. Accessed at: http://www.transceleratebiopharmainc.com/wp-content/uploads/2013/10/TransCelerate-RBM-Position-Paper-FINAL-30MAY2013.pdf
About The Authors:
Laurie Halloran, BSN, MS, founded Halloran Consulting Group in 1998. Her time as a pediatric ICU nurse had inspired her to start a company that helps move new therapies through FDA processes to get them into the hands of patients desperately in need. Halloran has been named 2018 Women’s Business Enterprise Star by the Women's Business Enterprise National Council (WBENC) and has won a 2017 Enterprising Women of the Year Award from Enterprising Women magazine. Halloran Consulting Group was also named an Inc. 5000 Fastest Growing Company for the past three years in a row.
Megan Brickley has concentrated experience in quality assurance and clinical diagnostics focusing in immunology and infectious disease testing. At Halloran Consulting Group, she is responsible for writing and evaluating procedures, identifying and executing process improvements, and assisting with CAPA development and various audits. Previously, Brickley was employed by Oxford Immunotec, where she worked as a quality assurance specialist providing support across a diverse operations group that included diagnostics, device manufacturing, and research and development. Brickley was a team member responsible for obtaining FDA (CBER) licensure for a biologic used in the screening of donated blood units for the detection of tick-borne parasite Babesia microti.