Lessons Learned From A Venture Into Risk-Based Monitoring
By Ed Miseta, Chief Editor, Clinical Leader
Ruth Ann Subach enjoys talking about risk-based monitoring (RBM). As the director of clinical operations for Trevena, she has been involved with clinical trials since 1993 and began her career performing research as an oncology pharmacist with investigational drug responsibilities. The knowledge and experience gained from directly caring for patients gave her a great appreciation of the benefits received from needed therapies. It also instilled in her a life-long passion for drug development.
Last year she decided to undertake her first study incorporating RBM. According to Subach, it’s an approach that is trendy and has everyone talking. “RBM requires you to find the most important aspects of your study and get closer to them to make sure you are doing everything accurately,” she says. “It allows you to navigate your study and make changes in the protocol and the way it’s conducted. At the end of the day, it allows you to set and assess the right goals for your trial.”
Subach notes RBM is not so much about reducing risk by selecting the right things to monitor. It is more about understanding the negative outcomes of your study that might result from conducting the trial in an inappropriate manner. It also improves the likelihood that a company is going to have interpretable results from the study because monitors are making sure sites are conducting your actual protocol, not just going through the motions and doing what they see listed on a piece of paper.
Select The Right Partners
Trevena was started in late 2007. Subach was hired as a consultant in 2009 and became a full-time employee in 2010. At the time, the entire clinical team consisted of her and a physician. On the clinical side, the company worked with CROs, using their forms and following their SOPs. When she began to hear talk about RBM, she knew it was something she wanted to try.
“I felt I wanted to do something that was sexy, innovative, and fun in clinical operations,” she says. “I found a trial where I felt we could implement RBM and shopped it around to several CROs to determine their interest. I wanted to know if they had ever performed an RBM study and whether they were willing to do it.”
She selected a clinical CRO as well as a data and statistical CRO. Like Trevena, neither one had ever performed an RBM study, which made the effort a bit more interesting. Since this was new for all companies involved, nothing about RBM was included in any SOPs. That meant all of the personnel involved with the project had to collectively figure out how to make it happen.
“Our data management group also had to figure out how they were going to do this in the context of their own SOPs,” says Subach. “One of the first things we did was to inform study personnel that not everything they did would be monitored. Everything would have to be input to the database correctly, but we would not be clearing every little thing that they do.”
A Scary Situation
From the very first discussion she had with her CROs, Subach tried to gain an understanding of what the trial would look like. She notes the initial discussions were a bit scary for everyone involved because specific trial documents had to be designed to cover what was being done. Then the internal quality and monitoring groups had to go back, double check the regulations, look at the guidelines document, and figure out how the company would create something that was completely new.
“At every opportunity during our kick-off meeting, we took the time to revisit what we were doing,” she states. “A document was eventually created that was appended in the monitoring plan and used throughout the course of the trial. That document, an Excel spreadsheet, had columns for the data field descriptor, variable type, risk level (high, medium, and low), whether the data was available electronically, the percent of source document verification recommended, the rationale, source document verification, and any comments.”
Next, the team went through the protocol and the case report form, taking every variable in the trial and thinking about it in terms of efficacy, how difficult it would be to collect, and how difficult it would be to transcribe it from the source into a case report form. If a sample could be collected in millimeters or centimeters, for example, an assessment was made as to how difficult it would be to collect and how easy it might be to make an error.
“We thought about all of the different pieces of information, what they meant to the end result of the trial, and how the information would get from the patient’s mind or a piece of equipment into the database,” notes Subach. “Then we assigned a risk level based on all those factors.”
Once that task was complete, a decision had to be made as to whether 100 percent source document verification would be performed. Rules for data management were created, as well as triggers that would cause a return to 100 percent source data verification. Monitors would be notified if a certain field hit one of those triggers.
For example, a monitor may have had to source document verify data for a particular data point on 50 percent of the patients. If the sites were having difficulties entering the data, resulting in an increase in queries, that task would be bumped to 100 percent verification until the sites were able gather and record the information correctly.
The document was looked at constantly and modified as necessary as the trial progressed. The monitoring plan also noted the Excel spreadsheet could be modified without altering the rest of the plan. By doing so, the spreadsheet became a living document that could be modified throughout the course of the study. According to Subach, it was indeed modified three or four times throughout the trial.
Adjustments Were Necessary
One of the first things Subach learned was that some of the criteria for returning to 100 percent source document verification were too conservative. Some of the variables were hitting their triggers quite frequently, and monitors were performing source document verification on variables where it didn’t make sense based on the level of risk. In those situations the criteria were rewritten.
There were also tasks Subach believed would be difficult for sites to perform. That turned out to not be the case. The sites had a wealth of experience with the type of study being performed, and some anticipated problems never materialized. As a result, the risk level for those tasks was reduced along with the level of source data verification.
Subach notes this resulted in a huge time savings for the company. “Initially it was taking us three hours to get through a case. Over the course of the trial we were able to get that down to about two hours. That wound up being significant because we were able to get casebooks monitored a lot quicker, which resulted in us closing out the trial a week earlier than anticipated. When people think about the benefits from RBM, I think very few would consider time savings to be one of them, at least when you first implement it. But that was certainly the case for us.”
Careful protocol development was also a time saver. Because a lot of thought was put into protocol development upfront, it did not require multiple amendments during the study. Risk-based monitoring allowed Trevena to put more thought into the protocol upfront, which allowed the company to get it right and save time down the line. Together, careful protocol development and risk-based monitoring allowed Trevena to save time at the end of the study without compromising data quality.
One Down, Many More To Go
Subach mentions another benefit of RBM was what she was able to learn about the role of the monitor. “We discovered things during monitoring of the oversight process that we wanted monitors to do, and that we felt should be part of their role,” she adds. “We have diaries for some of the patient assignments during a study. The diaries are administered by personnel at the sites. We often wondered if the sites were performing them correctly, but our monitors did not observe the process to see what was being done. It wasn’t easy, but we were able to get the CROs to train the sites to go out and watch the coordinators administer these tests and do the work.”
Adopting RBM requires almost everyone involved to adjust to changes in the conduct of the trials. That certainly includes the sponsor, the CRO, and even the sites themselves. For most trials, the sites are used to 100 percent source document verification, which plays the role of a safety net. Knowing that net would not always be there caused angst for some of the sites. They got more comfortable with it as time went on, especially once they saw that the system was working.
With one RBM effort now under its belt, would Trevena consider doing it again? “Absolutely,” says Subach. “If you have a CRO partner you are working with on the study, once you perform one RBM study you know what it looks like and what needs to be done. If we were to perform a similar study, we could take the spreadsheet already developed and use that as a starting point. This will allow for further efficiencies to be gained in future trials. This is a different approach than what we have used on other studies. In comparing and contrasting them, we found we really like the approach. We found our CROs also liked it and are willing to do it again.”