Data Integrity In Clinical Trials: 4 Key Concerns Of The FDA & MHRA

By Betsy Fallen, BAFallen Consulting LLC

For the first time ever, GCP collaborations between the FDA and Medicines and Healthcare products Regulatory Agency (MHRA) were the focus of a workshop for the drug development industry. In what is projected to be the initial installment in a series, the FDA hosted the MHRA and invited interested individuals to join the workshop “Data Integrity in Global Clinical Trials — Are We There Yet?”¹ I was one of the fortunate 150 persons to be confirmed for in-person attendance on day one, October 23. The meeting was attended by many regulators, including 12 FDA speakers and three MHRA speakers, who were actively engaged and engaging. It was also shared virtually (and day one was recorded) with over 3,000 connections in 73 countries, making it the largest single-topic event ever hosted by the Center for Drug Evaluation and Research’s Small Business and Industry Assistance (SBIA) team. Day two focused on case studies that each workshop table was assigned to evaluate and practice mock inspection observations.

Over the course of the workshop, questions were collected continuously via paper notes from those in person and via webcast from virtual attendees. Time limited the opportunity for many responses, and I encourage the regulators to address those with significant impact or broad interest in follow-up messaging.

David Burrow, director of the office of scientific investigations at the CDER of the FDA, welcomed the audience to the workshop. He described his thoughts on the topic and his personal vision.

“Data integrity, it’s not black or white. It’s an incredible continuum. There are shades of gray. The shades of gray here are what drives the conversation. Those shades of gray right now, for the first time in this conversation, are things that we are going to start to paint in color. And that is my great hope for this conversation.”

The keynote speech was delivered by Robert Temple, M.D. His ability to recount his experiences and critique them against today’s environment with humor did not disappoint. Referencing a trial filed but not approved in April 1980, Temple mentioned that he and the FDA team learned many lessons that continue to be practiced today. It is evident that data integrity and the processes, systems, and best practices behind it have come a long way.

Impacts to Data Integrity

Several areas were identified as key topics that influence data integrity. These themes appeared throughout the workshop. They included unblinding, data management, vendor selection, and audit trails.

Unblinding

In the dark ages of working in paper environments, maintaining the blind was a manual effort of locking a paper in a drawer, if there was blinding at all. Temple’s recount of his experiences showed us how what might have otherwise been a promising investigational compound was mishandled. As a result of openly sharing assigned treatments, decisions on outcomes were biased by those interested in a positive result.

In these days of interactive response technology (IRT), the maintenance of the blind is managed systematically. The sequence of events, from assigning treatment to the final data analysis, can be reviewed on a timeline represented by the indisputable audit trail.

Over time, regulators have seen the process of unblinding management go from nonexistent to digitally developed. The adoption of systems, either directly by sponsors or by partners on their behalf, is reflective of the regulators’ requirement for system validation.

All stakeholders who are accountable for the treatment schedule of a blinded trial should pay particular attention to the enforcement of the blind. Systems that have material that may unblind someone should be validated with clearly defined roles that have been tested to assure integrity. IP management systems, data repositories, laboratory data, and any essential document that may reveal the treatment assignments must be managed by process and system to assure the blind. Following that, breaches of the blind must be raised and handled promptly. The culture of the organization must be willing to allow the recognition of errors, not punishment for reporting them when found.

Data Management

When I began my career in data management (pre-CDISC, just to provide context), the processes were very simple and the number of roles involved was limited. In our team of three we knew what each of us was responsible for. The data manager, statistician, and clinician were typically in the same building sharing paper copies of the case report forms (CRFs). Our plan could’ve fit on a sticky note.

Today’s data management is a global operation involving many organizations, likely dozens of individuals, and numerous systems to collect, transfer, and store the data. A plan that defines the roles and responsibilities with a focus on responsibility and accountability is critical to the integrity of the data. Access to and control of the content, from beginning to end, must be defined, and the clinical Investigator or their delegate must control the data. The workforce in this industry is extremely mobile, so this road map brings consistency to the end-to-end process.

At this workshop, regulatory inspectors noted that in their assignment of ensuring the integrity of the data, they look for this documentation. They expect this record to define how the data points are handled, who handles them, and the specifications. Their inspection is driven by the determination that the data management plan was followed and by evidence of that effort.

Any organization that has a role in data management should document their expectations. Clearly not all data points are considered equal in the scope of a trial, and triaging data points by importance should be defined in a risk-based plan. System specifications, data management SOPs, guidelines, and work instructions should be authored and managed to assure all stakeholders are aware and have relevant training on the data handling requirements. Control should be tracked and documented. One recommendation is that at the end of the trial, the electronic CRFs (eCRFs) be electronically generated and transferred to the clinical site on portable media (e.g., CD/DVD). That CD should never be under the control of the sponsor, so the content cannot be revised.

Vendor Selection

Selecting a vendor is a sophisticated effort in today’s environment of sponsor accountability and vendor responsibility under the umbrella of sponsor oversight. With the release of ICH E6(R2),² how that selection is executed and documentation of that process have come under increased focus.

During the selection process, many aspects and expectations need to be assessed. Examination of the vendor process and system documentation as well as related training will reveal how that vendor will protect data integrity. The process by which the sponsor will ensure the execution of these efforts and the documentation of both the efforts and the sponsor oversight will be critical to data integrity.

I recommend any vendor assessment process document (typically referred to as an RFP) consider these criteria in each category for assessment. Poor processes or ineffective system validation can be revealed, and vendors not prepared to meet the requirements can be avoided. Frequently, the contract includes the sponsor requirements, and this is encouraged by regulators.

Audit Trails

The value of an audit trail to a sponsor is the reflection of metrics such as progress to milestones, quality, and contemporaneousness. This is a significant advancement over paper and manual measurements. That value is also brought to the regulatory inspectors. Most (if not all) of the workshop presentations and case studies included a references to audit trails.

Stephen Vinter, operations manager for the MHRA’s Good Laboratory Practice Monitoring Authority (GLPMA), shared his expectations that audit trails be used to reflect problems by regulators but should also be used by sponsors, so they need to be easy to understand. He shared, “Sometimes we are given a name or some kind of code. So, if we cannot understand that, how can people who need to use it understand it as well?”

The audit trail details available to inspectors are considered to be of significant value to assess security, control, and potential for abuse, but the inability to interpret the details is challenging to an inspection. Codes need to be clear and human-readable by both the sponsor and the regulator; otherwise, it is not usable. Vinter also advised to avoid merged cells in an Excel spreadsheet, as they disrupt the audit trail.

The takeaway message is that audit trails should and will be used to reveal digital revisions and can ensure compliance or reveal discrepancies by anyone using a system. The caution is that it may reveal more than one expects. An inspection is not a good time for surprises.

Case Studies: Sharing the Pain

The second day of the workshop was promoted as an in-person, dynamic event. The activities and attendees did not disappoint. The eight- to 10-person tables were each assigned one of the four categories and provided with case studies developed from actual, although compiled and anonymized, situations and documentation. Typical scenarios were reviewed, and the table discussions were lively and revealing. In our case studies, the cryptic audit trails were provided, and the challenge was evident. In the scenario review, it was revealed that two of our tablemates were from the FDA. Their insight brought observations not readily recognized, and their participation was appreciated.

The readouts for each topic revealed inspection experiences most of the attendees would not have otherwise been exposed to. Interacting with regulators in a non-sponsor, non-product event offered the opportunity to ask and discuss focused questions which may be sensitive in other situations.

Where Are We Headed?

The key stops on this journey are to determine the plan for a clinical trial, the roles of those involved (both organizations and individuals), the processes, and the technology involved. Then the journey must be planned, documented, and monitored with the ultimate destination  mind: data integrity.

Future workshops sponsored by FDA CDER SBIA (Small Business and Industry Assistance)³ are continually published. I strongly encourage the FDA and MHRA to plan the next workshop and, again, take all of us on their journey.

References:

1. https://www.fda.gov/Drugs/DevelopmentApprovalProcess/SmallBusinessAssistance/ucm620340.htm
2. http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E6/E6_R2__Step_4_2016_1109.pdf
3. http://sbiaevents.com/

About The Author:

Betsy Fallen is an authority on the business processes and associated use of information technology in drug development. A passionate advocate for moving life sciences business online, she is an expert on many areas of drug development, focusing on document management. Trained as an RN, Fallen is dedicated to ensuring the voice of the patient is heard as the drug development process continues the progression toward innovation and efficiency. She can be reached at (610) 716-3271 or betsyfallen@bafconsult.com