Self-Deception — The 3 Most-Repeated Lies Of Trial Master File QC

By Thomas Cocciardi, LMK Clinical Research Consulting

Achieving inspection readiness means that during all stages of a clinical trial, a regulatory inspector would be able to walk into the building and reconstruct the trial using only the documents and metadata present in the trial master file (TMF). Although the concept of an inspection-ready TMF may be simply described, inspection readiness is not easily achieved. The failure to achieve an inspection-ready TMF continues to be an area of growing risk for the clinical research industry.

TMF quality control (QC) systems are the primary tool to control risk associated with the TMF. At its core, TMF QC is a system meant to continuously verify the current condition of the TMF in comparison to a predefined set of expectations. But, lost in competing priorities, true TMF QC may never be fully implemented. Without true TMF QC, the TMF exists as a dark cloud just at the horizon, waiting to become a full-blown storm as the trial approaches close-out or when the inspector appears at the door.

In other domains of the pharmaceutical industry, and even in other areas of the clinical space, procrastination and systematic noncompliance on a large scale would be irresponsible, unthinkable, and unacceptable. The TMF, however, is an exception. Across the industry, decision makers are willing to overlook the risks of a TMF lacking a quality system. How, in one of the most heavily regulated industries in the world, can such self-deception persist? How can we expect to achieve anything without first defining a pathway to success? The failure of a TMF may end with widespread consequences, but it often begins with a few well-repeated white lies:

“Yes, we do TMF QC.”

Any organization that maintains a TMF will claim to “do” TMF QC. After all, nobody would readily express the desire to be noncompliant or proudly proclaim the intention to do a bad job.

But too often, TMF QC takes the shape of a list. This list of documents (or more likely an Excel tracker) is a useful tool, but it is only an inventory. A document inventory, although a valid component of a TMF QC system, is much like a list of the items that are already in your grocery cart. Unfortunately, when at the grocery store, what is really needed is the list of items that haven’t made it into your cart yet.

TM QC is a system that moves with and against the flow of data (not just following the flow of documents from site to sponsor). It is a process of verification based on preordained expectations. The documents that are expected in the TMF are an important part of achieving the three Cs of TMF quality: complete, correct, and contemporaneous.

A true TMF QC cycle consists of the following essential components and extends far beyond a list of documents:

• Planning: The TMF has a clearly defined scope, a matrix of responsibilities, and prospective timelines.
• Quality Thresholds: As part of planning, clear error thresholds and pass/fail criteria are established.
• Equipment: There are tools in place to capture and facilitate the QC process.
• Continued Verification: QC is conducted continually based on the established expectations.
• Corrective Action: Items identified during QC are resolved in a timely manner and tracked to completion.

A document inventory, in a vacuum, is simply not sufficient to achieve the TMF inspection readiness that regulators expect. An inventory cannot forecast trends, create placeholders for missing documents, or effectively tell the story of the trial.

“It’s not my job to do TMF QC.”

Clear assignment and equitable distribution of TMF responsibilities is critical because no one person can “fix” a TMF. There is simply too much work to do. It is human nature to perform best with clear expectations and perform poorly without them. Because of this truth, the absence of a well-defined responsibility matrix is a common thread among dysfunctional TMFs.

In any given TMF, thousands of individuals across hundreds of locations may be stakeholders. Before the trial begins, the sites, CRO, and functional groups should all understand their role in ensuring the TMF tells the story of the trial. Project-specific expectations should be solidified and communicated. Those involved must also realize that quality, especially document-level quality, begins at the document’s creation. But it should also be made clear that document quality is only one measure of the health of a TMF. It must be the prerogative of TMF leaders to ensure stakeholders know their role and contribution to the TMF as a whole. Simply emailing somebody else an acceptable document is not the fulfillment of any obligation.

If you have experienced a TMF in crisis, you have witnessed the scramble to collect “everything” in advance of an impending deadline. This mad dash is usually accompanied by heavy assignment of blame — with each party pointing accusatively at the other. This last-minute collection process damages the fabric of the team and the reputation of all involved and pushes the sites toward unresponsiveness. If responsibility is not assigned at the outset of the trial, blame will be inevitably assigned in its place.

“My TMF is inspection ready.”

If you approach any study manager or TMF stakeholder in a supervisory capacity and asked him or her if their TMF was inspection ready, they would likely reply “yes.” But does that yes really mean “I hope so”? Without a complete cycle of TMF QC that establishes expectations and provides verification that expectations are met, “I hope so” is the best one can hope to achieve.

Inspection readiness starts by understanding the regulatory expectations for the TMF. We have already considered that the TMF should, at all times during the trial, be sufficient for an inspector or regulator to understand the story of the trial. From this basic expectation, study-specific or regional requirements can be generated.

Creating project or study-specific expectations that complement regulatory expectations requires establishing a desired level of quality. Establishing and maintaining this desired level of quality utilizes all aspects of the QC cycle: determining the scope, setting thresholds, establishing timelines, and ensuring that the proper equipment is in place.

Scope: What is the extent of our verification?

Will a percentage of documents, countries, or sites be reviewed? Perhaps a risk-based strategy will be adopted that identifies sites requiring extra attention based on criteria like enrollment, a PI change, or the creation of several FDA 1572s over a short period of time. A functional TMF QC system sets the expectation of what documents should be reviewed and aims to strike a balance between available resources and the desire for a certain level of quality.

Quality Thresholds: How do we measure success?

What is acceptable for one study may not be acceptable in the next. For example, one sponsor may reject a financial disclosure form using a nonstandard date format while others might not. An e-signature may be acceptable in one study where it would be rejected in another. But thresholds do not just exist on the document level. An acceptable rejection rate or filing time might be established. Leveraging an eTMF’s ability to produce metrics is a powerful way to achieve inspection readiness. Importantly, with every threshold should come an established consequence or escalation pathway. A quality threshold is useless if it does not help efficiently direct the TMF QC effort.

Timelines: When will QC be conducted?

Your eTMF system is on your side when it comes to timeliness. Timeliness metrics are an excellent foray into the prospective, data-driven TMF. But are you tracking the timeliness of your documents in the absence of a timeline for the TMF? As part of the comprehensive construction of a TMF, the timeline of the TMF (and its QC system) needs to be considered as much as the TMF’s structure. The TMF and its expectations change over the course of the trial and therefore the TMF QC plan must be tailored to fit. Beyond the scope and the desired level of quality, without a defined frequency and duration, a QC activity is not complete. After solidifying the “what, why, and how,” consider the when: whether based on the calendar, a study milestone (such as a protocol amendment), or phase-based (such as post SIV, or pre-close-out). But don’t forget that just setting the date is not enough. How long should the review take? And most importantly, how long will the team have to close the resulting action items?       

Equipment: What systems and tools will be used? Do they support our QC objectives?

You might have thought that this paragraph would be all about robust eTMF systems with new features. It, however, is not. Although a functional eTMF system is essential to a healthy TMF, the depth of its features pales in comparison to the comfort level and expertise of the individuals who use them.

Equipment is not always something that takes a username or password, or even a computer. Pay attention to how and where the TMF QC activity is being completed in the day-to-day operations of your team. Those sticky notes, informal checklists, forms, and Excel trackers are equipment, too. Properly inventorying all the equipment used is a great first step. Understanding limitations is often key. Each system, whether paper, Excel, SharePoint, or a modern TMF system, has pros and cons and exists on a spectrum of responsiveness and convenience. The best equipment may not always be the most complex, but whatever equipment is used should be inventoried, accessible, supported by training, and fully integrated into a larger plan.

To Tell The Truth

We all engage in some self-deception in managing the TMF. When not supported with the culture and resources necessary, the path to success is narrow. Still, this self-deception about the TMF (justified or not) harms us all and puts the primary goal of a trial at risk: helping patients. If we are to overcome the risk presented by an inadequate TMF (and in doing so achieve the profound benefits of a modern prospective TMF), then we need to be willing to peel back the exterior of the most common TMF QC lies we are all repeating.

After a QC cycle is established, TMF systems are implemented, the desired quality and scope defined, and the frequency of QC determined — what happens then? Too often, after hundreds or thousands of hours are poured into TMF QC, the cycle of QC remains incomplete. Action items remain open and the documents are not located or filed. Corrective action, fundamentally, is the true destiny of QC. The expectation of the QC cycle is not to observe a perfect TMF, avoid inconvenience, or to assign blame. Corrective action means fixing it and once again uniting and aligning expectation with reality. Yet for most, QC continues to be a surveying mission, performance art — or worse, an outright battle.

We’ve discussed the systems, the metrics, and processes that comprise a TMF, but the success or failure of a TMF and its QC systems will remain with the people behind the TMF. To achieve TMF greatness, the approach to TMF QC must be holistic and human-centered. Our TMF lies are well intentioned. We want to be our best, even if our resources are limited. We want to focus and celebrate what we excel at and minimize areas of conflict and struggle. But the truth of the matter is this: The backbone of the QC cycle is human relationships, and the foundation of your TMF expectations begins with the culture your team chooses to create.

About The Author:

Thomas Cocciardi is a technical writer at LMK Clinical Research Consulting who is committed to expressing the human story behind each trial master file (TMF). In addition to technical writing, he also works as a TMF-focused clinical trial associate for a niche pharmaceutical company. Cocciardi has gained wide-ranging experience, both in and out of the pharmaceutical industry, working as a TMF health specialist for a large CRO, clinical research coordinator, preclinical data coordinator, intern medical writer, and intern brand writer. He holds an M.S. in regulatory affairs for drugs, biologics, and medical devices from Northeastern University.