Looking Beyond The Pandemic: Ensuring Data Integrity Post-COVID-19

By Todd Johnson, Patricia Davis, and Caroline Alix, Halloran Consulting Group, Inc.

The impact of COVID-19 has become clear on a more granular level as we continue conducting clinical trials in the midst of the pandemic. Importantly, trials that have continued are now producing data, and sponsors are having to adapt analysis and reporting strategies to fit their current circumstances. Data integrity is paramount now more than ever, and it’s imperative that industry keeps this top of mind.

Halloran’s latest COVID-19 Virtual Town Hall convened a panel of clinical and statistical experts to provide insight on how sponsor companies should approach documentation, analysis, and reporting in the COVID-19 era. Discussion centered around COVID-19 impact assessments, mitigations, monitoring visits, and documenting protocol deviations appropriately to ensure data integrity. The following strategies and processes are currently being utilized by key players in the industry during the pandemic and provide insight into how to best proceed in the future.

Assessing The Impact Of COVID-19

Performing an impact assessment has proven effective for focusing future efforts in the right direction. At the beginning of the pandemic, some pharmaceutical companies performed large impact assessments, with the goal of identifying studies and sites that could pause with the least disruption. Initially, the impact assessment was an enrollment issue, heavily focused on recruitment obstacles at the study level. As the pandemic progresses, impact assessments are becoming more detailed and focusing on number of “affected patients” (regardless of whether the study was or was not paused). The next wave of impact assessments should focus on data, specifically cleaning, management, and analysis. This effort will inform proactive mitigation efforts.

Assessing the impact of data will be protocol-specific and will be a balance between how much treatment a patient received and the endpoints of the study.  The assessment will require: 1) identifying how many patients do not have enough data to assess efficacy due to missed visits or other factors, and 2) determining if the duration of a patient’s treatment meets the protocol specific requirements to be deemed evaluable, if there is a necessary interruption or delay in their treatment due to COVID-19, and if the patient will be allowed to restart treatment or will be replaced to ensure adequate data for analysis.

Impact Mitigation

Mitigation strategies should be twofold: 1) mitigating the risk of patients not getting treatment, and 2) mitigating the risk of being unable to assess the patient. Extending visit windows or adding in additional visits are feasible mitigations (consider additional methodologies according to study goals). Sites should explore the possibility of local infusions or mailing oral investigational product to patient homes. It is important to utilize all available opportunities to keep the patient on treatment, always prioritizing patient safety. This should include offering training to caretakers and authorized personnel on how to properly conduct modified visits and assisting them in learning the new processes.

Mitigation discussions and decisions will need to be made on a protocol-by-protocol basis and should include a multidisciplinary team. Finally, ethics in these decisions are paramount, and patient safety should always be the first concern.

Utilizing Monitoring

A flexible monitoring strategy is key to mitigating study-level impact of COVID-19 and ensuring data integrity. Options for monitoring include:

Remote Monitoring

Especially with smaller sites that have less infrastructure, employing interview techniques can prove especially helpful in taking a pulse on site activities during COVID-19. Treat each site as a separate case. Ask coordinators for patient updates and safety updates. Leverage your relationship with sites now more than ever and support them however possible.

There has been a decrease in reported adverse events(AEs) during the pandemic, likely due to patients’ reluctance to visit sites in person so sites should be communicating regularly with the patients to stay close to understanding if any AEs need to be reported.

Flexibility in timing is critical. Sponsors noted that instead of conducting a virtual remote monitoring visit for 8 hours per day over one to two days, they are now allowing monitoring visits to be conducted in 2- to 3-hour periods of time spread out across the week. Both monitors and coordinators have reported this method to be effective and even preferable going forward.

Consider also conducting site initiation visits (SIVs) virtually. Sponsors note that virtual SIVs both before and during COVID-19 have proven successful.

Use lessons learned from remote monitoring during COVID-19 to inform future monitoring strategies.  Sites are becoming very familiar with and accepting of this method as we move into the future.

Centralized Monitoring

Consider employing or expanding centralized monitoring to identify additional data integrity issues and focus on critical data. Companies are now more than ever moving to a centralized monitoring approach.  The use and analysis of data to trigger monitoring visits and identify potential signals early is becoming a more widely used method across industry due to this pandemic.  This approach, however, requires almost real-time access to the data, understanding the appropriate KPIs and thresholds (which will likely require adjustments now under COVID conditions), and a dedicated internal team and process with the appropriate skillsets and training to employ these techniques.  

Centralized monitoring offers a more comprehensive view at the data, focused on finding trends, including trends across a portfolio.  Insights from centralized monitoring allow sponsors to determine areas of focus for monitoring and follow-up, identify training opportunities, and may even trigger protocol amendments when needed.

Consider conducting a gap analysis now to determine if processes implemented during the pandemic have been and continue to be effective moving into the future. Frequently reassess KPIs, thresholds, and quality tolerance limits (QTLs) for possible adjustments based on review of collected data. Review, in a risk-based manner, some of the remote monitoring and centralized monitoring findings, and modify the monitoring processes and strategies accordingly. Stay in close contact with monitors and sites. Ask how their monitoring visits went and if there was any feedback from coordinators or PIs. When on-site patient visits resume, compare protocol deviations that occurred during COVID-19 with the deviations that occur afterward. This should drive the on-site monitoring visits moving forward.

Data Integrity

Among our panelists, the most prevalent data integrity challenge during COVID-19 has been missing data due to missed visits. As we continue to move to converted or modified visits, coordinators are dealing with the increased workload by contacting patients, collecting as much data as possible despite missed visits, and entering data as best they can into the EDC.  Sponsors need to ensure that data entry occurs in a manner that will provide useful, meaningful information in the future.

Capture Protocol Deviations Appropriately

Not surprisingly, protocol deviations have increased during COVID-19, but the industry lacks consensus around documenting them. Differentiating COVID-related deviations from non-COVID related deviations is not necessary, and sponsors have found success in reporting both in the same fashion. Instead, monitors specifically state in visit reports that the patient visit was conducted during COVID-19. Because of the way deviations are written up, there should be clear attribution to COVID-19 if possible and if necessary.

One sponsor, with FDA support, created an overarching Note to File with a start and end date to address protocol deviations.  The note stated that in these unprecedented circumstances, between X and Y dates, the sponsor may deviate from the original monitoring plan or protocol but that the deviation will be driven by ensuring patient safety. 

• This overarching Note to File allows the sponsor to deploy processes that lie outside the monitoring plan without needing to update each individual plan.
• Each individual deviation should still be captured in monitoring visit reports or in EDC (as applicable), but the Note to File gives the sponsor permission to be flexible in adjusting monitoring approaches as needed.
• If, when typical study conduct resumes, monitoring is still being done differently than prescribed in the original monitoring plans, those individual plans should be updated to reflect the change.

Consider using a standard data collection method to capture information related to COVID-19 in one central place, such as adding uniform COVID-19-specific data fields to case report forms to ensure consistency across studies.

Before, During, And After: Evaluating Site-By-Site Differences

Accurately collecting and reporting data affected by COVID-19 means defining “before,” “during,” and “after” timepoints. Identifying what timeline constitutes before, during, or after is challenging due to the virus’ regional, local, and site-related variations. The recommended approach is to bookend the pandemic in the context of the trial.  Consider setting dates at appropriate inflection points such as the earliest date of COVID-19’s trial impact, the date when patients assessments could resume, etc. Attention will also need to be paid to what exactly was influenced at these time points. Sponsors should work with each site to confirm their impact dates (i.e. site closure, enrollment hold, visit conversion, etc.) as each site may have implemented precautions before regional impact was known. Sponsors should use these full timelines to document the parameters around the data and for guidance on coding needs to prepare for data analysis.

Regulatory authorities will need to see detailed data on a site level, including activity pause/resume dates. Sponsors must understand the effect of COVID-19 on a patient level, specifically concerning protocol deviations, identifying which deviations were due to COVID-19 and which were true site deviations, if possible. Sponsors should still report protocol deviations via their normal process but should also be building and planning a strategy together with their statisticians for how this information will be reported out at the end of the trial.  This will ensure they are capturing and collecting the right level of data to ensure data integrity into the future.

When To Intervene

Sponsors should perform a blinded summary, which is an assessment detailing how many patients they have lost, protocol deviations that could have impacted the evaluable population, any gaps in the data, etc. Ideally, if the study is still ongoing, sponsors should perform this summary before the trial stops recruiting, in case the sponsor decides that additional patients will need to be enrolled. Performing blinded reviews now will help sponsors assess questions such as “when a trial’s enrollment is nearly complete and then recruitment is paused, is the current enrollment sufficient to close out the trial?” or “when a sponsor is about to do an interim analysis and is only a few patients short, and then recruitment is paused and/or extremely delayed due to COVID-19, should the sponsor conduct an early interim analysis?”

Though most areas of the United States are still in the throes of COVID-19, it’s time for sponsors to begin looking beyond the pandemic, focusing attention on strategies for assessing the pandemic’s impact on data integrity, tactics for impact mitigation, and methods to ensure overall data integrity. Flexibility and forethought will be key as we aim to set our studies up with meaningful data for future success both during and after the pandemic.  There is a lot to be learned from this pandemic and we are hopeful that sharing some of this information will help others as they continue to gear up their studies and plans.

If you would like to be included in future Halloran Virtual Town Halls on COVID-19, please send an email request to halloranhelps@hallorancg.com.

Previous Articles In This Series:

• “Post-COVID-19 Clinical Trials: Finally Understanding The Meaning Of Patient-Centricity”
• “Developing A Clinical Trials Strategy For A Post-COVID World”
• “Clinical Trials At Home — Implementing Telemedicine & Virtual Visits During COVID-19”
• “What To Expect From The FDA During (And After) COVID-19”
• “Adjusting To Clinical Trial Remote Monitoring During COVID-19: Working Smarter & Safer”
• “Clinical Study Risk Management During The COVID-19 Pandemic”
• “Clinical Study Conduct During The COVID-19 Pandemic — Challenges & Solutions”
• “Assessing The Impact Of COVID-19 On Regulatory Interactions, Inspections, & Audits”
• “8 Significant Risks COVID-19 Poses To Clinical Trials — And How To Mitigate Them”

About The Author:

Todd Johnson, principal consultant at Halloran Consulting Group, has more than 20 years of experience in pharmaceutical development and clinical systems — from beginning his career as a clinical site monitor to developing operational performance dashboards and KPI/data reporting strategies. Having successfully managed clinical programs and assets, Johnson is knowledgeable in ICH/GCP, has clinical operations experience across multiple therapeutic areas, and has assisted in several successful IND and NDA submissions. Before joining Halloran, he led the Clinical Information and Informatics groups at Astellas and PPD, with an emphasis on operational performance metric/KPI assessment, data quality, benchmarking, dashboard development and reporting.

Patricia Davis, senior consultant at Halloran, has more than 19 years of clinical experience and 14 years of clinical research involvement with a focus in clinical operations and early phase oncology trials. She has direct experience in the areas of clinical monitoring, project management, and line management of multiple departments. Additionally, Davis has experience in leading new initiatives, departmental restructuring, and defining and refining process efficiencies. Prior to joining Halloran, She most recently worked for a mid-sized CRO, MedSource, as a project manager for early phase oncology trials. She is a licensed practical nurse since 2000.

Caroline Alix, associate consultant at Halloran, has experience in cross-functional project management and clinical operations, specifically within the oncology, immunology, and rare disease spaces. She has served as the clinical trial assistant and assistant project manager for a Phase 1 healthy volunteer study, and multiple global oncology trials, all from the start-up stage. Alix has supported and managed multiple aspects of clinical trials including protocol development, selection and oversight of CROs and vendors, oversight of timelines, and risk management. Prior to joining Halloran, she served as a protocol care admin specialist at Dana Farber Cancer Institute.