How To Spec, Purchase, And Implement The Right IRT System For Your Clinical Trial

By Tom Schiavon

During a study startup phase, interactive response technology (IRT) system development can be difficult for clinical teams to navigate successfully, especially when team members have limited experience building or using the technology. In this two-part article series, I share some of the lessons I’ve learned through various roles I’ve held in the IRT industry, thus empowering clinical team members to make better decisions when it comes to this crucial technology. In Part 1, we looked at what an IRT system is and when you will need it. In this second part, we take a deeper dive into key considerations when selecting a vendor, strategies to implement during the IRT design phase, and best practices for utilizing the system throughout the life of your trial.

Vendor Selection Considerations

Anyone who has participated in vendor selection knows it can be a harrowing process. Qualifying a vendor can be long and difficult, and even when due diligence is exercised, vendors sometimes underperform. Below are a few items from my experience working at a sponsor and with dozens of sponsors from the vendor side.

Compliance

• Audit Capabilities: It’s expected that an IRT vendor comply with 21 CFR Part 11 audit trail requirements. However, the format in which that data is stored, and the speed and readability with which the vendor can provide that information varies. Make sure the vendor has the ability to quickly access this information, can provide it if needed for an audit or internal review, and that the format allows an auditor to easily trace the data from its first occurrence through any updates. The FDA expects both the data and metadata that provides context to provide a clear chronology and explain “why” a change was made.¹

• Training: Access to and training with software are additional areas that may attract scrutiny, and patient privacy and control over their data has introduced new considerations into clinical trial software.² Ask the vendor what types of tools it can provide to show an audit trail of system access.

A “sandbox” training environment is also exceptionally helpful when training monitors and sites on system functionality. If a vendor provides access to a dummy data environment, users can become acquainted with system functionality at site initiation visits or when new staff members join a site.

Development and Testing

• Configurability vs. Programming: For much of the history of IRT, systems were coded line-by-line, and any updates to the system involved ensuring every related piece of coding was updated by a developer. Within the past decade, IRT systems have made use of prevalidated modules to speed delivery of systems and reduce risk. This approach uses “building blocks” of standard transaction elements as a starting point, customizing pieces as needed. It also allows a designer to configure a similar data point across an entire study through a user interface. The result is a higher level of quality, consistency, and trust in systems.

When validating a vendor, ask what percentage of their average system is configured. The closer their answer is to 100 percent, the more robust their configuration platform is.

• Startup and Amendment Timelines: Related to the above point, it’s important to understand how long an average system will take to deploy. A protocol’s complexity will have an impact on timelines, but a vendor should be able to commit to a standard system development schedule. If a vendor promises all systems can be developed within a set time frame, have them clarify how they will accommodate the additional effort within the standard timeline. A highly complex system or one that includes significant customizations will take additional time to develop and test, and compressing these requirements could result in a drop in quality.

• Risk-Based Validation Approach: As systems become more robust, more functions must be developed and tested. The industry has begun to embrace a risk-based approach to validation as a way to ensure quality and timeliness. With many functions being prevalidated, it becomes easier for teams to identify the high-risk items and focus testing in these areas. Ensure your vendor has a methodology for assessing risk and providing sufficient validation of any items that may impact patient safety or data integrity.

• One-Click Integrations: Most IRT systems integrate with another software: EDC, CTMS, depots, ePRO, labs, or proprietary software. Many vendors now allow one-click integrations, allowing the designer to incorporate standard integrations into the study without the need for custom programming.

Design Considerations And Strategies

When I worked for an IRT vendor, it was a proverb among service teams that we could build anything for which the client was willing to pay. Most IRT vendors can create custom functionality that will suit the needs of a protocol, but often smart choices about design preclude the need to make complex customizations to the base system.

Some considerations that can significantly impact study design are:

• What is the “source of truth?” Many sponsors have adopted a stance toward IRT that it shows the “intended” action, whereas EDC is the system that tracks what “actually” happened. Using EDC for reporting and data analysis can reduce the need to reconcile data between multiple systems.

• Is the data being collected elsewhere? If a sponsor’s preference is to reconcile data between systems, it’s useful to ask whether the data is being captured in some other system. I’ve worked with sponsors that wanted to collect a range of data in IRT, some of which was also entered in another system. When it came time to reconcile data during an interim or final analysis, study managers, data managers, monitors, and site personnel were involved to ensure data matched across the systems. Limiting the data collected in IRT to that which is needed for screening, randomization, or dosing can eliminate unnecessary operational impact.

• How will you track study operations? When I first began building requirements as a project manager, it was easy to assume that standard reporting configurations would be sufficient for sponsor needs. After building a few studies and needing to do an amendment not long after the initial study launched, I realized simple customizations could make sure study teams had the data they needed to make operational decisions. Will a certain country need to be monitored to ensure enrollment doesn’t go above a certain limit or percentage of total enrollments? Is there a stratification factor that will be especially important to track? Do summary reports need to include regional groupings? Speaking with data management, supplies, the biostatistician, and your CRO can prevent post-go-live amendments to your system.

• Always have a manual backup. Complex functionality that is essential for transactions to be processed should always have a manual administrative option. I managed a support and data management team for nearly two years, and during that time I participated in many hurried and harried calls where a failing function prevented a randomization or patient visit. This was most often the case with lab integrations that were used in randomization algorithms, usually to prevent unblinding. The best-designed studies included an administrative function where the data could be entered manually and allow the transaction to proceed.

• Avoid low compliance functions. Requiring unnecessary visits or functions that are known to be low compliance to be used in the IRT system  is a recipe for frustrating site staff and having incomplete data in the IRT system. If an item on the schedule of activities can be avoided without impact, it’s best to exclude it from the IRT.

In each system, biology’s Principle of Parsimony³ is a useful corollary for IRT design. Keeping a system as simple as needed to fulfill protocol and operational needs will yield a tool that is the easiest to use, least prone to error, and most economical.

IRT Operational Best Practices

Launching an IRT system is the easy part. Once it’s live, you may be using it for several years. It’s important to see your vendor as a partner and develop a plan for managing the study to its conclusion.

IRT vendors will be supporting your study 24/7/365, through every global holiday. Make sure you have clear guidelines around unblinding data points, acceptable data changes, and contacts for approving the unusual situations that invariably arise during the operation of a clinical trial.

Provide thorough examples of what your team expects from the vendor in particular circumstances. Create standards regarding:

• How to treat open label and blinded medication when a subject fails to take the dose dispensed by the IRT system
• What data points may be updated (e.g., date/year of birth, age, sex/gender) and what data points may never be updated (e.g., randomization treatment or date, stratification factors, re-randomization visit details)
• Requesting unblinded data updates, including who can approve changes

Having items such as the above detailed in a study wiki page or communication plan ensures the vendor is handling your data and study operations as you wish and reduces the number of emails or calls your team will receive seeking clarification.

Conclusion

Clinical teams have important choices to make when choosing an IRT vendor and translating a protocol into a system. These choices can have a significant impact on patient safety, data integrity, and site compliance, so an improved understanding of the technology empowers teams to make informed decisions. Remembering that each system is a tool that supports study operations can make decisions more simple, and a parsimonious approach to design can make systems which are more efficient, stable, and cost-effective while achieving study endpoints.

References:

1. https://www.fda.gov/downloads/AboutFDA/CentersOffices/OfficeofMedicalProductsandTobacco/CDER/UCM561491.pdf
2. https://www.thompsoncoburn.com/insights/blogs/life-sciences-decoded/post/2017-11-06/clinical-trials-part-ii-privacy-cybersecurity-risks-and-managing-ephi
3. http://www.oxfordreference.com/view/10.1093/oi/authority.20110803100346221

About The Author:

Tom Schiavon, M.A. is an experienced project manager who enjoys exploring the intersection of theory and practice to innovate or improve processes. He gained an understanding of supplies management while serving as an IRT project manager and manager of technical support and data management teams. He obtained his M.A. in English from Florida Gulf Coast University, where he also graduated summa cum laude as an undergraduate. A humanities wonk by nature, he enjoys contributing to the development of novel medications aimed at easing human suffering.