The Pros And Cons Of Synthetic Control Arms In Clinical Trials
By Harriet Gray Stephens, MFPM, medical director, Boyds
Synthetic control arms (SCAs) are an innovative approach that is increasingly being adopted in clinical trials and a type of external control arm study. External controls have been defined by the FDA as any control group that is not a part of the same randomized study as the group receiving the investigational therapy. SCAs are generated using statistical methods applied to one or more data sources, such as the results of separate clinical trials or real-world data (RWD). The data is imputed to make it comparable to the intervention population within the clinical trial.
SCAs are especially useful when conducting traditional randomized controlled trials (RCTs) with a placebo or standard-of-care arm. While RCTs remain the gold standard for evaluating the safety and efficacy of new medical treatments, maintaining a concurrent control arm is sometimes unethical, impractical, or infeasible and can lead to increased patient burden and threaten the completion of a trial. Using an SCA provides supportive evidence, contextualizing the treatment effect and safety profile in situations where this information would not otherwise be available.
An SCA provides an alternative to uncontrolled or crossover clinical trials. Uncontrolled trials are commonly used in orphan diseases, where there is a shortage of patients, or very serious disease conditions without other treatments where there are scientific or ethical concerns about treatment switching or disease progression that would contraindicate the use of crossover trial designs. While uncontrolled trials can produce important safety and efficacy data, there is a significant risk of generating biased data because of a lack of randomization and a control arm to understand alternative treatment options or baseline disease states.
What Are The Ideal Data Sources For An SCA?
SCAs are constructed using patient-level data, obtained from patients not involved in the investigational clinical trial. Patients are “matched” using statistical or analytical methods to achieve balanced baseline features such as demographics and disease composition to generate an SCA that closely matches the experimentally treated patients. This enables a direct comparison of the SCA with the investigational arm. Sponsors should design the SCA early in protocol development and implement it when the clinical trial protocol has been finalized. They should initiate recruitment to enable the matching of patients while avoiding biases arising from manipulating the clinical trial protocol to match the data available to generate the SCA.
There is no formal restriction on where and in what format data comes, provided it meets the required quality criteria. Primarily, data arises from large data sets of historical clinical trials and real-world data (RWD). This is one of the key advantages of an SCA: By combining multiple data sources including historical literature comparisons, real-world data, and clinical trial data, organizations can generate excellent participant matching, develop more precise estimates of the comparison group outcome, and explore subgroup effects within the synthetic control population.
There are advantages and disadvantages of using clinical trial data compared to RWD. Clinical trial data is generally lower volume but highly standardized and has good quality. However, clinical trial data may not represent the patient population owing to recruitment biases including under-representation of certain ethnic, socio-economic, or age groups. RWD is higher volume data but is often composed of multiple sources and with worse standardization. This can make its use more difficult or more resource-intensive as more data processing is required to standardize the data. Additionally, RWD is more likely to have missing data, so organizations must carefully consider whether this makes that patient’s data set unusable or how to impute for minor missing data.
Once the appropriate data has been processed and data sources are selected, data matching of the synthetic control individuals with the investigational participants can occur. Multiple statistical methods exist to match the individuals including propensity scoring methods. There is significant regulatory interest in these methods. Indeed, selecting a method acceptable to regulatory authorities is key, and early regulatory discussions regarding these methods are recommended to achieve approvals.
What Regulators Think About SCAs
In 2001, the FDA accepted the use of external controls, where justified, to support regulatory decisions, stating in its latest guidance (February 2023) that external controls, such as SCAs, should be considered on a case-by-case basis. The EMA recognizes that SCAs can be a valuable tool when conducting conventional trials with placebo or standard-of-care arms presents a challenge, such as in rare diseases or when studying treatments for life-threatening conditions.
Early engagement with regulatory agencies is recommended by the FDA, EMA, and MHRA to ensure that the agencies can provide feedback on the appropriateness of any approach proposed. It is important to present to regulators the reasons why the proposed study design is appropriate, what data sources will be used for the SCA (and justification for their selection), the proposed statistical analyses, and how the data is going to be managed and submitted to the agency for review.
The need to reduce bias in SCAs is extensively emphasized by regulators. The FDA highlights the need for early consideration of SCA design during the early clinical trial protocol development stage to inform factors such as eligibility criteria and covariates. It is important to consider early on what data is available for the SCAs and what data sources will be used. The FDA recommends that sponsors document and describe in any associated study protocol all data sources that were accessed and provide information on why data sources are excluded. The FDA, EMA, and MHRA do not recommend a particular approach to analyzing data from externally controlled trials, as the agencies acknowledge that no single statistical or analytical method will be suitable for all trials involving external control arms. Therefore, when working with the FDA, sponsors should discuss suitable approaches with the appropriate review division.
Lower Costs, Quicker Timeframes Are Just Some Of The Advantages Of SCAs
SCAs bring advantages to both patients and sponsors. For patients, SCAs remove the possibility of patients being assigned to a placebo or to standard care, which can typically present ethical issues that may dissuade them from participating in a study. Not including placebo control can additionally deter patients from opting to withdraw from the trial if their disease deteriorates and cause them to seek alternative treatment options, such as therapies approved outside of the trial. It may also reduce the burden of disease assessments, as the sponsors may need less frequent assessments to determine disease progression.
For sponsors, SCA trials can make recruitment and retention easier, as participants are more willing to participate, and within the orphan disease space where patients are more difficult to find as these studies are less likely to use placebo control, or have fewer patients within the placebo control arm which can dissuade some individuals from participating in a study. Moreover, SCAs can be more cost-effective and time-efficient compared to traditional RCTs because they avoid all costs associated with the recruitment and conduct of clinical trials.
An area of significant advantage to sponsors is if an RCT has otherwise been compromised, for example, owing to accelerated approval meaning that all patients are eligible for therapy outside of an investigational product setting so are more likely to drop out, or regulatory authorities may mandate crossover to treatments. In this situation, an SCA can replace or complement the RCT trial data, enabling the sponsor to continue to meet their data obligations on safety and efficacy to the regulatory authorities.
The Limitations Of SCAs
The validity and applicability of SCAs are limited by the quality and quantity of data fed into their algorithms and the nature of the algorithm used. The quality of the historical data used to construct the SCA is crucial to its success. Unfortunately, a lot of RWD databases are privately owned, so sponsors need to carefully select the appropriate data solutions company with adequate data to formulate an SCA that is relevant to their clinical trial, and understand what the cost of using this data for their SCA will be. In areas where there is limited data, RWD database providers may charge a premium to access this information. The alternative for a sponsor would be to expend large amounts of money acquiring the required data, which is generally time and resource-prohibitive. Like the SCA itself, RWD utility is also limited by its formatting quality and completeness. Missing data can limit how the data can be evaluated for matching to the proposed clinical trial and utilized as a comparator arm.
The FDA also warns that SCAs can be a source of selection bias if the outcome of a clinical trial used as data for the SCA is already known but is inconsistent with prior experience. Selection bias refers to a systematic error in the way participants are recruited to, or assigned to different groups in a clinical trial leading to data contained within the RWD database that is not representative of the general population. This is particularly problematic when standard-of-care treatments change for a condition that drastically improves outcomes, such as the introduction of immunotherapy, such as checkpoint inhibitors in cancer. The RWD needs to be curated and updated to represent the new baseline outcomes, rather than the old normal, to ensure that the additional benefit of the investigational drug is being compared with the new outcome. This bias can distort the apparent effect of the treatment being tested, either underestimating or overestimating the true effect. This means that the SCA may not represent true population outcomes.
It is important to consider that an SCA may not fully capture the diversity of patient populations or reflect changes in the standard of care over time, potentially limiting the generalizability of trial results. This can be particularly problematic in situations where recent changes in standard of care treatment have been implemented, rendering large portions of historic data irrelevant and resulting in less long-term safety and efficacy data being available. It is vital to consider this when selecting the data for the synthetic control group by involving subject matter experts who are aware of changes in standard-of-care treatments and can engage carefully with regulatory agencies to discuss the proposed data to be used in the SCA.
Finally, despite growing acceptance, regulatory agencies may still approach trials with SCAs cautiously, requiring robust justification and validation of the methodology. Designing and implementing SCAs requires expertise in statistical methods and access to comprehensive data sources, which may pose challenges for some research teams. Multiple companies exist that specialize in SCAs; however, the use of this data requires careful financial and contractual considerations.
Moving Forward With SCAs
SCAs offer a promising alternative in certain clinical trial scenarios, providing a balance between ethical considerations, efficiency, and scientific rigor. However, sponsors must carefully consider early on the justification for its use, the availability of data to make an SCA, the SCA’s design, and statistical methods. Early regulatory engagement is critical to designing an acceptable SCA to complement other clinical trials.
References:
About The Author:
Harriet Gray Stephens MFPM is a Medical Director at Boyds. Working in the clinical and medical department, Harriet provides strategic support to clients on the medical aspects of pharmaceutical development, including regulatory and clinical development activities. With over five years of experience in the industry, she has completed specialty training with the Faculty of Pharmaceutical Medicine, holds an MBA from Cambridge University, and works in a range of disease areas specializing in advanced therapy medicinal products (including cell and gene therapies) and rare diseases.