FDA Seeks Comment On Fit-For-Purpose COAs Guidance

On June 29, 2022, the FDA’s Center for Drug Evaluation and Research (CDER), Center for Biologics Evaluation, and Research (CBER), and Center for Devices and Radiological Health (CDRH) released for public comment Patient-Focused Drug Development: Selecting, Developing, or Modifying Fit-for-Purpose Clinical Outcome Assessments for industry, FDA staff, and other stakeholders.

This guidance is the third in a series of four methodological patient-focused drug development (PFDD) guidance documents that describe how patients, caregivers, researchers, medical product developers, and others can collect and submit patient experience data and other relevant information from patients and caregivers to be used for medical product development and regulatory decision-making. The four guidances are as follows:

• Guidance 1: Patient-Focused Drug Development: Collecting Comprehensive and Representative Input provides methods to collect patient experience data that are accurate and representative of the intended patient population.
• Guidance 2: Patient-Focused Drug Development: Methods to Identify What Is Important to Patients discusses approaches to identifying what is most important to patients with respect to their experience as it relates to burden of disease/condition and burden of treatment.
• Guidance 3: Patient-Focused Drug Development: Selecting, Developing, or Modifying Fit-for-Purpose Clinical Outcome Assessments discusses approaches to selecting, modifying, developing, and validating clinical outcome assessments (COAs) to measure outcomes of importance to patients in clinical trials.
• Guidance 4: Incorporating Clinical Outcome Assessments into Endpoints for Regulatory Decision Making will provide methods, standards, and technologies to collect and analyze COA data for regulatory decision-making, including selecting the COA-based endpoint and determining clinically meaningful change in that endpoint.

The purpose of the guidance is “to help sponsors use high quality measures of patients’ health in medical product development programs” to ensure matters important to patients are appropriately measured and provide meaningful information for medical product development and regulatory decision-making.

Clinical Outcome Assessments (COAs)

A COA is a measure that describes or reflects how a patient feels, functions, or survives. COA scores are used to support efficacy, effectiveness, and safety to determine the clinical benefit(s) and risks(s) of a medical product. There are four types of COAs the FDA prefers: patient-reported outcomes (PROs), observer-reported outcomes (ObsROs), clinician-reported outcomes (ClinROs), and performance-based outcomes (PerfOs).

Proxy-reported outcome measures are assessments in which someone other than the patient reports on patient experiences as if the individual were the patient. Proxy-reported outcome measures are discouraged by the FDA.

Measurements based on reports that come directly from the patient about the status of their health condition without interpretation of the responses by others is referred to as a PRO. PROs are used when the patient is able to provide reliable self-reported feelings or experiences known only to the patient, such as functioning or activity that is part of the patient’s day-to-day life, satisfaction or dissatisfaction with their treatment and/or functioning, and the impact on day-to-day life associated with one or more symptoms. Figure 1 provides a conceptual framework for concepts of interest assessed by two PROs.

Figure 1: Conceptual Framework for Concepts of Interest Assessed by Two PROs, per the guidance.

Assessments with observable signs, events, or behaviors related to a patient’s health condition that are reported by someone other than the patient or a healthcare professional are classified as ObsROs. ObsROs are reported by parents, caregivers, or someone who cares for the patient the most or spends significant time with the patient during the study. ObsRO measurements do not rely on medical judgment or interpretation by a healthcare professional. Figure 2 depicts a conceptual framework for a concept of interest assessed by a multi-item ObsRO.

Figure 2: Conceptual Framework for a Concept of Interest Assessed by a Multi-Item ObsRO, per the guidance.

ClinRO measurements are used when a clinical judgment by a healthcare professional is required to evaluate the patient’s condition. ClinROs include reports of clinical signs, ratings of a sign, and the healthcare professional’s assessment of the patient’s status. Figure 3 depicts a conceptual framework for a concept of interest assessed by a ClinRO.

Figure 3: Conceptual Framework for a Concept of Interest Assessed by a ClinRO, per the guidance.

Patient experience data that are best captured through performance tasks (physical/cognitive functioning or perceptual/sensory functioning) are referred to as PerfOs. PerfOs rely on standardized task(s) performed by a patient that can be administered and evaluated by an individual who is appropriately trained or are independently completed by the patient. The patient’s performance is then quantified and reported. Figure 4 depicts a conceptual framework for measures of two concepts of interest with indirect relationships to the patient’s specific health experiences for a PerfO assessment.

Figure 4: Conceptual Framework for Measures of Two Concepts of Interest With Indirect Relationships to the Patient’s Specific Health Experiences for a PerfO Assessment, per the guidance.

Concept Of Interest And Context Of Use

The sponsor’s proposal is required to reference the concept of interest, the context of use, a scoring system, and how the scores will be interpreted for the clinical study.

The concept of interest is the aspect of the patient’s experience, including clinical, biological, physical, or functional condition, the assessment is intended to capture. Treatments may improve a symptom(s) or specific function, avoid further worsening of a symptom(s) or further loss of a specific function, or prevent the onset of a symptom or a loss of a specific function. It is important to carefully select concepts in a clinical trial that, when measured properly, reflect an aspect of health that is important to patients, has the ability to be modified by the investigational treatment, and could demonstrate clinically meaningful differences within the planned time frame.

Many concepts of interest can be associated with a single disease or condition; selection of the concept(s) of interest and the aspect(s) of the concept(s) of interest are important for effective COAs. Sponsors should identify the primary manifestations of a disease or condition to concentrate on during medical product development.  The aspect(s) of a concept of interest most impactful for patients can often be identified by the patient and/or caregiver. Patient and/or caregiver input helps sponsors select or develop COAs that measure what is important to patients.

Conceptual models are useful for representing patients’ specific health experiences resulting from their disease/condition, the health concepts that describe those specific experiences, and the concept(s) of interest selected for assessment. Figure 5 depicts a conceptual model for activities of daily living. Conceptual models can be helpful for communicating with the FDA about the concept to be measured and determining whether a proposed COA fully captures the concept of interest.

Figure 5: Conceptual Model for Activities of Daily Living, per the guidance.

The context of use should clearly specify the way COA scores will be used, including the purpose for their use in a medical product development program. The appropriateness of a COA is evaluated within the proposed context of use. Context of use considerations include use of the COA, target population, study context, timing, and COA implementation.

A COA is considered fit-for-purpose when “the level of validation associated with a medical product development tool is sufficient to support its context of use.”  COA fit-for-purpose decisions are based on two considerations: First, are the concept of interest and context of use clearly described? And second, is there sufficient evidence to support the rationale for the proposed interpretation and use of the COA?

The guidance provides the following eight components comprising an evidence-based rationale for proposing a COA as fit-for-purpose:

1. The concept of interest should be assessed by [COA type] because . . .
2. The COA measure selected captures all the important aspects of the concept of interest.
3. Respondents understand the instructions and items/tasks of the measure as intended by the measure developer.
4. Scores of the COA are not overly influenced by processes/concepts that are not part of the concept of interest.
5. The method of scoring responses to the COA is appropriate for assessing the concept of interest.
6. Scores from the COA correspond to the specific health experience(s) the patient has related to the concept of interest.
7. Scores are sufficiently sensitive to reflect clinically meaningful changes within patients over time in the concept of interest within the context of use.
8. Differences in COA scores can be interpreted and communicated clearly in terms of the expected impact on patients’ experiences.

Patient-Focused Outcome Measurement

To ensure COAs are appropriate for the intended context of use, The FDA recommends sponsors seek FDA input pertaining patient-focused outcome measurements as early as possible and throughout medical product development. Figure 6 depicts a road map for patient-focused outcome measurement in clinical trials.

Figure 6: Road Map to Patient-Focused Outcome Measurement in Clinical Trials, per the guidance.

Understanding the disease or condition is the first step in developing COAs. COAs must identify patient subpopulations, the clinical environment, patient and/or caregiver perspectives, and the therapeutic requirements. Conceptualizing clinical benefits and risks involves considering which aspect(s) of the patient’s experience with the disease/condition and/or its treatment will be targeted by the medical product.

The steps involved in selecting or developing a COA to measure the concept of interest include selecting one of the four types of COAs (PROs, ObsROs, ClinROs, or PerfOs), evaluating the existing and available COAs measuring the concept of interest in the context of use, special considerations for selecting or developing COAs for pediatric populations, using digital health technologies (DHTs) to collect COA data, and COA accessibility and universal design (accessed, understood, and used by all people, inclusive of people with disabilities).

The conceptual framework summarizes relevant experiences of patients in the target population, specific concepts of interest targeted for assessment, type(s) of COA proposed for each concept of interest, and a representation of how the particular COA is intended to work in order to generate a score reflecting the concept of interest. The conceptual framework includes the conceptual model and the measurement model. Figure 7 depicts a generic conceptual framework summarizing which patient experiences will be targeted and how they will be measured.

Figure 7: Generic Conceptual Framework Summarizing Which Patient Experiences Will Be Targeted and How They Will Be Measured, per the guidance.

Measurement Models

Item response theory (IRT) models are used to evaluate the relationships between a concept of interest that cannot be measured directly (i.e., level of function, activity, satisfaction, or dissatisfaction) and the items planned to measure the trait.

The guidance refers to two IRT models that are useful for attributes that cannot be measured directly, the reflective indicator (RI) model and the composite indicator (CI) model (note: some sources will refer to this as a formative indicator [FI]). Both models utilize items (indicators) to analyze the concept of interest. There is a directional relationship between the items and the concept of interest.

The RI model (arrows from the concept to the item) suggests the assumption that the concept of interest causes the measurement of the items, signifying that a change in the concept of interest will cause a change in the items. Figure 8 is an example of a reflective indicator model. CI models (arrows from the item to the concept of interest) suggest the assumption that the items cause the concept of interest, signifying that a change in at least one item will cause a change in the concept of interest. Figure 9 is an example of a composite indicator model.

Figure 8: Reflective Indicator Model, modified from the guidance for clarity.

Figure 9: Composite Indicator Model, modified from the guidance for clarity.

Conclusion

This guidance helps sponsors determine the best method to measure COAs. Ensuring the correct measurement is utilized is crucial for collecting meaningful measurements and submitting patient experience data used for medical product development and regulatory decision-making.

Submit written comments by Sept. 28, 2022, to the Dockets Management Staff, Food and Drug Administration, 5630 Fishers Lane, Rm. 1061, Rockville, MD 20852 or electronic comments to https://www.regulations.gov. Please reference docket number FDA-2018-N-2455 with all comments.

About The Author:

Mark Allen Durivage has worked as a practitioner, educator, consultant, and author. He is managing principal consultant at Quality Systems Compliance LLC, an ASQ Fellow and SRE Fellow. Durivage primarily works with companies in the FDA regulated industries (medical devices, human tissue, animal tissue, and pharmaceuticals) focusing on quality management system implementation, integration, updates, and training. Additionally, he assists companies by providing internal and external audit support as well as FDA 483 and warning letter response and remediation services. He earned a BAS in computer aided machining from Siena Heights University and an MS in quality management from Eastern Michigan University. He holds several certifications, including CRE, CQE, CQA, CSSBB, RAC (Global), and CTBS. He has written several books available through ASQ Quality Press, published articles in Quality Progress, and is a frequent contributor to Life Science Connect. You can reach him at mark.durivage@qscompliance.com with any questions or comments.