Michael Kuss

 The use of patient-reported outcome (PRO) data in clinical trials has become more widespread, particularly for products developed to treat conditions such as chronic pain where the intention is to ameliorate symptoms, facilitate functioning, or improve overall quality of life. Faced with growing cost pressure from private and public third-party payers, and the need for product differentiation in a highly competitive market, sponsors are increasingly relying on patient views of product efficacy to generate value propositions that go beyond traditional safety and clinical efficacy messages. For analgesia clinical trials in particular, patient-reported evidence is essential for establishing efficacy, as well as the advantage of an investigational drug over existing products. 

In recent years, regulatory scrutiny of the use of PRO data in product approval and labeling claims has escalated, and some of the PRO measures used to support approval or labeling in the past are no longer considered adequate. In chronic pain, the use of PRO data as both primary and secondary endpoints to substantiate product efficacy is necessary for securing approval.

  The US Food and Drug Administration (FDA) and European Medicines Agency (EMA) have imposed standards that call for a higher degree of scientific rigor in the development and evaluation of PRO measures, as well as the level of documentation required to support use and content validity of these measures in confirmatory clinical trials. 

This article provides an overview of the use of PRO measures in analgesia clinical trials, including guidance on the evaluation and development of valid PRO instruments, statistical and regulatory considerations for the use of PRO data, and consensus recommendations on appropriate PRO instruments to measure core outcomes in chronic pain. 

PRO is an umbrella term covering a range of consequences of a disease condition and/or its treatment as reported by the patient, including symptom experience, sense of wellbeing, functional status, treatment adherence, and treatment satisfaction. For the assessment of pain, PRO measures are often the only viable clinical endpoints because there are no objective physical or physiological markers of disease or treatment activity that can be observed or measured. Although pain reduction and safety are necessary endpoints in analgesia clinical trials, they may not be sufficient for a comprehensive evaluation of the overall benefit or harm of treatment due to the complexity of pain and its negative impact on diverse aspects of function.

  Therefore, evaluation of analgesic efficacy requires the assessment of multiple PRO domains in order to adequately characterize treatment impact. 

However, variability in PRO measures across analgesia clinical trials may hinder evaluation or meaningful comparison of the efficacy of different treatments. As a result, the Initiative on Methods, Measure, and Pain Assessment in Clinical Trials (IMMPACT) identified six core outcome domains that should be considered in clinical trials of treatments for chronic pain, including: pain; physical functioning; emotional functioning; ratings of improvement and satisfaction with treatment; symptoms and adverse events; and disposition (e.g., adherence to the treatment regimen or reasons for premature withdrawal).

  Although assessment of these core domains is not a requirement for product approval by regulatory agencies, it facilitates comparison of data, encourages more complete investigation and reporting of relevant outcomes, and simplifies the process or designing and reviewing research protocols. PedIMMPACT later issued recommendations on core outcome domains and measures for pediatric acute and chronic or recurrent pain clinical trials.

Data from studies of post-operative pain suggest improved analgesia leads to better patient outcomes. However, a systematic review performed at Johns Hopkins found insufficient evidence to support subsequent improvements in patient-centered outcomes such as quality of recovery or quality of life.

  This underscores the need for validated instruments to measure patient-reported outcomes. 

A PRO instrument is a means of capturing patient-reported data, as well as all the information and documentation that supports its use. In general, this information includes clearly-defined methods and instructions for administration, a standard format for data collection, and well-documented methods for scoring, analysis, and interpretation of results in the target patient population.

The purpose of a PRO measure is to capture the patient's experience from his or her own perspective, so the PRO instrument chosen for a clinical trial should be supported by evidence of its usefulness in measuring the concept it is intended to measure in the intended patient population. When evaluating a PRO instrument to support a labeling claim, sponsors should consider the:

Population enrolled in the clinical trial

The primary challenge of using PRO instruments to support labeling claims is the lack of a gold standard against which to measure criterion validity.

  It is the sponsor's responsibility to provide sufficient justification and validation for PRO instruments used in clinical trials. New PRO instruments may be needed if there are no existing, well-validated measures. Instrument development and evaluation requires considerable time, effort, and resources, so sponsors should first determine whether a new measure is absolutely necessary. In some cases, adding and testing a few items to a well-validated scale eliminates the need to develop a new PRO instrument.

Development of a PRO instrument involves a series of steps and requires attention to the specific goals of the instrument, its intended uses, and the characteristics of the intended respondents. The FDA considers development of a PRO instrument to be an iterative process (Figure 1).

  PRO instrument development is a highly-skilled activity that should be undertaken by specialists in measurement and psychometrics, and researchers experienced in qualitative interviewing techniques. If the PRO instrument will be used in other countries, language adaptation may be required, and translation may be a complex and time-consuming task. 

The fundamental adequacy of a PRO instrument to support a claim depends on the instrument's concept framework. The conceptual framework is an explicit description or diagram of the relationships between the questions or items in a PRO instrument and the concepts being measured (Figure 2).

  Concepts comprise the specific measurement goals of the PRO instrument, and they represent aspects of how patients function or feel related to their health condition or its treatment. Concepts may be made up of multiple subconcepts called domains. For example, the concept of psychological function may include domains of emotional function and cognitive function. In order to be used to support a labeling claim, a PRO instrument must have a conceptual framework that corresponds to the clinical trial endpoints described in the clinical trial protocol.

Measurement properties considered in the review of PRO instruments used in clinical trials. 

The FDA considers three categories of measurement properties when reviewing the adequacy of PRO instruments used in clinical trials to support labeling claims (Table 1):

Reliability—ability of a PRO instrument to yield consistent and reproducible estimates of true treatment effect

Content validity—evidence from qualitative research demonstrating that the PRO instrument measures the concept of interest

Ability to detect change—evidence that a PRO instrument can identify differences in scores over time in clinical trial populations who have changed with respect to the measurement concept

Content validity must be supported by evidence from qualitative studies that the items and domains of the PRO instrument are appropriate and comprehensive relative to its intended measurement concept, population, and use. In addition, content validity is specific to the population, condition, and treatment to be studied. Documentation of patient input in item generation, as well as evaluation of patient understanding through cognitive interviewing, can contribute to evidence of content validity. Sponsors are reminded that evidence of reliability or other types of validity will not overcome problems with content validity when the FDA evaluates instrument adequacy to measure the concept represented by the labeling claim. Therefore, it is important to establish content validity before evaluating other measurement properties.

Electronic PRO (ePRO) measures are PRO measures collected by electronic methods, such as telephone systems, computers, and other handheld devices. A recent survey of analgesia clinical trial decision makers at 51 pharmaceutical and biotechnology firms in the United States and Europe demonstrated that nearly all respondents were likely or definitely going to use ePRO technology for future clinical trials.

  The adoption of ePRO measures is based on the potential of its technology to reduce administrative burden, increase patient acceptance, avoid secondary data entry errors, and improve the accuracy and completeness of data.

  For example, electronic diaries automatically time-stamp all entries and can be set up to only allow data entry within specific timeframes. This minimizes recall bias, reduces missing data, and eliminates the need for manual data entry, which can introduce transcription errors. In addition, ePRO instruments allow real-time monitoring of PRO data and patient compliance. When using PRO instruments that were originally validated in paper form, sponsors are required to establish equivalence between the paper-based and electronic modes of administration. The International Society of Pharmacoeconomics and Outcomes Research has issued guidelines on establishing comparability when migrating PRO measures from paper to electronic format.

Statistical considerations when using PRO measures for labeling claims 

According to the FDA guidance on using PRO measures to support labeling claims, the statistical analysis and interpretation considerations for PRO endpoints are similar to considerations for any other endpoint used in medical product development. The FDA usually views unplanned or post hoc statistical analyses conducted after unblinding as exploratory and unable to serve as the basis of a labeling claim of effectiveness.

For insight on recommended PRO measures for pain, sponsors should refer to relevant FDA or EMA guidance documents. In analgesia clinical trials, pain intensity is typically a primary endpoint, and secondary endpoints may include evaluation of mood, sleep, functional and social performance, and health-related quality of life (HRQL). HRQL is a multi-domain concept that represents the patient's general perception of the effects of illness and treatment on physical, psychological, and social aspects of life. In order to support a claim of statistical and meaningful improvement in HRQL, sponsors need to provide evidence that all HRQL domains studied are important to interpreting change in how study participants feel or function as a result of the disease condition and treatment measured. Sponsors also need to demonstrate general improvement in the domains measured, without decrement in any domain. In addition, sponsors are required to validate the PRO instruments used to evaluate all endpoints.

Multiple domains and composite endpoints. 

For a PRO instrument with multiple domains, combining the scores to calculate a general score creates a composite endpoint. The use of composite endpoints for confirmatory clinical trials claiming treatment benefit poses certain challenges, as the rules for interpreting composite endpoints depend on substantial experience with the PRO measure in the clinical setting. In addition, the individual components of a composite endpoint will not be considered adequate to support a labeling claim unless those individual components are prespecified in the protocol as separate endpoints. Generally, the FDA discourages the use of a composite endpoint for confirmatory clinical trials when large variations are predicted to exist between its components.

Missing data can render clinical trial results to be inconclusive by introducing bias in the analysis of treatment differences and conclusions about treatment effect. Therefore, the FDA encourages sponsors to outline prespecified procedures for avoiding missing data, as well as for obtaining data on each patient at the time of early withdrawal.

  The National Academy of Sciences also provides guidance on the prevention and treatment of missing data in clinical trials.

  Rules for handling missing data should be specific to each PRO instrument and should be determined during the instrument development process. ePRO instruments typically produce less missing data than paper-based measures,

  and are generally preferred by respondents.

Both the FDA and EMA provide guidance on the use of PRO assessments in clinical trials to support product labeling, although the EMA focuses on HRQL, rather than specific PRO measures. In general, both regulatory agencies consider HRQL/PRO-based claims based on:

Rationale for HRQL/PRO assessment in the context of the disease or medicinal product

Justification of the choice of HRQL/PRO questionnaire(s)

Objectives of HRQL/PRO assessment and the hypotheses of HRQL/PRO changes

Evidence of validation of the HRQL/PRO questionnaire(s)

Adequacy of the statistical analysis plan

The FDA PRO guidance is more detailed, extensive, and restrictive in nature than the EMA reflection paper. However, both regulatory agencies prefer blinded, randomized clinical studies and require sponsors to specify a hierarchy for endpoints in order to prevent multiplicity. In order to approve a claim for HRQL, both the FDA and EMA require sponsors to demonstrate improvements in all or most of the relevant HRQL domains, unless a specific claim based on a subset of HRQL domains is pre-specified in the statistical analysis plan.

The FDA guidance on use of PRO measures indicates that, in order for a PRO instrument to be adequate, an adequate number of patients should support the opinion that the questionnaire items are adequate and appropriate and the characteristics of patients in the questionnaire development phase should match the characteristics of the target population. The FDA guidance advises sponsors to consider the data collection method, as well as all procedures and protocols related to the administration of the PRO instrument. During the review process, the FDA will review data quality control procedures, including screen shots of ePRO instruments.

The EMA has no dedicated internal HRQL/PRO expertise. Instead, the EMA outsources HRQL reviews to academic or clinical research organizations, depending on the reporter country, resulting in challenges in the harmonization of reviews. The FDA, on the other hand, has a Study Endpoints and Label Development (SEALD) group dedicated to PRO reviews and trains its divisions in PRO issues on an ongoing basis. The FDA also provides a detailed PRO information template that sponsors are requested to follow in order to submit PRO documents (Table 2).  

Consensus recommendations on PRO measures and instruments

IMMPACT-II published consensus recommendations on specific outcome measures, including validated PRO instruments, for assessing each of the recommended core outcome domains for chronic pain. The criteria used to evaluate these outcome measures included: appropriateness of the measure's content and conceptual model; reliability; validity; ability to detect changes over time; interpretability; precision of scores; respondent and administrator acceptability, burden, and feasibility; availability and equivalence of alternate modes of administration; and availability and equivalence of versions for different cultures and languages (Table 3).

To measure pain intensity, IMMPACT-II recommends the use of numerical rating scales (NRS), except in clinical conditions where another pain intensity scale is routinely used. Although other commonly used methods of rating pain intensity (e.g., visual analogue scales (VAS) and verbal rating scales (VRS)) are also reliable and valid, NRS measures tend to be preferred by patients and demonstrate smaller amounts of missing or incomplete data.

  VRS is offered as an alternative for patient populations that have difficulty completing NRS measures. IMMPACT-II also recommends the Short-Form McGill Pain Questionnaire to assess sensory and affective qualities of pain.

For physical functioning, IMMPACT-II recommends the use of both disease-specific measures, if available, and generic measures to assess response to treatment. Validated disease-specific measures of physical functioning including the Western Ontario and McMaster Universities Osteoarthritis Index and the Roland and Morris Back Pain Disability Scale. Generic measures include the Multidimensional Pain Inventory (MPI) Interference Scale or the Brief Pain Inventory (BPI) pain interference items which assess general activity, mood, walking ability, work, relationships, sleep, and enjoyment of life. Both the MPI and BPI interference scales have been shown to be reliable and valid, and have been translated into many languages. IMMPACT-II also suggests the SF-36 Health Survey (SF-36) as a generic measure of HRQL.

  However, both the FDA and EMA now consider the SF-36 to be unsuitable for making claims.

In the domain of emotional functioning, IMMPACT-II recommends the Beck Depression Inventory and the Profile of Mood States in chronic pain clinical trials. For ratings of improvement and satisfaction with treatment, IMMPACT-II recommends the Patient Global Impression of Change scale, which is increasingly used in chronic pain trials reported in the literature. 

A third IMMPACT meeting, IMMPACT-III, established consensus recommendations regarding development and evaluation of new PRO instruments for analgesia clinical trials.

Articles

PRO measures enrich the evaluation of treatment effectiveness and help sponsors differentiate products.

Michael Kuss

Articles

Patient-Reported Outcomes in Analgesia Clinical Trials for Chronic Pain