Aman Khera

As citizens of the healthcare community, we’re all appropriately impatient for innovative medical products to achieve both regulatory approval and rapid market acceptance. To help meet those ends, the increased use of technology and new analytical capabilities has ushered in an influx of health-related real-world data (RWD) and real-world evidence (RWE) solutions.

Thus, draft guidance issued by FDA in late September provides welcome insights into how the regulatory agency views the underlying RWD that supports RWE. In brief, it reveals a greater acceptance of non-traditional study designs and data sources. However, the true significance of this guidance goes far beyond what can be read at face value. Indeed, the spirit of the guidance is as important—arguably even more important—than the letters written on its pages.

At its core, the guidance prompts all stakeholders to think much earlier and much deeper about two things: their data strategy, and their opportunities for collaboration.

The concept of “data strategy” is key. Throughout the guidance, FDA has raised the bar on the very mindset we must use to think about data. It cannot be an afterthought contemplated only at the later stages of development. Instead, it must be a strategic consideration at every mile along the product development roadway—from clinical research to commercialization. Furthermore, FDA has clearly indicated that it wishes to help provide useful roadside directions to those stakeholders who embrace early, collaborative conversations.

As with any guidance, there is much information to unpack and significant caution signs to observe. This article will delve into the speed bumps and the opportunities suggested by FDA’s guidance. It will also offer insights into factors FDA establishes for assessing RWD source and study design—including relevance, data capture, missing/absent data, validation, and study design elements.

FDA’s September 2021 draft guidance document titled, “Real-World Data: Assessing Electronic Health Records and Medical Claims Data to Support Regulatory Decision-Making for Drug and Biological Products” signals the regulatory agency’s willingness to consider the use of RWD and RWE at strategic points on the clinical development and commercialization continuum. Specifically, it opens the door to expanded labeling through the effective use of RWD as an alternative to traditional post-approval studies.

Although it is true that the guidance homes in on post-approval studies, we risk missing the forest for the trees if we fail to step back and observe it within the larger body of clinical research destined for regulatory review. This guidance is only the beginning.

Post-approval studies are the logical place to start RWD/RWE guidance because they represent the last leg of the research journey. The initial safety and efficacy mile markers have already been passed. So, the September document looks at the potential use of RWD and RWE, “…to help support the approval of a new indication for a drug already approved” or “…to help support or satisfy post-approval study requirements….”

While the guidance does not evaluate RWD and RWE in the context of pre-approval clinical trials, FDA unmistakably recognizes the constant need to pivot. The tone and tenor of the draft suggest openness to using RWD as a viable component of pre-approval drug development. By setting a precedent for the acceptance of RWD to expand a product label or for safety surveillance, the industry may reasonably anticipate a future for RWD in other aspects of clinical research as well.

That conclusion is bolstered if we take the September guidance in context. Consider other guidance on the use of RWD as an external control cohort, as well as moves such as FDA’s recently announced partnership with Health Canada and the United Kingdom’s Medicines and Healthcare products Regulatory Agency (MHRA). In late October, the three regulatory agencies unveiled their collective effort to provide “… guiding principles that we believe will support the development and maturation of good machine learning practice.”

Viewed holistically, FDA appears to be preparing for further change and industry disruptors on the RWD journey still to come. It also seems to recognize that continued engagement and collaboration will best position the agency to determine how to build a data-driven road forward that will help patients get to better outcomes faster. It will look favorably on stakeholders who are willing to smash historical siloes and engage in proactive collaboration both externally and internally.

There are valid reasons why silo mentalities exist to separate “clinical” and “commercial” life sciences endeavors. Yet, there is an increasing awareness that science and strategy must coexist and that both can be advanced through RWD and RWE. In fact, RWD and RWE represent the new and appropriate bridge between the two traditional land masses.

The September guidance gives a framework to sponsors that simultaneously advocates using RWD and RWE to springboard more cooperative discussions with regulators about data strategy. It acknowledges that sponsors and other industry stakeholders bring valuable perspectives and expertise to the table, and strongly encourages mutual dialog and informal data strategy partnerships. The agency wants to collaborate early and often:

“For all studies using EHRs or medical claims data that will be submitted to FDA to support a regulatory decision, sponsors should submit protocols and statistical analysis plans before conducting the study. Sponsors seeking FDA input before conducting the study should request comments or a meeting to discuss the study with the relevant FDA review division. All essential elements of study design, analysis, conduct, and reporting should be predefined, and, for each study element, the protocol and final study report should describe how that element was ascertained from the selected RWD source, including applicable validation studies.”

Sponsor companies and other industry stakeholders should embrace this new dynamic of collaboration. In fact, ignoring it could well be to their peril.

For sponsors, upfront regulatory input into a comprehensive data strategy could help prevent unnecessary risk and streamline costs and timelines. For regulators, more insights from stakeholders could help better refine FDA’s stance on RWD and RWE. As the entire field of data and analytics evolves, it is imperative to think about how working with various stakeholders—regulatory or otherwise—could help optimize data strategies to establish valuable RWE from multiple perspectives.

RWD and RWE can indeed be the critical bridge across traditional siloes, with plenty of positive implications for reaching the ultimate destination—better patient care.

Five factors to assess the veracity of RWD

It is important to note that FDA’s September guidance focuses narrowly on data sourced from electronic health records (EHRs) and medical claims, and on potential post-approval expansion of product indications. These data sets may describe the patient journey with considerable granularity, but neither was created to aid drug development or approval—and therein lies another crucial feature of the draft guidance. It acknowledges the limitations of data sources designed for other purposes, but at the same time, it proposes best practices for their selection and use.

The central “best practice” in any data strategy is determining whether the available RWD is of sufficient quality to be beneficial—or not—for its stated purpose. In every case, RWD needs such a purpose before it can be used effectively as RWE.

In selecting data sets, FDA guidance recommends starting with the hypothesis and working backward to understand the data elements and the sources. “Each data source should be evaluated to determine whether the available information is appropriate for addressing a specific study hypothesis.”

It is vital to be both transparent and humble when considering the data’s limitations. The potential to achieve economies and efficiencies through the use of RWD must never offset any data or analytical constraints. As promising as RWD may be, it is not an easy solution for replacing, augmenting, or complementing traditional studies. In truth, there may be many situations for which the value of traditional prospective RWE generation cannot be achieved through the use of RWD.

When contemplating the use of RWD and RWE, additional steps are necessary to assure FDA of the quality of the data and the analytical plan. Therefore, the guidance frames the following five factors for sponsors to evaluate when assessing RWD source and study design:

 Questions to ask include: What factors influence case inclusion, data integrity, and generalizability? We must clearly establish the reason for selecting any particular data source relative to the specific question being asked, and conscientiously explore all possible confounders. As the draft guidance notes, “For example, differences in the practice of medicine around the world and between healthcare systems may dramatically impact the usability of the data source considering the question being explored. Moreover, a particular RWD dataset may reflect disproportionate patient populations in terms of age, socioeconomic status, insurance coverage and access to care, risk factors and other potential confounders.”

 Sponsors must think about how a study protocol is addressed or accommodated by routine data collection in an EHR or claims system. In addition, how are linkages with other study components (e.g., PRO questionnaire processes) addressed? How are unstructured data (e.g., physician notes) acquired and used? The guidance explains, “For example, if the question pertains to real-world clinical effectiveness, would a patient’s use of non-prescription drugs—not typically captured in electronic medical records or claims datasets—‘cloud’ the assessment?” If such exposure is particularly relevant to the study question, the protocol should describe how the information gap will be addressed.

 How are missing (i.e., intended but not collected) or absent (not intended to be collected) data addressed? Furthermore, what are the reasons why the data are missing or absent? The guidance presents the issue this way:

“For example, lack of information about the result of a laboratory test could be caused by different circumstances: (1) the test might not have been ordered by the health care provider; (2) the test might have been ordered but not conducted; (3) the test might have been performed, but the result was not stored or captured in the data source; or (4) the test might have been performed and the result was stored in the data source, but data were not in an accessible format, or lost in the transformation and curation process when the final study-specific dataset was generated. Because providers might order a laboratory test based on a patient’s characteristics, the decision not to order the test or a patient’s decision to forgo the test may have implications on the data’s fitness for use in a proposed study.” It goes on to add, “Assumptions regarding the missing data (e.g., missing at random, missing not at random) underlying the statistical analysis for study end points and important covariates should be supported and the implications of missing data considered.”

The goal is to examine how data elements (study variables) are defined, obtained, classified, and verified. To that end, the guidance recommends these steps: “To understand how potential misclassification of a variable of interest (e.g., exposure, outcome, covariate) might impact the measure of association and the interpretation of results, sponsors should consider: 1) the degree of misclassification; 2) differential versus non-differential misclassification (e.g., differential misclassification of outcome by exposure); 3) dependent versus independent misclassification (e.g., correlated misclassifications of exposure and outcome when both are self-reported in the same survey); and 4) the direction toward which the association between exposure and outcome might be biased.”

 A data source should not define the study design, rather just the elements. The guidance states, “In other words, the question being explored must come first in order to provide the context necessary to ensure all the design elements—time periods, study population, exposure to and duration of treatment, outcome, covariates, etc.—have been addressed.”

Overall, the guidance suggests FDA wants sponsors to think carefully about the framework of these five points and form a data strategy—not merely react to data. The regulators invite sponsors to seek feedback while determining data relevance (factor #1), but they also want to see plans for the remaining four factors.

On a cautionary note, sponsors cannot overlook the fundamental fact that sometimes deficiencies in these key factors will not be addressable. The very nature of data collected for specific purposes and only available retrospectively is that it cannot be altered, and overly creative interpretations rarely will be acceptable.

In many ways, FDA’s guidance indicates a very thoughtful and appropriately cautious mindset—as befits its mission to protect the public’s health. As the agency ponders the value of alternative data sources, it advises against racing forward too quickly. It points out that data limitations can be significant and that RWD is not an easy solution. A proactive data strategy is essential for ensuring that quality data inputs result in sound conclusions, and for achieving overall operational efficiency.

Yet, the guidance also acknowledges the potential value of RWD and RWE, and it shows a new openness to alternative data sources. Equally enticing are the invitations it offers to take a partnership approach to elevate data strategy. Many forward-thinking industry stakeholders have explored the use of RWD to help guide decisions for years. Now, FDA appears to be leading the way toward better and more collaborative RWD regulatory frameworks.

Like traditional studies, RWD has its imperfections. Still, with substantial and conscientious documentation and explanation, RWE derived from RWD has exciting potential for improving research efficiency. With the appropriate safeguards, RWD can help us better understand drug safety and effectiveness under real-world conditions—much like being able to kick the proverbial tires and test-drive a new car in actual traffic, not just on a simulated, exclusionary test track. All stakeholders benefit—including sponsors, providers, and most importantly, patients—when we enable this to happen by maintaining RWD and RWE quality standards.

US Food & Drug Administration. “Real-World Evidence.” 

https://www.fda.gov/science-research/science-and-research-special-topics/real-world-evidence

US Food & Drug Administration. “Real-World Data: Assessing Electronic Health Records and Medical Claims Data To Support Regulatory Decision-Making for Drug and Biological Products.” September 2021. 

https://www.fda.gov/regulatory-information/search-fda-guidance-documents/real-world-data-assessing-electronic-health-records-and-medical-claims-data-support-regulatory

Corrigan-Curay, J. “The FDA Real-World Evidence (RWE) Framework and Considerations for Use in Regulatory Decision-Making.” U.S. Food & Drug Administration. May 12, 2021. 

https://www.fda.gov/media/148543/download

US Food & Drug Administration. “FDA In Brief: FDA Collaborates with Health Canada and UK’s MHRA to Foster Good Machine Learning Practice.” Oct. 27, 2021. 

https://www.fda.gov/news-events/press-announcements/fda-brief-fda-collaborates-health-canada-and-uks-mhra-foster-good-machine-learning-practice

SVP of Worldwide Evidence, Worldwide Clinical Trials and 

VP, Global Head of Regulatory Strategy, Worldwide Clinical Trials

Articles

Roadmap of twists and turns that lie ahead following most recent draft issued in September.

Reviewing FDA’s Latest Guidance on RWD 

The Clinical Trials Transformation Initiative (CTTI) presented its 

 for clinical trials in 2030 this week building on its six focus areas for clinical trial improvement. The COVID-19 pandemic has certainly disrupted clinical trials, pushing several focus areas to the forefront and driving the necessity for industry to reflect on needed changes for long-term success.

Quality by design (QbD) to write better protocols is particularly important. Researchers must ask themselves questions like: “Could this protocol be less burdensome to the site and patient without negating data collection or compromising safety?” As industry tries to strike this balance, it is imperative that researchers think critically when developing protocols to ensure a sound trial foundation.

Patient engagement certainly came to the forefront as researchers worked to troubleshoot how to both recruit and retain patients during a pandemic. 

Lastly, the need to shift to virtual trials in response to COVID-19 has made industry consider the benefits of digitization more than ever before. The pandemic has changed hybrid and virtual trials from ‘a nice to have’ to a ‘must have,’ or at least ‘must think about.’ Industry has had to get smart and embrace initiatives that have been waiting in the wings for quite a while. 

The COVID-19 pandemic has globally demonstrated how important clinical research is and also how quickly our industry can adapt and collaborate. It’s clear that we need to revisit the makeup of clinical trials to ensure we are conducting the most efficient and accurate trials we can.

I hope CTTI’s vision for clinical trials in 2030 will help provide a roadmap of tangible actions and activate industry change. We cannot go back to the way things were. We all need to play our part in rethinking clinical trials for the betterment of humanity. We must never forget that we are trying to progress healthcare for friends, family, neighbors, and our global community.

is the global head of regulatory strategy at Worldwide Clinical Trials

The COVID-19 pandemic has pushed several focus areas to the forefront of the industry.

Observations on CTTI’s Vision for Clinical Trials in 2030

The COVID-19 pandemic has swiftly altered the course of clinical trials globally. While much attention has been paid to its negative consequences, a recent set of surveys suggests COVID-19 may act as a pivot-point and change agent in four areas of the biopharma industry:

Artificial intelligence (AI) and machine learning

show that the pandemic has substantially influenced biopharma executives’ expectations in these key areas. This article will present those findings, as well as discuss some of their practical implications for clinical trials going forward.

The BCI is designed to measure the confidence of the industry’s C-Suite and executive leadership. Respondents represent biopharma companies with facilities in the United States, Europe, andthe Asia-Pacific region. The survey includes 259 questions about six key business indicators: capital markets, deal landscape, clinical development, regulatory affairs, commercialization, and business model and workforce.

The latest BCI survey was conducted twice:

in November 2019, just before the novel coronavirus was identified (n=113).

in April-May 2020, after the implementation of regional lockdowns due to COVID-19 (n=133).

Both surveys asked identical questions to enable valid comparison of executives’ sentiments pre- and post-COVID-19. In this article, we’ll specifically focus on takeaways gleaned from two questions:

Question 1: “Over the next 12 months, what is your confidence that the 

will have a positive impact on the biopharma industry?”

Responses indicate that executives’ sureness in digital health technologies rose 25% and their convictions about AI and machine learning increased 15%, but their confidence in the positive impact of the regulatory landscape dropped by 25%. (Other results will remain outside the scope of this article.)



Figure 1: % High Confidence of Positive Impact in Next 12 Months


Question 2: “What is your confidence whether that the following innovations will positively impact the biopharma industry in the next 12 months?”

Responses show that the percentage of executives who believe real-world evidence (RWE) capability will increase significantly went up by 12%. (Other results will remain outside the scope of this article.)



With all of the data collected by the BCI surveys, why emphasize these particular findings?

As we reviewed the data, many of the results simply confirmed already well-documented industry trends. However, the meaningful change in confidence surrounding digital technologies, AI/machine learning, RWE, and the regulatory landscape may reveal a larger interconnected picture.

Digital technology is not new to the clinical trial space, but one could argue that significant adoption has never reached much more than a slow simmer. Until the pandemic disrupted in-person contact, sponsor companies and regulatory agencies hesitated to embrace virtual trials. The BCI data affirms that COVID-19 is forcing the industry to think carefully about how to weave digital technologies into every stage of drug development—and it is seeing potential benefits for doing so.

Moving consent documentation from paper to glass, for instance, is just the very beginning of the digital technology evolution for clinical trials. The question is not 

to incorporate wearables, biosensors, etc. We have found that sponsor companies now are seeking guidance about 

they should use such technologies to find patients, retain them and collect necessary data.

In the current environment, digital technology has gone from a “nice-to-have” to an essential part of keeping trials up and running. Yet even when the disruption from COVID-19 subsides, it’s highly unlikely that the industry will ever go back to dismissing digital approaches within the drug development cycle. The ability to layer AI/machine learning on top of the data gathered via digital health technologies is opening new doors that won’t be closed again.

The intersection of digital, AI, RWE and regulatio

Although the spike in confidence in digital health technology earns merits on its own, it also probably plays a foundational role in how industry executives viewed the other elements addressed in the BCI surveys. The rising support for AI/machine learning and RWE, in particular, would seem to have strong links.

It stands to reason that increased reliance on digital technologies makes it easier and faster for sponsor companies to gather clinical trial data and RWE. Likewise, as digital technologies collect increasing volumes of data, AI/machine learning will be necessary to analyze it all with speed and accuracy.

What the BCI numbers may illustrate is that any skepticism about the use of digital health technology and AI to amass and analyze trial data and RWE is quickly vanishing. In the quest for a COVID-19 vaccine, for example, we see enthusiasm for using AI to help validate data and derive better answers for patients faster. The historic drug development cycle of anywhere from six years to 10 years or more

 is simply unacceptable. Thus, there is higher confidence that the right combination of digital technology, AI and RWE can safely speed up trials and overall development time.

The question, of course, is what that “right combination” looks like from operational and regulatory perspectives.

During the pandemic, some regulatory agencies such as the U.S. Food and Drug Administration (FDA) have worked hard to be collaborative and transparent in their processes. They have granted emergency use authorizations (EUAs)

and made other concessions to hurry potential therapies into clinicians’ and patients’ hands. They have even taken time to acknowledge their need and desire to work faster and more collaboratively.

Yet, in practice, a cautious approach is sure to remain—as is appropriate for the regulatory function. Some healthy regulatory skepticism is certain to arise out of lessons learned from COVID-19, as well as recent digital health misadventures

 such as the Theranos blood test violations.

For regulatory agencies worldwide, COVID-19 has become a pilot program. They’ve had to evolve in the public eye—and without any reduction in their non-COVID-19 demands. Many are figuring out how to work better, faster and with more scientific rigor so they can be of greater service to humanity. But the downward trend in regulatory confidence reflected in the BCI surveys illustrate that within industry leadership, there may not be enough appreciation for how complicated such evolution can be.

How the industry adapts to digital technology advances won’t resemble its past approach. The genie is permanently out of the bottle. So, although adaptation will mean different things to different stakeholders, all must be prepared for the effects of further digital health technologies and data.

Digital technologies and AI will affect how sponsors and contract research organizations run clinical trials and how regulatory agencies assess them—including how regulators view post-approval RWE requirements. They will pay additional attention to safeguarding and validating data collection, as well as its use and extrapolation.

Will regulators consider new levels of data protection in addition to HIPAA, GDPR and other privacy and security rules? What data extrapolations will be allowed to support a product’s safety and efficacy story? How will RWE analysis impact product approvals and sales strategies? These are just a few of the questions that the industry must address.

Greater confidence in technology, AI and RWE may let us step away from stringent trial inclusion/exclusion criteria earlier, allowing us to rely on evidence generated as therapies are used in actual daily life. Sponsor companies should expect to see the traditional “line in the sand” between clinical trial data and post-approval data blur. It’s likely to move backward, with an increased volume of Phase IV trials and RWE. While that would move products from trial to market sooner, it puts the onus on sponsors to collect, analyze and report data faster and more accurately within regulatory confines.

From a regulatory perspective, data validation and data disparities probably will come under greater scrutiny—especially for trials involving major indications across the globe. Careful consideration will need to be given to inconsistencies in the way data is collected and reported in different geographic regions and for different patient populations. 

The executive perceptions measured by the most recent BCI demonstrate the industry’s heightened confidence in the future direction of digital health technologies, AI/machine learning and RWE. They also highlight the need for regulatory agencies to keep pace with technology and data developments.

Each of these factors is but one piece in the clinical trial and drug development puzzle. Each presents benefits and challenges on its own, but they are interwoven as well. Viewed together, the double-digit changes in attitudes toward these four elements signal that biopharma executives believe the industry is accelerating toward a permanently altered state.

Change never comes without hiccups, but the acceleration is there. Transformations in these four areas in response to COVID-19 promise to converge, touching every aspect of clinical trials and drug development.

is the Global Head of Regulatory Strategy for

Deutsch J. How long will it take to develop a coronavirus vaccine? 

https://www.politico.eu/article/coronavirus-vaccine-how-long-will-it-take-to-develop/

Coronavirus Disease 2019 (COVID-19) Emergency Use Authorizations for Medical Devices. U.S. Food & Drug Administration. Available at: 

https://www.fda.gov/medical-devices/emergency-use-authorizations-medical-devices/coronavirus-disease-2019-covid-19-emergency-use-authorizations-medical-devices

European Commission, EMA and FDA agree new priorities to strengthen their collaboration on medicines. European Medicines Agency. Available at: 

https://www.ema.europa.eu/en/news/european-commission-ema-fda-agree-new-priorities-strengthen-their-collaboration-medicines

Meskó B. Digital Health Ventures That Flew Too Close To The Sun. LinkedIn. July 23, 2020. Available at: 

https://www.linkedin.com/pulse/digital-health-ventures-flew-too-close-sun-bertalan-mesk%25C3%25B3-md-phd/

Findings from the most recent Biopharma Confidence Index show that the pandemic has substantially influenced biopharma executives’ expectations in key areas like artificial intelligence/machine learning and real-world evidence. 

Aman Khera

Articles

Reviewing FDA’s Latest Guidance on RWD

Observations on CTTI’s Vision for Clinical Trials in 2030

C-Suite Survey Underscores Digital Technology’s Growing Role in Clinical Trials