Beth Schurman is Partner at Herspiegel Consulting.

Beth Schurman

 for a new Real-World Evidence (RWE) Program. The agency proposes to use the RWE Program to guide generation of data in support of approval for new indications or to help support post-approval study requirements. It is proposed to be used by both the FDA’s drug and biologic review programs. This article highlights the scope of this program, its opportunities and gaps, and the implications for the pharmaceutical industry.	 

	To understand the scope of this framework, it is important to understand the definition of real-world data (RWD) and real-world evidence (RWE) as defined by the FDA in the current draft framework:

 refers to patient data that is collected through a variety of sources such as electronic health records (EHRs), medical claims and billing data, data from product and disease registries, and patient-generated data (includes in-home-use settings and mobile devices).

 is clinical evidence obtained from the analysis of RWD that provides information about usage, risks, and benefits of a medical product derived from sources other than traditional randomized clinical trials.

	The FDA’s RWE Program will assess inclusion of RWD to support approval of new indications and to help to support or satisfy post-approval study requirements. The following may be included:

		Addition or modification of an indication-change in dose, dose regimen, route of administration

		Addition of comparative effectiveness or safety data

	RWD can be used to supplement traditional clinical trial data and improve the efficiency of these trials. Trials within the RWE Program scope can be one of two types:

 Traditional clinical trial supplemented with RWD from EHR, medical claims, or pharmacy databases.

 Contains some elements which closely resemble routine clinical practice.

	The hybrid trial approach may include the use of RWD as a control arm in a traditional clinical trial. Observational studies are anticipated within the pragmatic approach. The FDA intends to use an existing guidance, 

Best Practices for Conducting and Reporting Pharmacoepidemiologic Safety Studies Using Electronic Healthcare Data

, to evaluate epidemiologic studies as applicable for hypothesis generating pragmatic studies.

	Current challenges of the proposed framework for the pharma industry

	The new draft framework leaves some significant gaps with regard to a defined approach that pharmaceutical manufacturers should take in their RWE trial design for regulatory approval. The guidelines do not yet completely define the RWE trial design approaches that are approvable, nor the regulatory submission process. The FDA was actively soliciting feedback up until Feb. 5 that will be used in its final RWE guidance.

	The agency has stated it will review on a case-by-case basis and will use three criteria for evaluation.

		If the RWD selected is fit for use to generate data for product effectiveness decisions.

		If the study design used to generate RWE can provide robust scientific evidence to help address the regulatory question.

		If the execution of the studies generating RWE meets FDA regulatory requirements.

	The last criterium forms a catch-22 as the current FDA regulatory requirements regarding RWE do not specifically outline use in regulatory approval. Several FDA regulations and guidance documents are available for the use of electronic data, eHealth records in clinical investigations, electronic informed consents, and Q&As that could potentially be used for the FDA RWE Program. These include guidances such as 

Use of Electronic Source Data in Clinical Investigations

Use of Electronic Health Records in Clinical Investigations

, among others. Per the draft guidance: “FDA has gained considerable experience assessing electronic health care data (e.g., EHRs, medical claims data, registries) through experience with the Sentinel System and other data systems.” The FDA will be using this experience to guide its perspective on data submissions within the RWE Program.

Pharmaceutical manufacturers must seek guidance from the FDA for each trial design proposed to ensure regulatory processes are met. With this draft framework publication, a number of areas are yet to be defined, according to the FDA. The agency has stated the intention to update and design guidance on how to include RWD to design both randomized and non-randomized hybrid clinical trials as well as observational studies. To achieve this, the FDA directly outlines in the draft guidance the factors that would need to be considered, such as:

		Types of interventions or therapeutic areas for which routine clinical data could be used.

		The reliability and quality of data collected from the routine clinical care settings.

		If outcomes are rare, the number of patients need to be tested.

		How to account for variations in routine clinical practice

		The extent to which randomization can and should be included.

	With the eight-week solicitation period for suggestions and feedback ended in early February, a mature form of the framework is expected in the future. The FDA states that the RWE Program “…will involve the establishment of demonstration projects, engagement with stakeholders intended to promote shared learning and consistency in applying the framework, and the development of guidance documents to assist sponsors interested in using RWE to support their work.” The RWE framework will also include data standards for collection, analysis, and submission of RWD. The need for RWE data may spur the need to develop a common data model (CDM) in order to work with RWD across different sources. The CDM would unify terminology, vocabularies, and coding schemes. It will be incumbent on pharmaceutical manufacturers and clinicians to remain actively involved in the development of the final FDA RWE guidance.

	The FDA intends to review gaps in both the RWE framework and in RWE data systems, as well as the interoperability, and provide strategies to address them. The following summarizes gaps currently identified by the FDA that may need to be addressed while framing guidance for the RWE Program.

		Lack of enough data showing concordance between the results of traditional clinical trials vs. observational studies.

		Lack of reporting requirements for various observational studies.

		Lack of data-recording standards and interoperability across various healthcare data-collection systems.

		Challenges in integrating data for the same patient across various data sources.

		Challenges in collecting data from electronic patient reported outcomes (ePROs), wearables, biosensors, and mobile app.
		 

	Interestingly, the RWE framework also opens the door for possible inclusion of data derived from outside the US. The continued evaluation and publication of data and systems gaps with strategic recommendations by the FDA will have to address how to support these issues and how to address HIPAA privacy rules within RWE.

	Pharma will have to consider the most efficient way to approach RWE generation for regulatory submission, given the case-by-case approach from the FDA. Additionally, companies will need to address the agency’s stated concern that real-world results would be cherry picked for positive and beneficial data. Some of the questions that the pharma industry should consider with regard to RWE evaluation include:

		Does the pharma company’s current organizational structure (decision ownership and stakeholder involvement) align to efficient execution of RWE generation for regulatory approval?

		Have sufficient resources been allocated within the organization to handle RWE generation for regulatory approvals?

		What is the decision-making process to evaluate trade-offs between RWE vs. traditional trials to receive follow-on indication approvals for products?

		How cost effective would it be to choose the RWE program vs. traditional clinical trials?

		Will the FDA’s acceptance of RWE open the door for more partnerships between pharma and academic centers and how this will be handled?

		Is the company able to effectively choose between different types of RWD and patient reported outcomes?

		What discussions should be held with the FDA and when should they be held for RWE trial design and submissions-especially given the current case-by-case approach to review and approval?

	The RWE Program opens opportunities for more resource efficient approaches to follow-on indications. Increased definition by the FDA is needed to guide resource investment and trial design approaches to ensure efficient investment. Pharmaceutical manufacturers will likely need to evolve their internal structure and systems, including decision models related to RWE trial design and regulatory interaction to accommodate this new route of approval. We look forward to reviewing and understanding the updated guidance expected from the FDA in the future.

	1. Framework for FDA’s Real-World Evidence Program: 

https://www.fda.gov/downloads/ScienceResearch/SpecialTopics/RealWorldEvidence/UCM627769.pdf

	2. Statement from Scott Gottlieb, MD, on the FDA’s new strategic framework: 

https://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/ucm627760.htm

	3. FDA public docket collecting RWE draft guidance feedback: 

https://www.federalregister.gov/documents/2018/12/07/2018-26546/framework-for-a-real-world-evidence-program-availability

	4. Best Practices for Conducting and Reporting Pharmacoepidemiologic Safety Studies Using Electronic        Healthcare Data: 

https://www.fda.gov/downloads/drugs/guidances/ucm243537.pdf

Articles

A look at the scope of the agency’s draft framework for evaluating the use of RWE to support new drug approvals and the implications for sponsors.

The Framework for FDA’s  Real-World Evidence Program

	At the end of 2018, the FDA released a draft framework for a new Real-World Evidence (RWE) Program. The FDA proposes to use the RWE Program to guide generation of data in support of approval for new indications or to help support post-approval study requirements. It is proposed to be used by both the FDA’s drug and biologic review programs. This whitepaper highlights the scope of this program, its opportunities, gaps, and implications for the pharmaceutical industry.

	To understand the scope of this framework, it is important to understand the definition of

 as defined by the FDA in the current draft framework:

 refers to patient data that is collected through a variety of sources such as Electronic Health Records (EHRs), medical claims and billing data, data from product and disease registries, patient-generated data (includes in-home-use settings and mobile devices).

 is clinical evidence obtained from the analysis of RWD that providing information about usage, risks, and benefits of a medical product derived from sources other than traditional randomized clinical trials.

	The FDA’s RWE program will assess inclusion of RWD to support approval of new indications and to help to support or satisfy post-approval study requirements. The following may be included:

: traditional clinical trial supplemented with RWD from EHR, medical claims, or pharmacy databases

: contains some elements which closely resemble routine clinical practice

	The hybrid trial approach may include the use of RWD as a control arm in a traditional clinical trial. Observational studies are anticipated within the pragmatic approach. The FDA intends to use an existing guidance, the Pharmacoepidemiologic Guidance, to evaluate epidemiologic studies as applicable for hypothesis generating pragmatic studies.

	Current challenges of the proposed framework for the pharmaceutical industry

	The new draft framework leaves some significant gaps with regard to a defined approach that pharmaceutical manufacturers should take in their RWE trial design for regulatory approval. The guidelines do not yet completely define the RWE trial design approaches that are approvable, nor the regulatory submission process. The FDA is actively soliciting feedback (deadline: Feb 5, 2019) that will be used in the final RWE guidance.

	The FDA has stated they will review on a case-by-case basis and will use three criteria for evaluation: 

		If the RWD selected is fit for use to generate data for product effectiveness decisions

		If the study design used to generate RWE can provide robust scientific evidence to help address the regulatory question

		If the execution of the studies generating RWE meets FDA regulatory requirements

	The last criterium forms a catch-22 as the current FDA regulatory requirements regarding RWE do not specifically outline use in regulatory approval. Several FDA regulations and guidance documents are available for the use of electronic data, eHealth records in clinical investigations, electronic informed consents Q&A’s that could potentially be used for the FDA RWE program. These include Electronic Source Data in Clinical Investigations and Use of Electronic Health Records in Clinical Investigations, among others. Per the draft guidance: “FDA has gained considerable experience assessing electronic health care data (e.g., EHRs, medical claims data, registries) through experience with the Sentinel System and other data systems.” The FDA will be using this experience to guide their perspective on data submissions within the RWE Program.

	Pharmaceutical manufacturers must seek guidance from the FDA for each trial design proposed to ensure regulatory processes are met. With this draft framework publication, a number of areas are yet to be defined, according to the FDA. The FDA has stated the intention to updated and design guidance on how to include RWD to design both randomized and non-randomized hybrid clinical trials as well as observational studies

To achieve this, the FDA directly outlines in the draft guidance the factors that would need to be considered such as:

		Types of interventions or therapeutic areas for which routine clinical data could be used

		The reliability and quality of data collected from the routine clinical care settings

		If outcomes are rare, the number of patients need to be tested

		The extent to which randomization can and should be included

	The FDA provided this draft framework with an 8-week solicitation period for suggestions and feedback. A mature form of the framework is expected in the future. The FDA states that the RWE Program “…will involve the establishment of demonstration projects, engagement with stakeholders intended to promote shared learning and consistency in applying the framework, and the development of guidance documents to assist sponsors interested in using RWE to support their work.” The RWE framework will also include data standards for collection, analysis, and submission of RWD. The need for RWE data may spur the need to develop a common data model (CDM) in order to work with RWD across different sources. The CDM would unify terminology, vocabularies, and coding schemes. It will be incumbent on pharmaceutical manufacturers and clinicians to remain actively involved in the development of the final FDA RWE guidance.

	The FDA intends to review gaps in both the RWE Framework and in RWE data systems as well as the interoperability and provide strategies to address them. The following summarizes gaps currently identified by the FDA which may need to be addressed while framing guidance for the RWE program.

		Lack of enough data showing concordance between the results of traditional clinical trials vs. observational studies

		Lack of reporting requirements for various observational studies

		Lack of data recording standards and interoperability across various health care data collection systems

		Challenges in integrating data for the same patient across various data sources

		Challenges in collecting data from electronic patient reported outcomes, wearables, biosensors and mobile app

	Interestingly, the RWE framework also opens the door for possible inclusion of data derived from outside the U.S. The continued evaluation and publication of data and systems gaps with strategic recommendations by the FDA will have to address how to support these issues and how to address HIPPA within RWE.

	Further implications for the pharmaceutical industry

	Pharma will have to consider the most efficient way to approach RWE generation for regulatory submission, given the case-by-case approach from the FDA. Additionally, companies will need to address the FDA’s stated concern that real world results would be cherry picked for positive and beneficial data. Some of the questions that the pharma industry should consider with regard to RWE evaluation include:

		Does the pharma company’s current organization structure (decision ownership and stakeholder involvement) align to efficient execution of RWE generation for regulatory approval?

		Framework for FDA’s Real-World Evidence Program: 

		Statement from Scott Gottlieb, M.D., on the FDA’s new strategic framework:	

		FDA public docket collecting RWE draft guidance feedback: 

		Best Practices for Conducting and Reporting Pharmacoepidemiologic Safety Studies Using Electronic Healthcare Data: 

The FDA proposes to use the RWE Program to guide generation of data in support of approval for new indications or to help support post-approval study requirements.