Jules T. Mitchel, MBA, PhD

Within the domain of clinical trials, the goals of quality by design (QbD) are to assure that the outcome data are fit for their intended purpose and that there is an absence of errors that could impact decision-making. Since all products and services within a clinical trial are subject to regulatory oversight, all sponsors, vendors, and clinical research sites must always be prepared for that knock on the door when your administrator says, as mine said to me a few years ago, “The FDA has arrived and would like to speak to the person responsible for the clinical trial.”

A frequently asked question is what have been the barriers to entry preventing pharmaceutical and medical device companies from widely adopting and implementing paperless and digitally based clinical trials? Ironically, for more than a decade, technologies allowing for paperless and digital clinical trials have been encouraged by regulators. Fortunately, the time has finally come for the transition to a global acceptance by the pharmaceutical industry and regulators of the use of paperless and digital technologies to collect clinical trial data both within the clinic and remotely.

The FDA and others have issued multiple guidance documents addressing the use of digital technology tools when used in clinical trials. Now that the electronic world has finally entered the clinical trials space, as part of routine pre-approval inspections, there must be stakeholder assessments of the impacts of digital systems on 1) informed consent; 2) data collected to determine effectiveness; 3) patient safety by assuring that all safety events are accurately collected and reported 4) data integrity and accountability; 5) protocol compliance; 6) monitoring of the study, including risk-based monitoring; and 7) drug/device accountability etc.

Since digital-based companies have not been routinely subject to FDA laws, rules, and regulations, the following are key elements to be addressed and assessed by these organizations choosing to enter the clinical trials space.

 The basic goal of software validation is to assure that configured applications act as intended when used in clinical trials. Therefore, when regulators inspect software products, it is critical to document that the trial software applications are validated. Fortunately, starting in 2002, the FDA has provided validation guidelines for software supporting regulated products. For new products hitting the market, an assessment should also be made if FDA clearance of the software would be required through the device 510(k) or PMA regulatory approval routes. For global programs, local regulatory requirements must also be addressed.

 These are elements that must be considered during development of digital devices for use in clinical trials. FDA has issued a guidance document to assist the industry on how to follow appropriate human factors and usability engineering processes so that new medical devices will be safe and effective for the intended users, uses, and use environments. It is recommended that these studies focus specifically on the user interface, including displays, controls, packaging, product labels, and instructions for use. As part of their design controls, manufacturers must also conduct a risk analysis that includes the risks associated with device use and the measures designed to mitigate those risks.

 Some of those fully committed to using paper records often ask the question, if a system is fully electronic how do we know that the data are real if there are no paper source records to compare with the electronic record? To assuage some of these concerns, as part of a company’s SOPs, there must be supporting documentation that the vendor has been qualified and that the software is in a validated state. It is also critical that all roles and responsibilities are properly identified, as well as how the original source data are controlled. This includes the presence of date and time stamps for all transactions, and an audit trail of changes cannot be overwritten.

 One of the first things the inspectors may request is to talk to the head of quality as well as who was directly responsible for the trial. Be prepared to share your organizational chart as it relates to the study and copies of relevant contracts/agreements, although inspectors are usually not concerned about financial matters. The inspectors may also ask for the documents supporting vendor qualifications, software validation/testing, and risk mitigation plans.

Articles

Tech companies entering space must adapt and adhere to regulations set by FDA.

Regulatory Considerations for Digitally Driven Trials

The pharmaceutical industry is a stellar industry that has contributed greatly to the health of individuals as well as to the public health. Just look at the rapidity that multiple COVID-19 vaccines and antivirals have recently reached the market, and more to come. The industry has one of the smartest workforce and most come to work with passion in their bones, whether it be in their first job or those positions that followed. Thus, when all players cooperatively work together as a team with common goals, it is a win-win for society at large.

However, there are times during the development and post-marketing phases when drug and device companies identify a need to contract services when there are gaps in internal resources to fulfill milestones or when a decision is made to outsource key services rather than build an internal team. Today, the companies that are outsourced to are called contract research organizations (CROs), an industry with a rich history. One of the early CROs was Charles River Laboratories. According to its website, “In 1947, a young veterinarian purchased one thousand rat cages from a Virginia farm and set up a one-man laboratory in Boston, overlooking the Charles River. In an effort to fulfill the regional need for laboratory animal models, he bred, fed, and cared for the animals and personally delivered them to local researchers.” The rest is history.

In terms of clinical research, according to the IQVIA website, in 1974, Dennis Gillings the founder of Quintiles, signed his first contract to provide statistical and data management consulting for pharmaceutical clients. Then, in 1982, Quintiles, Inc. was incorporated in North Carolina. Again, the rest is history.

Currently, CROs provide services in the areas of 1) Good clinical practices (GCP), including protocol writing, site monitoring, statistical analyses, and medical writing; 2) Regulatory affairs, including interactions with regulators and the preparation and submission of electronic common technical documents (eCTD); 3) Good manufacturing practices (GMP), including the synthesis and manufacturing of drugs and biologics, as well as the development and manufacturing of devices and diagnostics; and 4) Good laboratory practices (GLP), including pharmacology studies in animal models of disease and toxicology studies in established animal study designs.

While CROs must run efficient and profitable businesses, there must be a balance with the three major stakeholders in all businesses, including customers, employees, and investors. The reason is that to be successful, CROs must retain long-term customer relationships, develop new customers on an ongoing basis, as well employ and retain highly experienced professionals. The reason is that without satisfied customers and satisfied employees, the investors will not get their expected return on investment (ROI). Likewise, if a CRO does not appreciate and understand that some new customers, or even existing ones, may require special treatments from time to time, including a discount here and there, some big clients may become small clients, and the opportunity to grow together with smaller startup companies could disappear.

With turnover in the pharmaceutical industry the way it is, it’s the CROs that often have expertise and experience in the medical indication under investigation. In other words, it is not uncommon for a CRO to provide scientific, clinical, and business continuity for pharmaceutical company sponsors. Thus, to maximize the CRO’s contribution to the overall development programs, the CRO should become an active member of the team. This is especially true when working with small-to-mid-size companies that look forward to listening to inputs and ideas from their CRO partner.

Hence, if a CRO can brag, as some do already, about 1) the number of regulatory interactions and approvals they have under their belt; 2) how they accelerate time to database lock; 3) the number of ongoing COVID-19 studies they are managing; and 4) the number of rare disease accomplishments etc., they will be very attractive to new and existing customers. 

Brief history lesson sets stage for current state of CROs.

What is a CRO?

The recent evolution within the pharmaceutical industry to the use of technologies associated with clinical trial data collection and storage is quite impressive. Even the risk-averse clinical trial sites are now comfortable departing from recording clinical trial data on pieces of paper to the use of technology-driven data collection solutions. As a result, new requirements must be put in place to ensure data quality and integrity (DQI). At the same time we must avoid adding new complexities to the clinical trial paradigm that attempt to solve problems that do not exist. We suggest referencing an article entitled "When Should the Audit Trail Begin? co-authored by the eClinical Forum, published in 2021, as a particularly good start.

 The article addresses issues such as evaluating software that has the capability to track keystrokes prior to hitting “submit” in an electronic data capture (EDC) system. The authors concluded that if companies or regulators would recommend this as a requirement, it would add additional workflow data for analysis without any proof of value to DQI, or the ability to address any perceived malfeasance. What all this means is that we have be cautious when instituting any new quality associated technology so as not to add requirements/features just because we can. The pharmaceutical industry and regulators must also not use paper-based processes as gold standards as they critically review the root reasons behind any new rules. Any new rules must be clearly documented why they are indeed necessary and how they add to DQI.

The historical record of writing things down started around 3100 BCE when people started to write on clay tablets.

 Around 3000 BCE, the Egyptians begin writing and painting on papyrus, a material prepared from the stem of the water plant Cyperus papyrus. Ordinary leather in the form of parchment began to be used for writing around 2500 BCE. During the second century BCE, a better form of parchment was introduced such that both sides could be written on, but it was not until 400 years later that parchment rivaled papyrus. According to Chinese tradition, paper as we know it today was invented in the year 105 CE by Cai Lun, a eunuch who resided at the imperial court. Fragments of this paper product, made from rags and the fibers of mulberry, laurel and Chinese grass, are currently available in museums. Nevertheless, parchment was the standard writing surface for European scribes up and until the 15th century. Historically, for the Jewish bible, a Torah or biblical scroll could only be written on parchment from the skin of a kosher animal, and the five pages of the U.S. Constitution as well as the Declaration of Independence, the Bill of Rights, and the Articles of Confederation were all written on parchment.

A detailed summary of clinical trials that were conducted between 562 BCE and 1537 was published in 2010.

 It should be noted that during these early times there were no regulators other than kings and curious researchers, and no papyrus or parchment records were being checked by clinical research associates (CRAs). According to the article, the world's first clinical trial is recorded in the Bible’s “Book of Daniel.” This experiment was conducted by King Nebuchadnezzar of Babylon who conquered Jerusalem in 586 BCE and destroyed the Jewish Temple. During his rule, there was a time when all subjects were ordered to eat only meat and drink only wine to keep them in sound physical condition. However, when several of the king’s relatives who preferred to eat vegetables objected, the king allowed them just to eat legumes and drink water for 10 days. When the experiment ended and those eating vegetables appeared better nourished than the “meat-eating controls,” the king graciously permitted the vegetarians to continue on their diet. According to the article, this may have been the first time in the evolution of humans that an open-label clinical trial guided a public health decision.

More recently, the first “controlled” clinical trial was performed by Dr. James Lind (1716-1794), who while working as a ship’s surgeon, was appalled by the high incidence and mortality of scurvy amongst sailors. As a result, he planned and executed a comparative trial of the most promising cures for scurvy. His insightful and vivid description of the trial covers the essential elements of a controlled clinical trial. Dr James Lind's “Treatise on Scurvy” was published in Edinburgh, Scotland in 1753.

Early computers had a type of disk storage composed of a thin and flexible disk using magnetic technology. The original “floppies” stored 384K of data. Later, 720K (double-density) and 1.44MB (high-density) disks were developed. In the mid-80’s, Dr. Fred Wolfe’s rheumatology clinical research unit in Wichita, Kansas had electronic records stored on a computer with backups on floppy disks. Dr. Wolfe had a plan to ship the floppy disks directly to the data management department for processing at the pharmaceutical company where the senior author of this article was running the clinical trial. This could have been the beginning of direct data capture and the elimination of source data verification (SDV). Unfortunately, the idea did not gain acceptance and was abandoned.

Evolution of paper-based data collection in clinical trials

Until the onset of the use of EDC systems in the 1990’s, clinical trial case report forms (CRFs) were composed of three-part NCR (no carbon required) paper with the original copy remaining at the site, and the other copies submitted to the sponsoring pharmaceutical company. The capture of clinical trial data directly onto the paper CRFs was not permitted. Instead, an independent original source record was to be maintained at the clinical research site and the basic function of the CRAs at the “site visit” was to compare the precision and accuracy of the CRF data transcribed from a source record onto a paper CRF, a process defined as source data verification (SDV). In order to complete the data validation process, the paper CRFs were then sent to the pharmaceutical company for data entry into the study database by two separate individuals, a process known as double-key data entry. These data were then compared and any inconsistencies resolved.

Once EDC systems were introduced, paper CRFs were eliminated, and data entry into the study database was shifted to the clinical research sites. This process did not change the SDV workflow as it was still no longer certain that site staff always entered the data accurately into the EDC system. While the data within the databases of the EDC systems could have been used as source data, it was not initially accepted that data entered directly into EDC systems could replace the “source” and “transcribed copy procedures” adopted during the use of paper CRFs.

Figure 1: EDC data workflow when paper source records are used

The transition to direct data capture and eSource technologies

In 2011, a study evaluated the effects of SDV on the mean and standard deviation values of five variables when using an EDC system together with paper-based source records.

 The protocol was for a new treatment in men with lower urinary tract symptoms associated with benign prostatic hyperplasia. Data were collected from 566 subjects who signed informed consent and from 492 of these same subjects who were subsequently randomized. Results showed that post data validation, there were virtually no differences in mean values of the five key variables and a very minor decrease in the standard deviations.

In 2012, in lieu of initially capturing all source data using paper records, a US-based clinical trial evaluating the pharmacokinetics of a topically applied drug product allowed the clinical site to use direct data entry (DDE) into an EDC system at the time of the clinic visit.

 The clinical research team also implemented a risk-based monitoring (RBM) plan which defined the scope of SDV 1) if paper records were used, 2) the frequency and scope of online data review, 3) the role of centralized statistical monitoring (CSM) and 4) the criteria for when to perform in-person monitoring at the clinical research site. As a result of this novel approach to clinical research operations, there were1) no protocol violations as screening errors were identified prior to randomization, 2) minimal transcription errors as very few paper source records were used, and 3) major reductions in onsite monitoring tasks compared to comparable studies that used paper as source records. As needed, EDC edit checks were modified early in the course of the clinical trial, and compliance issues were identified in real time and corrected. From the safety perspective, there was rapid transparency and detection of safety issues. The clinical research site estimated that just in terms of data entry, it was able to save 70 hours of labor by eliminating paper as the original source records. The article concluded that when DDE and RBM are properly implemented, there could be major increases in productivity for both sponsors, clinical sites, and CROs, as well as reduced times to database lock and the statistical analyses.

Figure 2: Direct data entry/eSource workflow

In terms of the impact on query generation, an article published in 2014 reported the results of study that evaluated the use DDE at the time of the patient encounter on the effect of the query effectiveness ratio (QER), defined as the ratio of the number of queries leading to change divided by the total number of queries issued.

 The study looked at 38,207 forms in a phase 2 study for the treatment of migraine. Results showed that of the 1,397 (3.7%) of forms that were queried, only 782 (2.0%) forms required changes to the database. Of these changes, 294 (37.6%) were in the concomitant medication forms and none of the changes had any impact on the study results.

Traditionally, the major roles of the CRAs were to train the clinical sites on the steps needed to execute the clinical trial and then to assess, via SDV, how the clinical sites could accurately transcribe data from paper source records onto paper CRFs. Data management wrote programs to look at logical errors and inconsistencies among and between forms, and communicated these issues directly to the CRAs or via a query directly to the clinical research site. The number of data entry errors was often used as the main metric to see how well a site was executing the study. As a result, this approach focused more on process conformance rather than on protocol compliance. Unfortunately, due to this manual effort, by the time an error was identified by the CRAs or data management, the same error could have been repeated multiple times due to the time between data entry by the site and monitoring by the CRAs.

Currently with the application of DDE in various software solutions including EDC, Electronic Patient Reported Outcomes (ePRO), Electronic Clinical Outcome Assessment (eCOA), Electronic Informed Consent (eICF), Mobile Medical Devices, etc., there is no need to question the data as the system validation efforts at the time of data capture assure the data integrity. Real-time remote monitoring by CRAs and data managers can now focus on 1) protocol compliance, 2) illogical results/outliers, 3) high risk issues related to safety and efficacy endpoints, and 4) evaluation of potential protocol deviations and violations. As a result, DDE logic errors that are systemic and not user specific can be corrected rapidly and communicated to those performing data entry. It is also now possible to evaluate the performance of the CRAs in detecting issues.

However, if debunking “accuracy” is no longer highly critical in the data validation process as one usually assumes, then what is? The recently published ICH E8(R1) emphases the need for “completeness (lack of missing information)” and “consistency (uniformity of ascertainment over time).”

 Thus, it is reasonable to expect that narrow-focused / accuracy-specific edit checks will play a lesser role over time, while at the same time, the part of aggregate (“across subject”) and complex statistical checks will expand. Data management and clinical operations professionals should not only be ready for this paradigm shift, but lead the transition. This advancement will demand a shift in perception and processes and equally importantly, the development of new skills. Let us now recognize that “Data Science” is no longer exclusively the domain of statisticians, but is now associated with all clinical research operations dealing with data collection, validation, and monitoring.

It is now time to follow the new definition of Data Quality documented by the Clinical Trials Transformation Initiative (CTTI), and redesign the data validation processes from eliminating all errors to the elimination of errors that matter to decision making. It is also time to focus on potential errors which have a meaningful impact on the safety of trial participants, the credibility of the results and the potential impact on the future care of patients.

Technology is providing major contributions to the execution of clinical trials and has assisted in the improvement of DQI. However, there are still major challenges ahead to increase efficiencies, avoid redundancies, and ensure compliance with all aspects associated with data capture, data transfer and final reporting. The creation of lean documented processes and procedures will enable technology and human interactions to expedite clinical development, while at the same time protect the data and the basic principles that govern quality as documented in existing regulations.

With the implementation of DDE, RBM and technology solutions, we can now early in a clinical trial identify potential risks to the patients and to the study itself. Real time risk mitigation strategies must address those risks that are most critical for the evaluation of the safety and efficacy profile of the drug or device under study. Regulators can now shift their focus from how well a data point is transcribed from one medium to another, to issues that could put patients/study participants at risk and compromise the clinical trial outcome, such as lack of protocol compliance, incompetent or unscrupulous investigators and biased sponsors. It is now time to also avoid the profound effects on patients and sponsoring companies where studies fail due to lack of quality rather than lack of efficacy.

., is the President and CEO of THI Pharma Services Inc.

, BS, is the President and CEO of Fasor Technical Services, Inc.

, BE, is the Regulatory Affairs Manager for Videoc Technologies.

, MS., is the Faculty / Program Coordinator / Senior Lecturer, MS in Regulatory Science & Food Safety Regulation Programs, at Johns Hopkins University.

Sr. Director, Data Quality Science and Programming, at BioMarin Pharmaceuticals, Inc.

Mitchel, J., Sugiura, Y., Arnera, V. et al. 

When Should the Audit Trail Begin? Applied Clinical Trials

Papyrus: A Brief History – Dartmouth Ancient Books Lab

Evolution of Clinical Research: A History Before and Beyond James Lind. Perspectives in Clinical Research

Evaluation of Data Entry Errors and Data Changes to an Electronic Data Capture Clinical Trial Database

. Drug Information Journal, Vol. 45, pp. 421–430, 2011.

Mitchel, J., Schloss Markowitz, JM, Hua (Helen) Yin, et

. Lessons Learned from a Direct Data Entry Phase 2 Clinical Trial Under a US Investigational New Drug Application

. Drug Information Journal 46(4) 464-471 2012.

. Query Effectiveness in Light of Risk-based Monitoring (RBM). Data Basics Winter

Time to Change the Clinical Trial Monitoring Paradigm. Applied Clinical Trials

ICH E8 (R1) GENERAL CONSIDERATIONS FOR CLINICAL STUDIES 

https://database.ich.org/sites/default/files/E8-R1_Guideline_Step4_2021_1006.pdf

CTTI’s QbD Project. Recommendations available at: 

https://tracs.unc.edu/docs/regulatory/CTTI_Recommendations-Quality_by_Design.pdf

New requirements must be put in place to ensure data quality and integrity.

The Transformation of Clinical Trials from Writing on Papyrus to the World of Technology

At the October 2021 DIA conference on Digital Technologies in Clinical Trials, Pfizer reported the following timeline for the Pfizer-BioNTech COVID-19 vaccine
clinical program:

On May 4, 2020, the Phase I study was initiated, followed by initiation of the Phase II/III program on July 27, 2020. Positive interim results were reported on November 18, 2020 followed by the Emergency Use Authorization (EUA) submission request on November 20, 2020. The first Food and Drug Administration (FDA) EUA clearance occurred on December 2, 2020. The program, which evaluated over 46,000 subjects at approximately 150 clinical trials sites in United States, Germany, Turkey, South Africa, Brazil and Argentina, took just 8 months from first patient first visit (FPFV) to FDA clearance.

This major achievement by Pfizer, as well as the rapid development of products to prevent and treat the novel coronavirus disease 2019 (COVID-19) by others in the pharmaceutical industry and academic centers, are to be commended. Similarly, the FDA must also be commended on the speed that products have received EUA and full marketing approvals during the COVID-19 pandemic.

While clinical trials are costly and time consuming, they are necessary to assure that safe and effective products reach patients. As it is well known, the steps of drug development to approval involve multiple factors that could be improved in order to accelerate the process. The processes, resources and strategies to maximize efficiencies need to be revisited in order to continue exploring innovative study designs and methodologies that could result in reducing timelines and costs to product approval. On this topic, a recent article addresses the value of randomized clinical trials as well as the value of real world evidence together with novel approaches to statistical methodologies.

 Another recent article addresses the suitability of registries for embedding clinical trials.

Going forward, efforts need to be magnified to encourage biotech and pharmaceutical companies to continue promoting solutions to benefit of patients who are expecting solutions for their unmet medical needs. With this premise in mind, the current article first describes recent approaches to accelerate the marketing approval process, followed by a proposition of alternative strategies to accelerate the regulatory review approval process once it is concluded that an investigational product has passed the initial threshold of demonstrating safety and effectiveness in one or more clinical trials.

Current FDA programs to accelerate access

In order to accelerate the product approval process, four regulatory programs currently exist to reduce development and review times for products that address unmet medical needs for the treatment of serious or life-threatening conditions. In May 2014, FDA issued a Final Guidance for Industry entitled, “Expedited Programs for Serious Conditions–Drugs and Biologics” which addresses fast track designation, breakthrough therapy designation, priority review designation and accelerated approval.

In addition to this guidance document, under section 564 of the FD&C Act, 21 U.S.C. 360bbb-3, in a situation where the Secretary of Health and Human Services (HHS) issues a declaration of emergency or threat justifying authorization of emergency use for a product caused by chemical, biological, radiological or nuclear (CBRN) agents, as well as an infectious disease, the Commissioner of the FDA may authorize an EUA of an unapproved product or an unapproved use of an approved product. In 2004, the EUA authority was incorporated in the Project BioShield Act of 2004, which amended the FD&C Act. In January 2017, the “Emergency Use Authorization of Medical Products and Related Authorities” guidance was finalized.

Section 112 of the Food and Drug Administration Modernization Act of 1997 (FDAMA), entitled “Expediting Study and Approval of Fast Track Drugs,” mandates the facilitation of the development and processes to expedite review of therapeutics intended to treat serious or life-threatening conditions presenting with unmet medical needs. Fast Track designation, which also allows for the possibility of a “rolling review” of New Drug Applications (NDA) and Biologics License Applications (BLA), can be requested early in the development process at the time of Investigational New Drug (IND) filing or completion of Phase I studies. When a product receives a Fast Track designation, there is a strong emphasis on the importance of continuous communication with FDA so that questions and issues can be resolved quickly. Examples of diseases/therapeutic areas considered to be serious conditions include AIDS, Alzheimer’s disease, heart failure, cancer, epilepsy, depression, and diabetes, among others.

In contrast to Fast Track designation, a Breakthrough Therapy designation is assigned to an investigational product that is intended alone or in combination with one or more other products to treat a serious or life-threatening condition, and where there is preliminary clinical evidence that indicates that the investigation product may demonstrate substantial clinical benefit over existing therapies. Breakthrough Therapy designation requests usually occur at the end of Phase II. A drug that receives Breakthrough Therapy designation is eligible for all of the features of Fast Track designation.

The Accelerated Approval Program supports the concept that a treatment may be conditionally approved based on a validated surrogate biomarker or an intermediate clinical endpoint that strongly support the possibility of clinical benefit. Accelerated Approval agreements with FDA occurs close to the NDA or BLA submission. The regulation also requires that upon receiving Accelerated Approval, sponsors must perform a confirmatory study to demonstrate clinical benefit. If data from the confirmatory study does not demonstrate a clinical benefit, the product will be withdrawn from the market for use in that indication.

Under the 1992 Prescription Drug User Act (PDUFA), FDA committed to specific goals for improving drug review times. As a result, a two-tiered system of review times, Standard Review and Priority Review was created. A Priority Review designation means that the goal is for the FDA to act on a marketing application within 6 months of receipt, rather than a Standard Review of 10 months, as long as the investigational product can be shown to provide significant advantages such as:

Evidence of increased treatment effectiveness

Elimination or substantial reduction of a treatment-limiting adverse drug reactions

Documented enhancement of patient
compliance

Evidence of safety and effectiveness in a novel subpopulation of a disease

FDA informs the applicant the status of a Priority Review designation within 60 days of the receipt of the original BLA, NDA, or efficacy supplement.

Current FDA approaches to accelerate clearance of medical devices

In 1997, in order to improve the efficiency and timeliness of the 510(k) process of certain low-to-moderate risk new medical devices, the Accredited Persons Program was included in FDAMA. In 2018, a slide deck was published by FDA that describes how it is updating its 510(k) Third Party (3P) Review Program in order to avoid the routine re-review of 510(k) submissions already reviewed by a 3P Review Organization.

 As a follow-up, a March 12, 2020 final guidance document was issued entitled “510(k) Third Party Review Program Guidance for Industry, Food and Drug Administration Staff, and Third Party Review Organizations.”

 This guidance document provides medical device manufacturers with a voluntary process where FDA-authorized Third Party Review Organizations (3P510k Review Organizations) are permitted to evaluate and recommend approval or rejection of new devices. The guidance also delineates how decisions are made if certain medical devices are eligible for review under the Third Party Review Program. This new authority had been codified in the FDA Reauthorization Act of 2017 (FDARA) and allows discretion as to which devices are eligible for third party review. Approximately 50% of 510(k) applications now fall under this program.

The following section addresses proposed changes to existing FDA rules and regulations using a parallel format of the four phases of clinical trials during drug development. However, before moving forward, the are questions of special interest that should be addressed:

Under whose authority would it be to propose and implement these changes?

Would these proposed changes all fall within the jurisdiction of FDA rules and regulations, or would some require legislative action such as modifying the FD&C Act?

Based on lessons learned from the product approval process during COVID-19, and as previously published over 20 years ago, it is critical that sponsors continue to interact with FDA to discuss their development plans.

 In addition, FDA should expand the scope of exempt studies falling under IND and Investigational Device Exemption (IDE) regulations, and thus allow sponsors, with FDA concurrence, to proceed to clinical trials without FDA clearance of an IND or IDE. However, in order to control this process, each sponsor would be certified through an independent audit, that it complies with all relevant FDA rules and regulations, and training procedures are in place to guarantee that experienced staff manage all aspects of Good Laboratory Practices (GLP), and Good Manufacturing Practices (GMP) and Good Clinical Practices (GCP). The auditors will be certified by FDA based on their curriculum vitae and experience, and there will be an agreed upon fee across the industry, which will be paid by the company being audited. Each company will do yearly internal audits to assure continued compliance, and at the time of a marketing application, this process will be subject to regulatory review during preapproval inspections.

In addition, as is current practice, a clinical trial cannot be initiated without: 1) review and approval of the protocol and informed consent by an IRB or Ethics Committee; 2) review and signoff by an independent study Principal Investigator; and 3) review and signoff by each clinical site. Some academic organizations also have separate protocol review committees that must approve all new protocols.

From the data analysis perspective, the regulatory guidance ICH-E9 R1 is a key instrument to ensure more robustness and consistency during protocol development and statistical analysis planning. In order to minimize bias and increase the precision around the interpretation of the estimates associated with the clinical trial, the guidance forces trialists to provide more details about how the research questions, objectives, endpoints, and statistical analysis should be codified.

In terms of study designs, the discrepancies between the requirements and magnitude of the thresholds to demonstrate clinically significant safety differences, in most cases, is usually higher than the expectations for the efficacy profile. Thus, to have confidence that a study design is adequate to establish that an investigational product shows success both in terms of efficacy and safety in relationship with a comparator arm, the following issues must be addressed in all applicable phases of clinical development:

Clearly define the research question and hypothesis

Definition of the estimand and its attributes for the evaluation of all endpoints, including:

The endpoints and variables to be evaluated, and the validated instruments to be used for each case

Intercurrent events (ICE) that could impact the evaluation and interpretation of all endpoints, and the approaches to be used for each ICE

Population level summary for the evaluation of endpoints including the analysis dataset that will be used for each case

Sensitivity and supplemental analyses, for example:

Using the per protocol population or completers

Considering other missing data mechanisms different to the assumptions to the one used in the evaluation of the primary endpoint

Tipping point analysis under different scenarios

Supplemental analysis using different statistical models/methods

Testing strategies to control the familywise type I error

Assessments to capture the data associated with all endpoints using validated instruments

Description of the evaluation of the safety profile of drugs and devices including:

Treatment Emergent Adverse Events (TEAEs)

Treatment Emergent Serious Adverse Events (TESAEs)

Adverse Events of Special Interest (AESI)

Serious Unexpected Serious Adverse Reactions (SUSARs)

A general framework currently exists when FDA requests product approval recommendations from Scientific Advisory Boards for drugs and biologics and Third Party Review Organizations for devices. Thus, in order to expedite the review of marketing applications and reduce the time for a final marketing decision, in lieu of a detailed FDA review of all marketing applications, it is proposed that a framework be established for drugs, biologics and high risk new medical devices such that external expert committees, comprised of individuals certified by FDA, also be able to review the marketing applications and make product approval or rejection recommendations to the FDA review divisions responsible for final decision-making. However, in the event there are disagreements or basic questions that need to be resolved during the committee review, a process must be put in place for FDA, the expert review committee and sponsor to rapidly reconcile any and all issues.

For drugs and devices approved outside of the United States, the same processes should be followed by the expert committees, while taking under consideration any significant differences in regional differences in study designs and/or the population(s) studied.

Phase 3 - Post marketing safety monitoring

Currently, companies with FDA-regulated marketed products are required to track and report post marketing safety events either as expedited reports or as part of required annual reporting. Examples of the sources of these safety events include ongoing clinical trials, direct reports to companies of adverse events, data from electronic medical records, foreign data reports, the literature and data from patient registries and professional societies. Patient advocacy groups like the Cystic Fibrosis Foundation and professional societies like the Society for Thoracic Surgeons maintain patient registries.

 The Cystic Fibrosis Foundation registry tracks adverse events for all marketed and off label use of drugs for the treatment of patients with cystic fibrosis. In 1989, the Society for Thoracic Surgeons established the STS National Database to support the quality of patient care and to track safety events occurring during surgical procedures performed by their members. While some patient registries are being mandated by FDA to be used for safety monitoring, the use of patient registries should be expanded especially in rare diseases and in special patient population where there are active registries. However, if registries are mandated as a source of safety monitoring, the cost associated with this requirement should be reasonable and transparent so as remove any burdens, especially on smaller companies.

In the Food and Drug Administration Amendments Act (FDAAA; H.R. 3580) of 2007, Congress mandated that FDA track safety signals of marketed products. Subsequently, the Sentinel System was established in 2009 that uses electronic health data to assess post‐market medical product safety.

 The full Sentinel System was officially launched in February 2016. Through Sentinel, FDA assesses the safety of approved medical products including drugs, biologics, and medical devices. Sentinel’s data are based primarily on administrative claims and electronic health records (EHRs). The core Data Partner organizations include national and regional health plans partners, integrated delivery systems, and Medicare‐fee‐for service data. Sentinel partnerships also include EHR‐based organizations and networks, including CA Healthcare, the National Patient‐Centered Clinical Research Network (PCORnet), and several EHR data aggregators (TriNetX, Veradigm, IBM Explorys) who contribute data on a case‐by‐case basis.

There are currently over 70 million patients actively providing data to Sentinel. However, since there is a reported lag of 6-9 months from the date of health care and the data are only refreshed quarterly,

 we are pleased that FDA’s FY 2022 budget includes $82.9 million for an agency-wide data modernization and enhanced technologies initiative that supports medical product safety programs. These investments will modernize FDA’s data infrastructure by using current technology innovations that will allow FDA to more efficiently to gather data; identify, analyze, and respond to potential problems more quickly; and further improve review times for medical products.

Data from large national pharmacy chains can also be used to rapidly capture and report adverse events immediately upon product launch. For example, on May 20, 2021 CVS announced that their “CVS Health Clinical Trial Services is working with key stakeholders in the biopharmaceutical industry and across the clinical trial ecosystem to design and deliver innovative approaches to research and real-world evidence generation.”

While COVID-19 and previous pandemics have brought terrible devastation throughout the planet, government and societal reactions to COVID-19, for the most part, have demonstrated that it is possible to develop and release vaccines and other therapeutics in record times, and that humans have the ability and capability to collaborate for the common good. Regulators have also demonstrated that it is possible to accelerate the approval of much needed therapeutics in novel ways, and that if the proper resources are allocated both within and outside of governments, it becomes a win-win for citizens today and for all future generations.

Edwards, SJL, Bock, T., Palm, U., et al. The Case for Methodological Pluralism in Medical Science, The American Journal of Bioethics, 20:9,
39-41. 2020

Mikita, S., Mitchel, J., Gatto, NM, et al. Determining the Suitability of Registries for Embedding Clinical Trials in the United States: A Project of the Clinical Trials Transformation Initiative. Therapeutic Innovation & Regulatory Science. 2020

FDA: Guidance for Industry Expedited Programs for Serious Conditions – Drugs and Biologics. 2014

FDA: Emergency Use Authorization of Medical Products and Related Authorities. 2017

FDA: Eliminating Routine FDA Re-Review of Third Party 510(k) Reviews. 2018

FDA: 510(k) Third Party Review Program Final Guidance for Industry, Food and Drug Administration Staff, and Third Party Review Organizations. 2020

Mitchel, J. FDA Relations During Drug Development. Dialogues in Clinical Neuroscience. 2(3): 213–217. 2000

ICH: E9(R1) Statistical Principles for Clinical Trials: Addendum: Estimands and Sensitivity Analysis in Clinical Trials. 2021

Cystic Fibrosis Registry Patient Registry | CF Foundation. 2019

Society for Thoracic Surgeons: STS National Database. 2021

How Sentinel Gets Its Data - Sentinel Initiative. 2019

Cocoros, NM, Fuller, CC, Adimadhyam, S., et al. A COVID‐19‐Ready Public Health Surveillance System: The Food and Drug Administration’s Sentinel System. Pharmacoepidemiology and Drug Safety. 30(7): 827–837. 2021

Martinez, AI. Monitoring Medication Use During the COVID-19 Pandemic in the Sentinel System: The Case of Anticoagulation for Thrombosis. 2021

FDA Press Release: FDA’s Budget: Data Modernization and Enhanced Technologies. 2021

CVS Health introduces Clinical Trial Services | CVS Health. 2021

President, CEO, THI Pharma Services Inc., 

Faculty, Program Coordinator, Senior Lecturer, MS in Regulatory Science & Food Safety Regulation Programs, Johns Hopkins University

Lessons learned from FDA’s current process and proposed alternative review strategies.

Analysis: FDA’s COVID-19 Accelerated Pathways 

	With expansion of clinical research across the globe, organizations need to consider a wide range of regulations, and guidance documents from multiple regulatory authorities, in addition to the country specific laws. Currently, there is an extraordinary number of regulations and guidance documents pertaining to electronic clinical research data that have been issued from ICH, FDA and EMA, PMDA, and MHRA, with full or partial adoption into similar documents by other regulatory bodies.

	Sponsors and Contract Research Organizations (CROs) must interpret and attempt to follow them all if they are doing submissions in > 1 region. It is a daunting task. Each company must decide which are most constrictive/comprehensive and which are encompassed by others. In addition, these documents are of different “ages” and do not always agree with each other. They must also be interpreted in a way that is consistent with industry thinking, with backing to ensure a defendable position. It is imperative to garner this information in a practical way.

	There are no “universal” guidance documents or tools … but by combining like items, the eClinical Forum has offered a method to assist in this review.

	The eClinical Forum is a non-commercial, non-profit global organization with over 45 member companies active in regulated clinical research processes and technologies and has been involved since 2006 in the establishment of conformance criteria for electronic systems used in clinical research. The “eClinical Forum Requirements for Electronic Data for Regulated Clinical Trials” (or simply, “eCF Requirements”) is a list of succinct statements, all based on regulatory documents, that guide a user through this complex task. This is a significant accomplishment by the eClinical Forum’s “Regulatory Expert Group” (REG) as they have put in countless hours over several years to be able to produce such a comprehensive checklist for evaluating electronic systems, that will manage data used in regulated clinical research, against regulations and guidance from ICH, FDA, EMA, PMDA, and MHRA. This work is a result of the vast experience of the REG members who come from a variety of different eCF Member companies. They have spent hours debating each regulation or guidance and how to word the eCF Requirements to correctly meet the needs in the regulatory document.

	The eCF Requirements are available publicly and for free via the eClinical Forum website. They are based upon statements in documents prepared and issued by regulatory authorities that pertain to the design, development, implementation, and management of electronic systems that support clinical research data, as well as those statements that pertain to the handling of data that will be used in a regulated clinical trial.

	Each eCF Requirement has, as its basis, one or more statements from one or more of the regulatory authority documents. The eCF Requirements are in a linked .pdf file such that one can review the requirement and then the statements from regulatory authority documents used as a basis for the eCF Requirement.

	The eCF Requirements can be used to determine if systems, which originate, process, manage and/or retain data that will become part of a regulated clinical trial are consistent with regulatory requirements and recommendations. In particular, the eCF Requirements can be used to assist with self-assessment of systems, planning for system upgrades, writing proposals, Requests for Information (RFIs) and contracts, writing system requirements, writing system test scripts, etc. Depending on the type of system being evaluated, not all eCF Requirements may apply. It is up to the user of these eCF Requirements to review the underlying regulatory basis statements of any eCF Requirement to determine if it should be included in what the user is working on. When it is used to develop test cases, the user should review all statements from the predicate regulatory documents in the mappings to be sure test suites confirm all required items.

	There is also a separate tool called eSRA (eSource-Readiness Assessment), based on the eCF Requirements, which can be used for the evaluation of Electronic Health/Medical Record systems (EHR/EMR systems). It can be found on the eClinical Forum website in the eSRA tab. Sponsors are encouraged to use this assessment rather than their own custom assessment template such that sites can use the same completed eSRA for all of their sponsors.

	The eCF Requirement work is an active process. eCF continues to proactively evaluate and map new regulatory authority documents as they come into effect. It is our intent to release updated mappings to the eCF Requirements yearly. The newest version is released for eCF members only with the previous release provided publicly for free.

 'Determining if Data from Electronic Health Record Systems Can Be Trusted in a Clinical Trial Setting.'

 is the North America Facilitator for eClinical Forum, USA; 

 is the Quality Manager for PCG, Sweden; 

 is the Senior Director, Regulatory Risk and Compliance, Oracle, UK; 

 is the Manager External Data Integration for Bristol Myers Squibb, Belgium; 

 of the Department of Health Informatics, Rutgers the State University of NJ, USA; 

 is the Senior Compliance Manager for CSV, Boehringer Ingelheim, Germany; 

 is the Regulatory Policy and Intelligence Director for Novartis, Switzerland; 

 is the President of Target Health, USA; 

 is Scientific Advisor and General Manager for ERT Geneva, Switzerland; 

 is the Director of Quality Assurance for eClinical Solutions, USA; 

 is the Director of Electronic Systems Quality Assurance for Astellas Pharma Global Development, Inc., USA; 

 is a Quality Consultant for Eli Lilly and Co, Italy.

Keeping Up with the Regulatory Expectations

The FDA and other regulatory bodies have issued multiple guidance documents addressing technology tools that are used in clinical trials. FDA guidance documents include Computerized Systems;

 is also available online. In 2010, the European Medicines Agency (EMA) issued a Reflection Paper on electronic source records in clinical trials,

 and recently, the Medicines and Healthcare Products Regulatory Agency (MHRA)

 issued a document on data integrity. In 2016, the Chinese FDA issued Technical Guidelines on Electronic Data Capture for Clinical Trials.

 issued a document addressing the use of CDISC standards to facilitate the use of electronic source data in clinical trials.
	
	The regulatory push to the paperless clinical trial has occurred despite a pharmaceutical industry that has been risk-averse in adopting modern-day technology tools that could support clinical trials. This risk aversion is in part due to fear by sites and sponsors of receiving a FDA Form 483.

 What if the FDA turns down an application if it discovers that, for example, a patient was born in 1982, when in the study database it is recorded as 1983, even when there is no impact on the study results? The clinical sites are also fearful of losing business as a result of any FDA Form 483 finding, however minor.
	
	The major stakeholders in the clinical trial enterprise include study subjects/patients, sponsors, clinical researchers, regulators and the public at large. Now that the electronic world has entered the clinical trials space, as part of routine pre-approval inspections there will be an assessment of the impact of

 electronic systems on 1) informed consent; 2) data integrity and accountability; 3) protocol compliance; 4) monitoring of the study by the sponsor, including risk-based monitoring; 5) patient safety by assuring that all safety events are accurately collected and reported, and 6) drug/device accountability.
	
	The following are examples of regulated clinical trial software currently being used:

		Electronic patient reported outcomes (ePRO)

		Electronic clinical outcome assessments (eCOA)

		Dedicated tablets collecting and storing data in real time and transmitting data on a schedule to the study database

		Web-based systems collecting data in real time which transmit the data to the study database only after the source record is securely received and stored in an independent eSource storage location under control of the study site

		Mobile-based medical apps whose functionality could pose a risk to a patient’s safety if the app were to not function as intended


	
	It is strongly recommended that a risk-based approach to software validation be assessed for any medical software/app, even those not being used in clinical trials, as there are moral and ethical issues if any software used for medical decision-making does not act as intended.
	
	For software to be “reliable,” it needs to validated. Therefore, assuring that software is in a validated state is a key factor when regulators look at software products used in clinical trials. Fortunately, in 2002, FDA provided validation guidelines for software supporting regulated products.


	
	The following sections address some of the needed steps to assure that software used in clinical trials will be adequate to support regulatory scrutiny during pre-approval inspections and the review of marketing applications.
	
	


	
	Here are some critical questions to be asked concerning the “intended use” of the software, and depending on the answers, appropriate steps must be put in place.
	
	1. Why am I using the software?
	2. How is it being used?
	3. Will the data be used to support decision-making such as:

	4. Will the data be used to support primary and secondary outcomes?
	5. Will the data be used to support subject’s safety?
	6. How will patient privacy and confidentiality be maintained?
	
	


	
	A software product used in clinical trials must be validated and compliant with the Code of Federal Regulations (CFR) incorporated in Title 21 (Food and Drugs), Chapter I, Subchapter A, Part 11.

 21CFR Part 11, as it is commonly referred to, sets forth the criteria under which the FDA considers electronic records, electronic signatures, and handwritten signatures executed to electronic records to be trustworthy, reliable, and generally equivalent to paper records and handwritten signatures executed on paper.
	
	There is a distinction between validation and compliance. Validation is objective evidence which demonstrates that a given piece of software performs as intended against the user needs and the documented specifications and requirements.

	While it is true that software requirements and resulting features within the software can support compliance with 21 CFR Part 11, it is also true that no software or hardware system can be 21 CFR Part 11 compliant on its own. The software implementation, including processes and procedures for use of the system, are what constitute 21 CFR Part 11 compliance.
	
	And validation of the software goes beyond just the software requirements and resulting features necessary for Part 11 compliance. Validation is necessary for the entire software application and should be a defined part of the Software Life Cycle (SLC).
	
	


	
	There should be very clear SLC management documentation that contains the software development process from cradle to grave. In fact, many SLCs can go beyond the development process, to include ongoing support and even consideration for end-of-life for the software.
	
	In terms of popular development methodology choices, there is the Agile development methodology versus the traditional Waterfall approach. For the former, when it is very clear what the deliverable will be, one makes a detailed plan and then implements the plan. In contrast, the Agile approach accepts that there will be changes during software development and has built in flexibility to change direction at a moment’s notice.
	
	The first validation element is the software concept/requirements, or in other words, what is the problem we are trying to solve and why do we need this software? Usually an executive or the person driving the software writes the concept document to convey the mission and rationale of this new endeavor. Next, the “requirements” phase must clearly document what the software intends to do. For example, the software should allow for study subjects to enter study outcome data over the web using any device and any browser.
	
	Once the requirements are buttoned down, there needs to be clarity on how the requirements will be executed, which comes under the heading of “functional specifications.” For example, if the requirement is that “changes to the data base need to tracked,” then the functional specification could say that there needs to be an audit trail which must display the original value, the new value, why the change was made, who made the change, and what was the reason for the change.
	
	The next phase is left to the programmers and architects who create the very technical “design document.” This document basically is in “geek-speak” and we leave that document to the specialists. Once programming begins, the programmers work closely with the “testers” to test and debug the program until at some point the development team says the software is ready for a formal user acceptance test (UAT). In an Agile methodology, the design document, test plan and test scripts are written concurrently with development, iterating as features are refined. In a Waterfall methodology, the test plan and test scripts are often written with the design document, all of which is completed before development even begins.
	
	Once the “test plan and test scripts” are written and signed off, formal testing can usually begin right away. During testing, the testers document the “errors, bugs and fixes,” and when all that is completed, the software is “released” and signed off by those responsible and accountable. Post-release, “version control and change management” kick in. A plan is also put in place for the retirement of the software when it will no longer be supported or commercially available.
	
	


	
	A quality by design (QbD) methodology, which includes a risk assessment and risk mitigation strategies, is a critical exercise when building software. Using a Boeing 787 analogy, a plane has several electrical systems that control the engine and the coffee machine. We should not assign the same risk to the coffee machine not delivering hot coffee as we do to the engine failing on takeoff. Therefore, in a clinical trial, we propose the following for each risk:

		Assign a value of the risk to the study participant if a software failure happens

		Assign a value of the risk to the study itself if a software failure happens

		Assign a severity score if a software failure happens

		Assign a likelihood of detecting the software failure

		Then create a score and a risk-mitigation strategy for each risk


	
	A well-designed and well-written protocol can be viewed as one approach to an ideal quality manual. Therefore, in addition to standard protocol elements that we are all familiar with, the protocol should identify where and when electronic systems will be used, and then explicitly describe at a minimum, 1) all electronic systems being used; 2) the reason for their use; 3) how the systems will be used; 4) how the software will be controlled; 5) the basic elements of the risk assessment and risk mitigation strategies and; 6) data ownership and access controls.

	The inspectors will not usually do an assessment of the protocol design, as that was already done by the clinical team at the regulatory agencies. What the inspectors will do is attempt to determine whether 1) the protocol was followed; 2) there are SOPs supporting the protocol; 3) the site and sponsor followed what was stated in the protocol and SOPs; 4) the electronic case report form (eCRF) was intelligently designed to support protocol compliance; 5) there were system-generated management reports of site communications and protocol deviations/violations; 6) there was an integrated risk management plan and risk mitigation strategy; 7) there was a substantiated and complete root cause analysis of observed issues/events; and 8) there were follow-up corrective and preventative action plans (CAPAs) and that the follow-up actions were resolved as planned.
	
	From the safety perspective, the inspectors will want assurances that there was accurate reporting of safety events by evaluating 1) how safety data were collected; 2) the process for determining the source of original safety data based on a data source agreements; 3) a sample of source data from disparate sources, including review of source records written on pieces of paper, if relevant, and; 4) subject records maintained at the study site.
	
	

Are electronic source data reliable and fit for purpose?


	
	We are asked many times by those committed to paper records-sponsors or regulators-that if the system is fully electronic, how do we know that the data are real if there are no source records with which to compare the electronic record? Ironically, you can ask the same question about paper records; unless there are major controls of paper, notebooks, etc., nothing prevents a piece of paper from being destroyed and replaced by a new one. Bottom line, as some suspicious data trends are impossible to spot through manual review of records, 

fraud and dishonesty can usually only be identified through adaptable, validated checks and the use of informatics and multidisciplinary expertise. Clearly then, misconduct is not dependent of the collection medium of the original record (paper vs. electronic). Consequently, in a double-blind, randomized trial, one effect of fraud, if it occurs, could affect the study results by demonstrating that a potentially effective drug is ineffective from a statistical perspective (Type 2 error). This is where study monitoring techniques and fraud detection programs become vital.
	
	Therefore, when using off-the-shelf or proprietary software systems, the following are some of the quality questions that should be considered:

How to assure that software used in clinical trials will support regulatory scrutiny during pre-approval inspections and application review.