Check out the latest features and columns!

Applied Clinical Trials September 2022

Experts discuss solving the EHR-compatibility puzzle in point-of-care trials

‘Record’ Gains in Bringing Research Closer to the Patient

ACT's biennial Salary Survey takes a look at the future of industry following increased adaptation of decentralized technology 

Salary Survey: The Age of COVID

High inflation is rocking the markets, as interest rates are skyrocketing, and access to capital is dwindling. However, some small biotech companies are weathering the storm. in this interview, Robert Foster, CEO of Hepion Pharmaceuticals, will discuss strategies for being competitive in this harsh economic environment.

Moe Alsumidaie: What are the three most significant challenges that Hepion Pharma's clinical operations are facing, and how are they being addressed?

 The first significant barrier is funding for our Phase IIb clinical trial. We must keep a close eye on our spending to survive the trial. The second most difficult challenge is determining how long the study should last. Many other companies needed 12, 15, or 18 months to assess. However, based on the mechanism of action of our drug, rencofilstat, we decided that 12 months would be sufficient. This was one of the factors we considered when designing the duration of this trial, as the primary goal is to see if there is a decrease in fibrosis or a change in steatosis. Our third challenge is to enroll patients quickly. The sooner we can provide interim or final information to our shareholders, the better. One strategy we employed was increasing the number of sites, so we opened 121 sites. The third challenge is related to the first because the cost increases as the number of sites used increases. You must concentrate on the bottom line regardless of the size of your biotech company.

MA: What is the best way to balance funding, study operations, and patient recruitment?

 I've learned that if funds and financing are available to run your programs or business, you should use them. People frequently believe that accepting the money now will be dilutive, so they wait, hoping that the share price will rise. In contrast, my experience has taught me that predicting what will happen in the market is impossible. It would help if you also considered inflation and the global economy's strain. You can be caught in a downdraft regardless of your program’s effectiveness. As a result, despite the pandemic, we could accept a substantial amount of cash when the opportunity to finance arose at the end of 2020 and the beginning of 2021. The second point to mention is that we also hired exceptional talent. We hired a highly experienced chief medical officer from a large pharmaceutical company a year and a half ago. I've always believed that surrounding oneself with the most talented brightest people is beneficial.

MA: Hepion Pharma is conducting a placebo-controlled, double-blind study for the Phase IIb "ASCEND-NASH" trial. Why is this strategy so crucial for commercialization and competitiveness

 To ensure that the drug's effect can be accurately measured, FDA requires a placebo-controlled blinded trial. However, if you are interested in reducing fatty liver, one challenge involves lifestyle change, such as diet and exercise. Some people may be more likely to watch their diet and exercise when they begin a trial, and chronic liver disease and now NASH, are examined, there is frequently a high placebo effect. For instance, if your drug has a 25% effect, you must consider how much of that effect is due to the drug alone and how much could be due to a change in lifestyle. A placebo-corrected response is possible, and that is what we are aiming for here. FDA will require a pre- and post-trial biopsy to be competitive. We attempted to establish a "safety valve through a series of interim analyses." The Drug Safety Monitoring Board (DSMB) will evaluate rencofilstat's safety, tolerability, and effectiveness. They cannot tell us the efficacy of the trial because it is blinded, but they may be able to give us a general sense of whether they see compelling reasons to continue the study. We are testing three different dosages, and the DSMB may inform us that the low dose does not have many effects. Or they could say that high doses are risky, so you should discontinue either the low dose for lack of efficacy or the high dose for safety. We anticipate that reconfilstat will be effective at all three doses being tested.

MA: What are the benefits of directing AI POWR with multi-omics?

 Our drug classes are entirely different from our NASH drug development market competitors. Our proprietary artificial intelligence is known as AI POWR (AI). We received drug approval for lupus nephritis, a drug we discovered and developed at our previous company, in January 2021. AI and our efforts to make this molecule successful played a role in its success. We're taking the same approach at Hepion, but we're stepping up our efforts. The genome, transcriptome, proteome, lipidome, and metabolome are all studied. Millions of data points are generated for each patient when the multiome is combined with patient characteristics such as BMI, gender, and age. Then the effect of rencofilstat on the movement of all these different markers is examined. We are distilling this massive data set down to the elements that will tell us who will respond best to the drug. We can avoid the pitfalls that have plagued other companies with mediocre response rates, especially considering the placebo effect. And we're looking into ways to boost response rates to 70%, 80%, and 90% by prescribing our drug to suitable patients. Fatty liver disease affects one-third of the population in the United States, and we want to make our therapy available to those who benefit most from it.

MA: In Phase IIb, you have 121 sites and 336 patients.

 Yes, they will be divided into three dose cohorts. We divide dosages into three categories: low, medium, and high. The low dose is 75 milligrams, the medium dose is 150 milligrams, and the high dose is 225 milligrams. These three doses will be given once a day for a year. For this trial, we will recruit the majority of subjects with a NASH score of F3

MA: Have you considered incorporating decentralized clinical trial models into Hepion's Phase IIb?

 I prefer a more conservative strategy in a more controlled setting. Because patients are frequently quite ill with liver disease, so returning to clinics is highly inconvenient. We want to ensure that our trials are as precise as possible when comparing the placebo response to the drug treatment response.

MA: At the moment, inflation is causing capital markets to tighten. Is this a challenging situation for a pharmaceutical startup?

 Yes. It impacts access to capital, and many businesses are in limbo. At the same time, there are a lot of great deals on the market right now. This is something I've been doing for about 35 years. I've seen these cycles come and go, and I know they always come back. Fortunately, we invested during a period when the market was highly volatile—excited and eager to invest in companies like ours. Given the strength of our cash position and the tenacity of our team, we will do everything we can to be resourceful and weather the storm. Hepion is an excellent company with unique action mechanisms. We have an oral drug that directly targets or affects fibrosis. It is highly safe and well-tolerated, and the cost to produce the drug is affordable.

Articles

Robert Foster, CEO of Hepion Pharmaceuticals, discusses strategies for being competitive in the current economic environment.

Start-Up Pharma Weathering the Inflation

Emmes announced that it has acquired Clinical Edge. Clinical Edge trains and certifies visual function examiners at investigator sites conducting Phase I to IV ophthalmic clinical trials.

The combination of Optym, Emmes' ophthalmology certification unit, and Clinical Edge will increase the origanization's skillsets in ophthalmic certification and training services.

Company's fifth acquisition will advance its ophthalmology skillset and enhance global reach.

Emmes Acquires Clinical Edge

Caris Life Sciences announced that Indiana University Melvin and Bren Simon Comprehensive Cancer Center has joined the Caris Precision Oncology Alliance™ (POA). The POA is a growing network of cancer centers that collaborate to advance oncology and biomarker research. 

Established in 1992, the IU Simon Comprehensive Cancer Center is Indiana's sole National Cancer Institute (NCI)-designated comprehensive cancer center. It conducts all phases of cancer research, from laboratory studies to clinical trials to population-based studies.

IU Simon Comprehensive Cancer Center joins Caris' network of cancer institutions committed to utilizing clinical data to advance patient care and outcomes.

Indiana University Melvin and Bren Simon Comprehensive Cancer Center Joins Caris Precision Oncology Alliance

Quanticate and Cancer Research UK have launched the DETERMINE study which will test a range of therapies specifically targeting key genetic changes in cancer cells.

The DETERMINE study will enrol patients with rare cancers, and aim to find out whether existing licensed drugs could be used to treat their type of cancer. Cohorts will be formed of patients who have an identifiable genetic alteration in their cancer that can be targeted by treatments already approved for use in other cancer types.

Study will test a range of therapies specifically targeting key genetic changes in cancer cells.

Quanticate and Cancer Research UK Launch DETERMINE Study

Jill Wechsler is ACT's Washington Correspondent

As the deadline nears for enacting legislation to renew user fees to fund significant FDA operations, legislative leaders are looking to approve a bare-bones measure as part of a short-term government funding bill. Congress must enact a continuing resolution (CR) by Oct. 1, to prevent a federal government shutdown, and the legislators now have agreed to include a five-year fee reauthorization in that measure. Such a continuing resolution (CR) for fiscal year 2023 is expected to run through Dec. 16, after the upcoming mid-term elections.

A key point of debate is whether to include in the CR package provisions to continue and revise a number of FDA programs also slated to expire next week. These and a number of broader reforms were included in the user fee reauthorization bill that was approved by the House months ago and has been pending in the Senate for weeks. It would make significant changes in FDA programs and operations, including policies for how FDA regulates diagnostics, dietary supplements, and cosmetics.

Also off the table is the proposed Cures 2.0 legislation, which was pulled from the main user fee package due to Republican opposition. That measure supports and revises a range of programs to advance biomedical research, but now its fate is uncertain.

Without reauthorization of new fees for drugs (PDUFA), generic drugs (GDUFA), biosimilars (BsUFA), and medical devices (MDUFA) before the current fee programs expire next week, FDA will lose its authority to collect fees from manufacturers, along with requirements for meeting certain timeframes and for utilizing agreed-on policies for assessing and approving new drugs and medical products.

FDA also would have to lay off thousands of staffers who are supported by fee revenues of potential layoffs, a prospect that has eroded agency morale and undermined efforts to recruit and hire needed employees. The agency has struggled for months to maintain and expand its professional workforce in a very competitive market for skilled biopharma experts, and the agency’s funding uncertainty has worsened the situation.

Medical product user fees generate about $2 billion in agency revenues, about 40% of FDA’s non-tobacco budget. While Congressional leaders on both sides of the aisle support the agency’s programs as important for ensuring the health and safety of Americans, the broader political debate over government funding has erected added hurdles for timely reauthorization of new user fee programs. And some consumer advocates have broadened their criticism of expanded user fee funding of FDA by regulated industry as leading toward bias and undermining agency decision-making. FDA Commissioner Robert Califf has said that the agency has sufficient carryover funds to maintain operations for about five weeks into the new fiscal year that begins Oct. 1, but FDA still will have to notify more than 3,000 staffers of “impending employment actions” without user fee reauthorization by Congress.

Seek pared-down measure in funding bill to avert shutdown.

Congress Slated to Reauthorize User Fees, Without FDA ‘Improvements’ 

The clinical trials industry depends on internet surveys for competitor analysis, customer satisfaction/feedback, and benchmarking.

 The results of these surveys provide insights that guide business and clinical decisions. Like any information from the web, however, internet surveys are prone to noise and misinformation.

In this article, we will show you how to improve the quality of your internet survey data by identifying these sources of the noise and misinformation using careless responding analysis. A fundamental cause of the many forms of internet misinformation is carelessness.

 People who pass along misinformation on the internet are more lazy than biased: They respond to survey items without carefully reading the survey item or thinking deeply about their response.

 The result is careless responding that degrades data quality.

Careless responding occurs when a respondent is within the sampling frame but does not give an accurate response due to inattention or inadequate cognitive processing.

 More specifically, careless respondents may not carefully read and comprehend survey items, retrieve the appropriate information from memory, make a deliberate judgement, or provide an accurate response that aligns with that judgement.

 Carelessness may be induced by the survey structure (e.g. a long survey with complex, poorly written items), respondent characteristics (e.g., fatigue or limited reading ability), or contextual factors (e.g., distractions or responding on a smart phone).

 Any of these factors can lead to inaccurate responses due to carelessness.

Careless responding is biasing a lot of the internet surveys.11,12 Up to 30% to 50% of survey responses are careless,

 but carelessness begins to bias results bias when about 10% of the respondents are careless.

 The specific effects of careless responding is that it attenuates means and correlations, decreases the reliability of measures, reduces validity, and increases the risk of Type 2 error.

 Put another way, careless responding creates a lot of noise and obscures your ability to detect a signal in the data.

Identifying careless respondents through survey construction

Survey construction can help deter careless responding and enhance your ability to identify careless respondents. In this section, we describe a variety of tools that can be built into a survey as careless responding detectors. You may not need every tool but should adjust how you use these tools based on the purposes of your research.

. Build into the survey some items that will allow you to assess the qualifications of the respondent. This step is analogous to specifying the inclusion/exclusion criteria in a clinical trial. Items that are constructed as continuous variables (e.g., years of experience or ‘rate your expertise in this area’ on a 7-point scale) are more useful than dichotomous response items (e.g., ‘are you qualified’ yes/no). These qualifying items don’t need to go up front in the survey, even though you may think about them first. They may end up at the back of the survey with other demographic items.

 Most of the common survey tools will allow you to measure the amount of time it took to complete the survey, but not the specific time-per-item. Still, you can calculate the average time-per-item. As you are writing items, consider the complexity of the questions and estimate the time respondents should need on each item. Respondents often take less time than you think, but this estimate will give you a starting point for analyzing the amount of time taken, which we address in the next session.

 Carelessness increases significantly as survey length increases. The rates of random responding doubled toward the end of long surveys.

 It is a good rule-of-thumb to keep the number of items to less than 150 items. This includes all the demographic, qualification, and careless detection items. Reduce the number of items if you have long instructions or complex items. We realize the organizational pressures that lead to long surveys. Have a strong governance structure in place beforehand to keep the survey focused on the research question.

 You can improve the number of respondents with generous incentives, but you might also attract careless respondents. Think about your target respondent and see if you can identify something that will be uniquely valuable to them while not attracting the attention of professional surveyists.

. Typical basic instructions emphasize honesty and accuracy. Adding a warning (e.g. ‘your responses will be analyzed for quality’) will reduce carelessness.

 An occasional reminder, whether written or through a virtual presence, can boost the effect of warning instructions. The decision to assure respondent anonymity is tricky. When respondents enter their name, carelessness is reduced. But identified respondents can become overly agreeable—tending to agree with items even when the agreement is not logical.

. These are items not related to the purpose of your research but are included to detect careless responding. These items increase cognitive engagement by warning respondents to be careful to look out for tricks.

 For example, you can include instructed response items (e.g., ‘fill in a 3 here’) to detect carelessness.

A different type of careless detection item is to create pairs of items in the survey that ask the same question. The simplest approach is to repeat the item later in the survey to see if you get the same answer. You can also change the wording with synonyms or antonyms to check agreement. You could also reverse-coded the repeated items. In general, items are scaled so that a higher number means that there is more of the construct or that it is better. In reverse-coding, the scaling is flipped, so that a rating of more or better requires a lower number. Reverse coding is a good way to detect carelessness but they can become so convoluted that they lead to misresponses, or honest mistakes that comes from poorly worded items.

 When using careless detection items, it is good practice to include them early in the survey to serve as a speed bump or warning and then every 50 – 75 items, with a maximum of 3 reverse-coded items per survey.

Identifying careless respondents in the analysis

In this section, we will discuss simple analytic procedures that you can use to detect careless respondents. When you find a careless respondent, don’t delete them from the dataset. Instead, create a 0/1 categorical variable where a 1 signifies a careless respondent. Someone may ask you to do a post hoc analysis to understand the effect of removing the careless respondents from the analysis. Whether you should comply with such a request is a different story. Carelessness is a type of data contamination.

 These respondents don’t belong in the sample so a request to examine the effects of including these respondents is the same as asking ‘What is the effect of including respondents outside the sampling frame?’ Still, it is good to retain the data in case you need it.

The definition of careless responding rests on the assumption that the respondents are qualified to be sampled.The qualification analysis will vary based on how you structured the survey and the burden of proof you specified in the SAP prior to launching the survey. If you have created the qualifying variable(s) as continuous, examine histograms to assess how many respondents qualified themselves as completely unqualified (e.g., a ‘1’ on a 7-point scale) and then unqualified. Look at a frequency or histogram to see how the respondent’s cluster and identify a logical cutoff point. It may help to create a categorical variable out of the continuous variable because it helps to make the data presentation more understandable.

 Hopefully, though, you don’t need to categorize this variable because all your respondents were highly qualified.

 Ideally, you want to know the amount of time the respondent spent 

 Most of the commercially available survey tools don’t provide this information. Instead, you usually see the survey start and finish times. From this, you can calculate the total amount of time taken on the survey and then an average time per item.

What is too fast and is therefore eligible to be considered careless? We don’t have a fixed speed limit. First, we sort the data from shortest to longest and look for a breakpoint between those respondents carelessly speeding through the data and the careful-but-fast-readers. If you don’t see it, try graphing the time-per-item variable. We usually see a break around 3 seconds-per-item, with an average time-per-item of around 8 or 9 seconds. If you think about all the steps involved in responding to an item (reading, accessing memory, judging, and responding), a respondent is clearly not being careful if they are averaging less than 2 or 3 seconds.

What about respondents who take a very long time on the survey? We commonly see respondents who take a couple of days to complete the survey! Obviously, these people left the survey and came back to it or abandoned the survey and closed their browser tab later. We usually can also see a break point in the histogram on the long side of survey time, usually less than an hour. We look at the long responses and often find they were abandoned, but occasionally see someone who was just interrupted and seems to be responding carefully. Don’t be too hasty to label a long-respondent as careless. Check each one of these long respondents to assess the quality of their responses. We usually see the point where they left the survey because the quality of the responses suddenly gets much worse.

. The end of the survey typically has some innocuous items like demographics that everyone should be able to respond to. Sometimes careful respondents get interrupted by an important phone call and never get back to the survey. Most of the time, however, we find abandoned surveys are careless. Also check the number of missing data points. Again, this is helpful, but not definitive. You might have a highly qualified respondent who is a specialist and only responding to some topics within their area. Most cases that look like they were abandoned with a lot of missing data, however, will probably be careless respondents.

. Careless respondents often default to a pattern of respondents instead of reading and evaluating each item. The most common pattern we see is monotonic responding (e.g., 3, 3, 3, 3...). Less commonly, respondent will alternate their responses (e.g., 3, 4, 3, 4....) or use a progression (e.g., 1, 2, 3, 4, 1, 2, 3, 4...).In a small dataset, you can just scan the matrix for patterns. If it is a larger dataset, screen each respondent for standard deviation and frequency. The standard deviation will allow you to pick up monotonic responses and the frequency will identify progressions.

. If you have instructed-response items, check those first. Then you can check the agreement between the repeated items, synonyms, and antonyms to detect inconsistency. If you have blocks of similar items (i.e. loading on a single construct), you can check the even-odd correlations.

 What is an acceptable level of agreement or correlation? Depending on the burden of proof you are using—addressed in the next section—we recommend levels of at least .5 to .75. A cutpoint of .5 is a low burden of proof—it is a coin flip. A cutpoint of .75 seems more reasonable. At this level, the shared variance means that the items share more variance with each other than other nonrelated items.

 These consistency items will help you identify the most subtle cases of carelessness.

Imagine that you conduct your analysis and find some evidence of possible carelessness. What should be your burden of proof? In other words, how strong does the evidence need to be and how much certainty should you have to declare a respondent careless? The standard that you apply should depend on the type of research that you are doing and is analogous to how courts require a different level of proof in trials. In exploratory research, the standard for carelessness should be quite high, meaning you should only label obvious cases as careless. This is analogous to how courts require proof beyond a reasonable doubt in criminal cases. The prosecution must show there is no other reasonable explanation for the evidence. In exploratory research, you can tolerate some noise in your data because you are just trying to identify the issues involved with this research question. You want to identify all the relevant issues, so you are willing to accept some carelessness to get more honest, open, and free-wheeling responses for an exploratory research question.

Descriptive research requires a higher burden of proof. In descriptive research, you are observing what is happening and making estimates of how often it happens.

 Noise in your data from careless responding begins to obscure your observations. As a result, you need to be more sensitive to careless responding, like how courts apply the standard of clear and convincing evidence, used in many family or administrative law cases. Clear and convincing evidence requires proof that the evidence is highly likely to be true. In descriptive research you do not to be 100% convinced, but it seems highly likely that they were careless.

Causal research demands the lowest burden of proof. With causal research, you must be very sensitive to carelessness because you are trying to explain 

 something happens. These causal mechanisms are subtle and easily obscured by data in the noise, so you should use the lowest burden of proof to establish carelessness. This is analogous to the burden of proof in civil lawsuits, where you only must be convinced that it is more likely than not. You still need meaningful evidence that they were careless but being ‘mostly sure’ is enough.

Although the clinical trials industry depends on internet surveys, these studies are prone to bias from careless responding. This article describes how clinical trials executives can clear out the noise from careless respondents by how they structure the survey and conduct the analysis.

Removing careless respondents might make trained statistical analysts uncomfortable. They have received many warnings against data manipulation and case deletion. But removing careless respondents is not case deletion or even deleting extreme outliers. Careless respondents are a type of data contamination consider carelessness a type of data contamination and “...contaminated observations can and should be minimized by careful research procedure and data preparation.”

 Removing careless respondents is good research practice, is not research misconduct, and is ethical.

We realize this message goes against the current data culture where larger datasets are considered better. In this view, if you have a dataset of 200 responses to a survey that is noisy, the best solution is to increase the number of responses to 1,000 or even 10,000. In our view, a better approach is a careless responding analysis that might cut the n to 100 to obtain a clear signal from the analysis. In this way, careless responding analysis can enhance research efficiency. Rather than focus on the size of your dataset, we argue that the care with which you collect and analyze data is more important.

Clinical trial strategy and operations must rely on internet surveys if it is to remain responsive to industry changes and patient needs. Noisy data can create misinformation about industry competitor analysis, customer satisfaction, or patient outcomes. Careless responding analysis is a way to clear out the noise in survey data to get a clear signal in your analysis.

, PA-C, MBA, PhD, Clinical Professor, LeBow College of Business, Drexel University, and 

, MBA, Senior Director, Market Development, Phreesia, Inc.

Dumais, K., & Raymond, S. (2021). Considering Patient Burden in Oncology. 

Getz, K., & Kim, J. (2022). Measuring Patient Satisfaction as a Primary Outcome for Patient-Centric Initiatives. 

Henderson, L. (2022). Salary Survey: The Age of COVID. 

Pennycook, G., & Rand, D. G. (2019). Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning. 

Lazer, D. M., Baum, M. A., Benkler, Y., Berinsky, A. J., Greenhill, K. M., Menczer, F., . . . Rothschild, D. (2018). The science of fake news. 

Meade, A. W., & Craig, S. B. (2012). Identifying careless responses in survey data. 

Ward, M. K., & Pond III, S. B. (2015). Using virtual presence and survey instructions to minimize careless responding on Internet-based surveys. 

Swain, S. D., Weathers, D., & Niedrich, R. W. (2008). Assessing three sources of misresponse to reversed Likert items. 

Huang, J. L., Curran, P. G., Keeney, J., Poposki, E. M., & DeShon, R. P. (2012). Detecting and deterring insufficient effort responding to surveys. 

Parush, A., & Yuviler-Gavish, N. (2004). Web navigation structures in cellular phones: the depth/breadth trade-off issue. 

International Journal of Human-Computer Studies, 60

Barge, S., & Gehlbach, H. (2012). Using the theory of satisficing to evaluate the quality of survey data. 

Tuten, T. L., Urban, D. J., & Bosnjak, M. (2002). Internet surveys and data quality: A review. 

Nichols, A. L., & Edlund, J. E. (2020). Why don’t we care more about carelessness? Understanding the causes and consequences of careless participants. 

International Journal of Social Research Methodology, 23

The signal and the noise: Why so many predictions fail-but some don't

Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). 

Internet, phone, mail, and mixed-mode surveys: The tailored design method

Berry, D. T., Baer, R. A., & Harris, M. J. (1991). Detection of malingering on the MMPI: A meta-analysis. 

Baumgartner, H., Weijters, B., & Pieters, R. (2018). Misresponse to survey questions: A conceptual framework and empirical test of the effects of reversals, negations, and polar opposite core concepts. 

Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2013). 

Applied multiple regression/correlation analysis for the behavioral sciences

Johnson, J. A. (2005). Ascertaining the validity of individual protocols from web-based personality inventories. 

Netemeyer, R. G., Bearden, W. O., & Sharma, S. (2003). 

Scaling procedures: Issues and applications

Babbie, E., & Mouton, J. (2001). The practice of social science research. 

Clinical trials must rely on internet surveys to incorporate the needs of industry and patients, but how can results be improved?

Anticipating Careless Responders in Survey Design and Analysis

The clinical trial landscape has changed significantly post-pandemic, and many industry experiences and strategies emerged at DIA 2022. in this interview, Barbara Lopez Kunz, president, and global chief executive at DIA, discusses key themes on the current state of the clinical trials industry and its future.

Moe Alsumidaie: Post-pandemic, what are the most significant changes in drug and clinical development?

 We've all seen drug development evolve over the past few years, but the pandemic catalyzed some of the things we've discussed. It turned our talk into action, which we love, and hope will continue.

As the pandemic spread, we all faced the same challenge. As a result, international collaboration increased, and as part of this international exchange, formal work-sharing initiatives and dialogue emerged, and there is a strong desire to use the pandemic response's flexibility and innovation to address unmet needs in other disease areas. This has been DIA's mission for nearly 60 years.

We’ve seen a few emerging trends. First, we see a sharper focus on including diverse patient populations in clinical research and development so that the solutions are effective for those who need them most. Pandemic urgency helped integrate clinical research into communities that may not have felt welcome or understood how to participate. While diversity inclusion has made strides, we still need to improve our outreach and integration of all communities.

Second, there is expanded collaboration; since it doesn't matter how effective therapies are if patients can't access them, DIA has brought in new players. Pharmacy benefits, reimbursement, health technology assessment, and other experts engage earlier in the drug development lifecycle so all perspectives work together.

Last, decentralized clinical trials have emerged quickly and are here to stay. Tufts University's Center for the Study of Drug Development showed that decentralized processes increase drug development efficiency. Our DIA Global Annual Meeting 2022 program included many DCT themes.

MA: What are some companies' DCT implementation challenges?

 DCT adoption is a significant change, requiring the proper methods, tools, and training. DCTs will bring new challenges, such as ensuring appropriate training and delivery across sites and improving data collection and management. Until new approaches are solidified, drug development will use proven methods. As in change processes, some companies will move forward with decentralized trials while others wait for a larger mandate, monitoring understanding, process, training, and regulatory guidance. Clinical trials are the most time-consuming and expensive part of drug development, so process changes should reduce risk.

MA: Post-pandemic, what regulatory changes have you seen?

 Regulators want to keep pandemic-era flexibilities. Regulators are accelerating changes in structures and processes to be more patient-focused. I'm seeing more patient engagement and patient-centric drug development now.

In the US, the FDA published guidance on patient-focused drug development, including collecting patient input, identifying patient priorities, and developing meaningful outcome assessments. In Europe, the EMA endorsed the PREFER initiative's patient preference, integration, and drug development framework. The ICH is working to harmonize data submission requirements for patient experience. Patient-centricity is entering countries' regulatory agendas. Asia, the Middle East, and elsewhere are seeing new patient engagement communities. DIA has long supported these efforts, and we're actively generating new patient engagement knowledge and learning opportunities for our community.

New data collection and analysis pose challenges throughout the drug development pipeline. Patient privacy, data ownership, and data integration will determine how we use these data to drive R&D efficiency now and in the future. The EMA and HMA (Heads of Medicine Agencies across Europe) maintain a collaborative data steering group to easily integrate big data and advanced analytics into their regulatory assessments. Signal detection uses analytics to automate case adjudication and identify false claims, adverse events, and illegal marketing. Darwin-EU enables real-world data analysis and post-market surveillance and decisions. The FDA has seen a rise in AI/ML-related submissions in the US.

MA: Where do you see clinical development headed in an uncertain world?

 We're all in transition. New covid variants are emerging, and we're preparing a response. In addition to the pandemic, geopolitical issues, war, and unusual weather are causing health-related crises. We've made great strides in healthcare, and I'm proud of that, but we must continue collaborating and sharing our findings. We must create clinical trial designs that address our challenges. We must better prepare for collateral challenges, such as supply chain issues. Many companies and regulatory agencies must stabilize their supply chains to meet demand and launch new products. Global crises will persist. As a neutral convener, DIA will continue to address these issues. And we'll keep putting patients first as a healthcare community. DIA's sharing and learning environment fosters ideas and creates momentum and practical ways to implement them. As new knowledge emerges, we revise and refine our learning offerings to ensure they offer the latest regulatory, clinical, and data science content our community needs.

MA: How is DIA supporting clinical trials' future?

 One DIA board member said years ago that we’d create it if DIA didn’t exist. I couldn’t agree more. In 1964, our founders couldn't have imagined how important a trusted, neutral, global platform would be for therapeutic development today. We knew years ago that we needed to improve access to DIA's knowledge, so we digitized all of our content and created a multi-year archive that we updated with new information. Creating a digital DIA was unrelated to recent crises. This work was done so that life science professionals worldwide could access the latest knowledge of science and policy. All stakeholders developing new therapies should use the latest information.

We've created a digital platform to keep clinical development experts like you informed and educated and to share what you've learned. In the early days of SARS-CoV-2, we launched the DIA Direct series to share China's experiences with healthcare delivery, drug use and repurposing, and clinical trials. This helped us engage thousands of members and stakeholders in knowledge exchange, accelerating the response to covid. Today, we're publishing a new DIA Direct series on clinical trials and the Ukraine war. This series has helped clinical research, and care communities in Ukraine share their experiences so others can learn and support them.

DIA offers regulators and the healthcare community a global, digital, virtual, and in-person neutral platform. We'll keep building strategies to prepare for the next disruption. You may remember that we presented a snapshot of the future of healthcare at DIA 2022. Next year's DIA Global Annual Meeting 2023 (in Boston) will focus on clinical research and healthcare challenges. This meeting's theme, Illuminate, aims to highlight the latest in drug development and our global collaborative network, which can transform individual professional expertise into actionable progress that benefits everyone. The DIA 2023 Call for Abstracts is open until October 6.

Barbara Lopez Kunz, president, and global chief executive at DIA, discusses key themes on the current state of the clinical trials industry and its future.

Key Clinical Trial Industry Themes Post-COVID

Greenphire announced results from its 2022 annual market trends survey which examined top challenges and opportunities for sponsors, CROs, and sites across the world. While survey respondents represented a variety of therapeutic areas, including cardiovascular, medical device, rare diseases, infectious diseases and oncology, responses across the board revealed payments as a central concern for all stakeholders.

Results reveal shift in clinical trial priorities; payment efficiencies emerge as leading concern for sponsors, CROs, and sites.

Greenphire Announces Results From 2022 Annual Market Trends Survey

Phesi identified 117,463 patients who participated in disease modifying trials; representing a significant portion of the patient population. The Phesi analysis reveals the decline that occurs in Alzheimer’s patients over a single 12-month period, with the average 72-year-old suffering from Alzheimer’s losing 18% of cognitive function (measured by MMSE) in 12 months.

Analysis of 117,000 patients reveals rapid rate of decline in Alzheimer’s patients with 18% loss in cognitive function over a 12-month period.

New Phesi Analysis of Alzheimer’s Patients Highlights Need for Preventative Approach that Includes Younger Participants

Although randomized control trials (RCTs) have long been considered the gold standard of clinical trial design, many life sciences companies are beginning to utilize external control arms (ECAs) when designing studies to support regulatory submissions.

 This follows a trend wherein more and more companies are incorporating real-world data (RWD) and real-world evidence (RWE) into clinical trial design. This trend gained traction during the COVID-19 pandemic, when physical distancing requirements meant that trials needed to be run partially or fully remotely.

 Because life sciences companies were unable to recruit patients and conduct studies in-person, many turned to existing RWD data sources—such as claims and electronic health record (EHR) data—to bolster their research and offer complementary data.

As ECA studies become more common, companies are beginning to recognize the advantages that they offer. With a variety of RWD sources such as prior clinical trials, medical claims, and electronic health records, researchers have a wealth of data that they can leverage to create ECAs.

Advantages of external control arm studies over randomized control trials

Although RCTs with placebo or standard of care control arms are the traditional approach—and placed at the top of the evidence hierarchy—they are not always ethical or feasible. If researchers are investigating a potentially life-saving treatment they believe will be particularly effective, it can be unethical to randomize the trial participants to a placebo arm. With ECAs, this is not an issue. Because the comparison group is outside of the study, there are no patients in the study who miss out on active treatment.

ECAs can be particularly useful in the rare disease and rare oncology spaces. Because these disease types have very small patient populations, it can be impractical or prohibitively difficult to find enough patients to enroll in an RCT, which can lead to sample sizes that are too small to obtain meaningful results. When faced with a small patient population, enrolling study participants can be challenging and can result in delays to the study timeline. In these circumstances, incorporating an ECA using RWD can allow researchers to expand the sample size, ensure all study participants have access to the new medication, and accelerate the study timeline.

Researchers can also create a synthetic control arm as an alternative to an ECA constructed using RWD. A synthetic control arm is a type of ECA that enables researchers to generate a large comparator arm for a study. Using machine learning, researchers can generate synthetic patient data and create ‘digital twins’ or hypothetical patients whose characteristics are identical to the actual patients in the study. By creating digital twins, (triplets, or quadruplets,) researchers can investigate the impact of the treatment on the actual study participants while also predicting outcomes for the synthetic patients who did not receive the treatment. This technique is especially useful for rare diseases, where patient populations are so small that even obtaining enough RWD to generate RWE is challenging. Creating a synthetic control arm essentially gives researchers a larger quantity of data to study that increases statistical power even when the treated patient population is small.

Both external and synthetic control arms provide additional data for analysis, helping to expand the interpretability of the results and increase the overall impact of the study. With FDA increasingly open to the use of RWE as a factor in decision-making, this ability to provide meaningful comparative analysis can strengthen regulatory submissions for life sciences companies and support market access with payers.

Use cases for external control arm studies

Currently, ECAs are most often used in oncology. For example, Roche utilized an ECA when investigating the drug alectinib as a treatment for non-small cell lung cancer (NSCLC). Alectinib first received accelerated FDA approval to treat this form of lung cancer, and it was later conditionally approved in the EU. However, the EU required additional evidence of alectinib’s effectiveness relative to the drug that was the standard of care at the time for metastatic NSCLC. Instead of waiting for Phase III clinical trial results (which would have slowed down the approval timeline), Roche utilized an external control arm to provide the necessary evidence.

 The use of an ECA advanced coverage of Roche’s drug by 18 months in 20 EU countries.

Another study showcasing the value of ECAs can be seen in Pfizer and Merck’s investigation of the drug avelumab for metastatic Merkel cell carcinoma (mMCC), a rare type of skin cancer.

 In this case, an RCT was not practical for several reasons. The rare nature of the disease meant that researchers had limited patient data, and the short patient survival times typical of the cancer precluded recruiting enough patients, not to mention the ethical concerns of denying treatment for those in the control group. When seeking approval of avelumab, Pfizer and Merk incorporated an ECA constructed with EMR data into their submission. This approach was successful, and avelumab was approved for treating mMCC in 2018.

These examples illustrate how ECA studies can accelerate the time to approval and avoid potential challenges inherent in RCT study design. ECAs are an important tool to speed access to critical treatments in diseases with high unmet need. Although ECAs are currently primarily used in oncology, their potential impacts stretch into many other disease spaces, and we will undoubtedly see their use increase in the coming years.

How to leverage data analytics to create external and synthetic control arms

To create an ECA, researchers can employ a technique called propensity score matching (PSM). This process involves taking existing RWD, such as EHR or claims data, and applying the inclusion and exclusion criteria used for the trial. With this technique, researchers can identify individuals outside of the study who match study participants. They can then compare the study participants who receive the treatment to these individuals outside of the study who do not, balanced according to characteristics that might influence either treatment or outcome.

The purpose of PSM is to try to reduce confounding variables that could be attributed to basic differences between the control group and the study group, as opposed to the impacts of the treatment. To ensure that individuals are closely matched, the key is to apply select inclusion and exclusion criteria. The goal is to line up the control arm as closely as possible with the treatment arm in demographics, medical history, co-morbidities, and other characteristics.

PSM can be done using data analytics software that is capable of mapping RWD sources, applying inclusion and exclusion criteria, and identifying external patients whose characteristics match the treatment group. The ideal analytics platform contains RWD from a variety of sources including claims, EHR, registry, and more, and is purpose-built for healthcare data. Such software allows for the closest matches between the treatment and control group and thus offers the most meaningful results.

Like PSM, generating synthetic data and digital twins (triplets, or quadruplets) can be accomplished using data analytics software with machine learning capabilities. Before running any predictive analyses, the model must be trained with significant amounts of data. Once adequately trained, the machine learning model can be used to generate synthetic data and predict outcomes for the digital patient copies. Effective analytics software will simplify this complicated process and enable researchers to conduct meaningful studies using both real external and synthetic control arms.

How to ensure strong regulatory submissions

When assembling a regulatory submission in industry, the data must be relevant and reliable, and the analytics must be transparent and reproducible. Fortunately for life sciences companies, the top analytics platforms enable regulatory-grade submissions using RWD and RWE. Because ECAs are still a relatively new phenomenon, regulatory bodies such as the FDA can be skeptical of the data and the platforms used to conduct the analysis. Using an analytics platform that is well-known in the industry and that automatically captures all the methods used can mitigate issues around quality and ensure that all aspects of the submission are fully transparent and thoroughly documented.

To set themselves up for success with regulatory submissions involving ECAs, researchers must have a deep understanding of the data and be able to clearly articulate their research questions. The data used to construct the ECA must be accurate, complete, and relevant to the research question. Employing effective data analytics software as described earlier will also equip researchers with regulatory-grade data and insights that lead to strong and acceptable submissions.

FDA has published four draft guidances on how it will consider real-world data (RWD) in regulatory decision-making, including the use of electronic health records and registry data. More guidance documents are planned to detail study designs that incorporate external control arms.

What does the future hold for ECAs? To date, ECAs have been used in studies of rare disease and in oncology, where single-arm trials are the most practical or ethical option. No marketing applications in oncology have used an external control arm as part of the primary efficacy analysis. Instead, ECA data have been used to establish the natural history of the disease or in a comparative efficacy and safety analysis.

 The movement in 2022 is towards a true contemporaneous comparator arm, wherein RWD sourced from EHRs, administrative claims and registries are expected to be as broad and as deep as clinical trial data.

RWD data—because they were created for a different purpose than biopharmaceutical research—are by nature messier than clinical trial data. Some outcomes are captured less frequently or not at all; concepts of interest may be recorded in a non-standardized way or exist only in physician notes rather than in a coded field. One way to address data ‘missingness’ is to link various data sources to capture more fully the longitudinal patient journey through the healthcare system and to check the accuracy of self-reported information. There are several companies that assist with this data linkage by assigning unique identifiers called tokens to each patient in each data source. The tokens assigned to one source can be matched to a token in another source so as to be able to identify a single patient who exists in, for example, both a claims data source and a registry.

This ‘messiness’ of an RWD-informed ECA can introduce some error in the estimation of treatment effect, but if the effect size is large enough, the difference (favorable or unfavorable) will manifest. This makes an ECA a useful option for new and promising treatments but not for those likely to demonstrate only a modest benefit. Moving forward, ECAs are likely to play a larger role in assessing benefits and risks for treatments that receive accelerated approval or in diverse patients who were not included in pivotal clinical trials.

As more life sciences companies begin to utilize comparator arms, both researchers and patients will see numerous benefits. First, ECAs are more ethical and acceptable to patients because there is no risk of being assigned to placebo or standard of care. This is most important when the treatment cannot be blinded, and patients know that they are not receiving a promising new therapy. Second, ECAs can help to shorten study timelines and reduce costs. A shorter study timeline means increased speed to actionable insight for life sciences companies. When investigating life-saving treatments, this is especially critical. When researchers can complete studies, uncover insights, and make regulatory submissions more quickly, these treatments get into the hands of the patients who need them sooner. Ultimately, ECA-supported studies can enable faster approval of therapies and lead to more lives save or improved.

, PhD, Executive Director of Solutions, Panalgo

https://www.statnews.com/2019/02/05/synthetic-control-arms-clinical-trials/

http://www.sas.com/offices/pdf/emea/phuse-2020-rw02-addressing-industry-disruptions-in-clinical-trials.pdf

https://www.fda.gov/science-research/science-and-research-special-topics/real-world-evidence

https://www.futuremedicine.com/doi/full/10.2217/cer-2018-0032

https://www.statnews.com/2019/03/06/whats-scary-and-appealing-about-real-world-evidence/

https://pubmed.ncbi.nlm.nih.gov/29799096/

https://www.fda.gov/regulatory-information/search-fda-guidance-documents/considerations-use-real-world-data-and-real-world-evidence-support-regulatory-decision-making-drug

https://www.raps.org/news-and-articles/news-articles/2022/6/rwd-in-clinical-trials-external-control-arms-take

https://www.annalsofoncology.org/article/S0923-7534(22)00006-0/fulltext

https://datavant.com/wp-content/uploads/dlm_uploads/2018/09/WhitePaper_-De-Identifying-and-Linking-Structured-Data.pdf

https://www.fda.gov/regulatory-information/search-fda-guidance-documents/real-world-data-assessing-registries-support-regulatory-decision-making-drug-and-biological-products

Data from past clinical trials has provided researchers with a good starting point to create more ECAs.

Featured Content

Applied Clinical Trials September 2022

‘Record’ Gains in Bringing Research Closer to the Patient

Salary Survey: The Age of COVID