Despite ambitions to streamline regulatory review, FDA’s Elsa platform has been prone to hallucinations, prompting internal scrutiny and questions about AI reliability and governance.
In a July 9 interview with Applied Clinical Trials' sister publication, Pharmaceutical Executive, Marcel Botha, CEO of 10XBeta, mentioned the initial challenges FDA’s artificial intelligence (AI) model, dubbed Elsa, experienced when it came to accuracy.
“One of the challenges that came out from the initial release of the Elsa model for FDA is that it was prone to hallucination,” Botha said. “By that, I mean it was making stuff up. … We can't have our AI do that when it comes to critical analysis of core ingredients and component structures that are required. These are elements where a slight deviation makes something safe or not.”
According to a Wednesday report from CNN,1 the problems with Elsa have continued. Quoting six anonymous FDA officials, some still with the agency and others recently departed, the report explained that while the large language model can generate meeting summaries, it has experienced hallucinations by citing studies that don’t exist.
“Anything that you don’t have time to double-check is unreliable,” one of the anonymous FDA officials told CNN. “It hallucinates confidently.”
Elsa is a large language model assistant meant to help FDA employees “read, write, and summarize” faster. The tool can summarize adverse events to support safety assessments, perform rapid label comparisons, and even generate code for internal databases, while running inside a secure GovCloud environment that does not train on industry submissions, protecting that crucial research data.
FDA Commissioner Marty Makary, MD, MPH, unveiled Elsa in a June video,2 saying the agency is using Elsa to “expedite clinical protocol reviews and reduce the overall time to complete scientific reviews.”
Makary said in the video that one reviewer said that a task that took a couple of days now takes 6 minutes. FDA Chief AI Officer Jeremy Walsh said: “Today marks the dawn of the AI era at the FDA with the release of Elsa, AI is no longer a distant promise but a dynamic force enhancing and optimizing the performance and potential of every employee.”
In an interview with CNN as part of their report, Makary acknowledged that Elsa could hallucinate like any large-language model, and that employees are using it for organization and efficiency, and use of it is optional. He added that Elsa will improve, and FDA officials will also get better at using it.
One of President Donald J. Trump’s focus points in 2025 is to reduce the federal workforce, including at the FDA, where about 3,500 positions were cut. Many other high-profile officials have left for jobs in industry or elsewhere.
This has led to some slowdowns. In April, the Wall Street Journal reported that the FDA was missing deadlines and ignoring industry messages, resulting in trial delays. In June, the agency missed the PDUFA data for KalVista Pharmaceuticals’ sebetralstat, citing “heavy workload and limited resources,” according to a KalVista news release.3 The drug was since approved on July 15.
On April 1, former FDA Commissioner Scott Gottlieb, MD, weighed in, concerned that the FDA would bring back the days of “drug lag.”
“Through a generation of congressional actions, investments in expertise and hiring, and careful policymaking, we built the FDA into the most efficient, forward-leaning drug regulatory agency in the world—and established the U.S. as the global center of biopharmaceutical innovation,” Gottlieb posted on X,4 formerly Twitter, on April 1. “Today, the cumulative barrage on that drug-discovery enterprise, threatens to swiftly bring back those frustrating delays for American consumers, particularly affecting rare diseases and areas of significant unmet medical need.”
Avoiding these delays is where AI comes into the administration’s plans. HHS Secretary Robert F. Kennedy, Jr. and Makary have beat the drum on using AI such to streamline and speed up the review and approval process. But according to the CNN report, it’s not there yet. Two anonymous FDA staffers said Elsa cannot yet assist with review work.
External observers have also begun to raise concerns. In a client alert, the law firm Hogan Lovells questioned what kind of oversight governs the tool’s outputs, whether benchmarks exist for evaluating its performance, and how “human-in-the-loop” processes are being enforced. “It’s not clear how the agency defines success for Elsa, and whether guardrails are in place to prevent AI-generated errors from influencing regulatory decisions,” the firm wrote.
Makary’s ambitions for AI go far beyond administrative tools. In a June opinion piece in JAMA, Makary and co-author Vinay Prasad, MD, the FDA’s director for the Center for Biologics Evaluation and Research, outlined a vision for “rapid or instant reviews” of drug applications, faster food safety alerts, and the rebuilding of public trust through transparency and innovation. He compared the goals of Elsa and other agency AI tools to the pace of Operation Warp Speed, which helped deliver COVID-19 vaccines in record time.
In his interview with Pharm Exec, Botha said AI can, theoretically, help the FDA with review work.
“It can look at complex submissions and to do a lot of the grunt work in analyzing ingredients, processes followed, checking for completeness, and other things that a human would historically have to do,” Botha said. “It can also very quickly check for any anomalies that need to be flagged by the human because it doesn't follow protocol or understood protocol based on past experience or data sets.”
But like any large-language model, time and training data is key.
“In order to have a successful AI strategy within the FDA, the organization must start with the training data set for the model being used to build the LLM specific to drug discovery,” Botha explained. “What's the framework that it gives this large language model to reference best past practice, and what are the guardrails being put in place to ensure its not just repeating the status quo from the last 70 years?”
He added: “The benefit of what AI is going to bring to the FDA is still very much on the edge of machine learning.”
Stay current in clinical research with Applied Clinical Trials, providing expert insights, regulatory updates, and practical strategies for successful clinical trial design and execution.
FDA Issues Complete Response Letter to Replimune for RP1 Combination Therapy in Advanced Melanoma
July 22nd 2025The FDA’s complete response letter cited concerns that the Phase I/II IGNYTE (NCT03767348) trial in advanced melanoma was not an adequate, well-controlled study and that its heterogeneous patient population limited interpretability, preventing approval in its current form.
Unifying Industry to Better Understand GCP Guidance
May 7th 2025In this episode of the Applied Clinical Trials Podcast, David Nickerson, head of clinical quality management at EMD Serono; and Arlene Lee, director of product management, data quality & risk management solutions at Medidata, discuss the newest ICH E6(R3) GCP guidelines as well as how TransCelerate and ACRO have partnered to help stakeholders better acclimate to these guidelines.
Improving Relationships and Diversifying the Site Selection Process
April 17th 2025In this episode of the Applied Clinical Trials Podcast, Liz Beatty, co-founder and chief strategy officer, Inato, discusses a number of topics around site engagement including community-based sites, the role of technology in improving site/sponsor relationships, how increased operational costs are impacting the industry, and more.
New FDA Initiative Reveals Common Reasons for Drug Application Rejection
July 11th 2025By publishing more than 200 complete response letters, the FDA is offering new visibility into the issues that most often delay drug approvals, including safety concerns, trial design flaws, and manufacturing gaps.
