Shifting the Large Simple Trials Paradigm

Conducting Large Simple Trials (LST) to answer questions about “real-world” medical interventions is not a recent concept. The often-cited ISIS 2 trial (~16,000 patients, 416 sites, 16 countries)¹ completed in 1988 was so successful that it quickly prompted standard-of-care changes that are now saving thousands of lives annually. Clearly there is power in study simplification and large numbers, but large trials require big budgets and consequently, sponsors have been reluctant to undertake these prospective trials, preferring to seek answers from existing sources such as medical databases.

A report from a recent Institute of Medicine LST workshop comments that a typical randomized clinical trial (RCT) has an average total of 13 endpoints, including one primary, five secondary, and a number of tertiary or exploratory endpoints with an average of nearly 170 procedures, only half of which support primary and secondary endpoints. An RCT also typically averages 35 eligibility criteria and a case report form of nearly 170 pages, which calls for 11 visits from a study participant over a mean period of 175 days.²

Such complexity drives up costs and can impose considerable burden on investigators and patients. This, in turn, leads to poor recruitment and retention rates with an accompanying reduction in the validity of study results. Moreover, important but rarer safety issues are not being identified in smaller RCTs (e.g. fen phen withdrawal), only being detectable with exposure in larger populations. In contrast, as well as reducing the cost per patient, a well-designed and appropriately purposed LST can significantly enhance the generation of outcome data more representative of real-world healthcare settings.  

The soaring costs and limitations of RCTs, along with availability of eClinical technology to mitigate operational challenges of LSTs, are changing attitudes among sponsors and regulatory agencies. Presenting at the 2013 Clinical Trials Transformation Initiative LST experts meeting,³ Dr. Kweder, FDA Deputy Director, Office of New Drugs (CDER)⁴ indicated these key benefits:

• Effectiveness - Can reliably detect small-to-moderate effects of particular treatments or exclude with statistical certainty the possibility of such effects

• Relevance - Results are directly relevant to the wide range of patients seen in clinical practice and may be more rapidly incorporated into standard of care

• Best Uses - Well-suited for post-marketing safety studies, minimize the potential for bias yet are still relevant to real-world clinical practice

Acknowledging that large trials are frequently seen as prohibitively expensive, Kweder further remarked that a recent analysis⁵ suggests that streamlined approaches to mega-trials could potentially reduce costs by more than 90 percent.  She also emphasized that principles of LSTs such as risk-based monitoring, quality-by-design and reduced data collection were consistent with FDA thinking, particularly in the context of safety assessment. Similar views have been expressed by the EMA.⁶

In general, LSTs are characterized by large sample sizes; broad entry criteria consistent with the approved medication label; minimal, streamlined data collection requirements; objectively-measured endpoints (e.g. death, hospitalization); and follow-up that minimizes interventions or interference with normal clinical practice.⁷ The design can be used to further evaluate known safety issues, examine rare but significant safety signals, as well as the impact of treatment on disease outcomes.  

No specific sample size defines a LST, although broad consensus suggests ~1,000 patients represents the lower end expanding up to tens of thousands of patients and hundreds of sites. Sample size is determined by the nature of the research question, the rarity of the event of interest, and what constitutes a meaningful clinical difference between comparative interventions. However, it can be limited by the rarity of the disease or a narrow indication. As patients are selected to be representative of the ”normal” way the drug is used, LSTs tend to seek patients outside of traditional clinical trial settings, being conducted across more general healthcare settings, and often involving investigators with little or no previous clinical trial experience.

Protocol Optimization is Critical to LST Success

Overall goals are to maintain routine care as far as possible, streamline study procedures, and reduce complexity. There are many components to protocol optimization but a major one is data collection, focusing on absolute ‘must have’ variables and excluding variables for exploratory analyses.

Regulators are increasingly supportive of such targeted data collection. Consider these excerpts from FDA guidance⁸: “Arduous and excessive data collection may be a major disincentive to investigator participation in clinical trials. There is also growing interest in and a need for larger, simpler trials to obtain outcome data, data on long-term effects of drugs, and comparative effectiveness and safety data, but excessive data collection requirements may deter the conduct of these types of trials...[M]ore selective safety data collection may (1) improve the quality and utility of the safety database and safety assessment without compromising the integrity and validity of the trial results or losing important information, (2) ease the burden on investigators conducting and patients participating in a study, and (3) lower costs, thereby facilitating increased use of large, simple trials…”

When conducting LSTs, sponsors should consider the following principles essential:

• Simplified Protocol - Distinct effort to minimize additional tests beyond usual care (e.g. blood draws, certain imaging, etc.). Several documents offer guidance for developing streamlined protocols, and both the FDA and EMA have expressed a willingness to work with sponsors to help ensure protocols reflect their intended purpose, namely efficacy, safety or both⁹  

• Normal Care-Settings - Both intent and size dictate using normal care centers. It is the most effective way to enroll patients and capture normal care practices and outcomes 

• eClinical Technology - With a large number of sites, often wide geographic dispersion, and relative investigator research inexperience, eClinical tools such as EDC, CTMS and multiple communication avenues are important enablers 

• Risk-based Monitoring - Usual level of monitoring would be prohibitively expensive, and centralized monitoring approaches are required as well as use of eClinical tools¹⁰

The expanding body of LST-relevant guidance¹¹ to help sponsors implement these principles is evidence of broadening regulatory acceptance. One excellent source is the EMA Guideline on Good Pharmacovigilance Practices, Module VIII.¹² This mainly covers Non-interventional Studies but also provides useful insights into the conduct of clinical trials. A good review of issues relating to LSTs is presented in the Institute of Medicine (IOM) 2013 IOM Workshop Report.¹³ Access to electronic health records (EHR) can also provide significant benefits for feasibility, recruitment and even sourcing data for the trial.

LST Operational Challenges

Study size and investigator inexperience are responsible for most LST operational challenges. Patient recruitment efforts still rely greatly on the investigator reviewing their lists for suitable candidates, but today’s world of multimedia communication has extended patient reach and provided those seeking a research treatment option the means to find and evaluate potential trials.

Site start-up offers its own challenges. Staff may be unfamiliar with clinical trial and regulatory requirements. While these may be criteria for rejecting a site for some RCTs, for a LST this may just be a trigger for additional training, support and oversight. Site monitoring is a major cost driver and without the budget to monitor data to the same degree as a traditional RCT, a targeted monitoring approach is the only practical solution. Experience to date confirms that technology tools can help identify issues with particular sites or with parts of the data. This enables monitoring activities to be focused on areas where there is most risk to study integrity, patient confidentiality and data quality.

It is worth noting that a LST may require long-term follow-up, sometimes several years. Following a burst of activity during the early stages, it can be difficult to maintain motivation for both patients and investigators. Patients who initially visit frequently may drop to visits every three or six months or greater. Patients may relocate, change physicians, or get better and not understand the need for follow-up, get worse and need to change treatment, or suffer a new disease or exacerbation of an existing disease that could be either more debilitating or impactful on their quality of life. All such events make patients more difficult to reach and maintain their engagement. Clinical staff turnover is inevitable and requires training updates and communications that similarly maintain their engagement. 

Clear regular communications between staff and patients is imperative, as is communication between sponsors and sites. Investigators value seeing collated summaries of their own data and sponsors need to think about how they can provide this information without impacting on overall study integrity. Ongoing feedback, such as interim study results, can also help maintain motivation by informing care practices and providing publishing opportunities. Patients also want to know their time and effort has been beneficial in gaining a broader understanding of the disease and improving treatment for future patients.

Technology Use to Enable Study Design Options and Smooth Operations

Given the challenges of study size and investigator inexperience, effective use of eClinical technology is a critical enabler in driving down cost and ensuring high quality results; otherwise there are too many moving parts to effectively manage LSTs. To derive the most benefit from technology when designing and conducting LST, the following best practices should be followed: 

• Usability - A ‘Human Factor Engineering’ designed intuitive interface (think of online banking), simplified processes, and ‘minimalized’ EDC are essential for reducing learning curve and the ongoing administration burden. Both patients and staff must be able to easily work in the system. Working with the system should be as intuitive as working with an ATM.

• Automation - Wherever possible automating procedures and workflow, removing sources of error and speeding the process.

• Simplification - It is easy to over-engineer technology use rather than rely on the power of LST simplification principles; for example, using reduced data collection and not cleaning data more than necessary.

• Site Assistance - As many will not be familiar with various activities required and will get stuck occasionally, making it easy for sites to get fast effective help by providing contact person and contact details is important.

• Reporting - Recognizing limitations imposed by using some data that may not have been cleaned, it is nonetheless important to be able to rapidly generate summary data; for example, to know exactly when approaching a specific study endpoint so as to avoid unnecessary cost and effort.

Translating these principles into practice is a major area of expertise for some large CROs. For example, traditionally site start-up has been one of the more onerous activities, which can put off potential physicians from participating. The PAREXEL Site Start-up system, for example, quickly and with minimal fuss and effort leads clinicians through the process of getting up and running for the study. For a large simple trial, such efficiency is extremely valuable. Another powerful feature of the Site Start-up system is that it directly inserts electronically signed site regulatory documents into the correct folder of the electronic trial master file (eTMF), and automatically updates the document status in the CTMS, saving considerable administrative effort, avoiding human error and making the documents and statuses available earlier.

Conclusions

LSTs constitute a powerful evidence generation tool for sponsors delivering important insights on the relative value of a particular medication in normal clinical practice settings. The merits of large sample size and focused objective endpoints can be harnessed to identify and validate safety signals and treatment outcomes. While careful study design, intelligence-led operational planning, and assistance of new eClinical technology streamline the operational approach significantly reducing costs-per-patient.

BOX1 – Technology

eClinical Technology is Critical to LST Success

Given LST’s size and investigator inexperience, eClinical tools such as EDC, CTMS, and multiple communication avenues are important enablers. The following best practices principles are suggested:

• Usability - A ‘Human Factor Engineering’ designed intuitive interface (think of online banking), simplified processes, and ‘minimalized’ EDC are essential.

• Automation - Wherever possible automating procedures and workflow

• Simplification – Do not over-engineer technology, rather than rely on the power of LST simplification principles;

• Site Assistance -Make it easy for sites to get fast effective help by providing a contact person and contact details.
• Reporting - Recognize limitations imposed by using some data that may not have been cleaned, it is important to be able to rapidly generate summary data;

BOX2 – LST Principles

Key Principles for LSTs

LST are generally used to evaluate known safety issues, examine rare but significant safety signals, and examine the impact of treatment on disease outcomes. They can also reduce cost. Broadly, LSTs have the following attributes:

• Large sample sizes (~1000 or more)

• Broad entry criteria consistent with the approved medication label

• Minimal, streamlined data collection requirements

• Objectively measured endpoints, e.g. death, hospitalization

• Follow-up that minimizes interventions or interference with normal clinical practice

Peggy Schrammel, Nick Darwall-Smith and Gary Coward, PAREXEL International

References

1. Randomised Trial Of Intravenous Streptokinase, Oral Aspirin, Both, Or Neither Among 17 187 Cases Of Suspected Acute Myocardial Infarction: Isis-2, The Lancet, Volume 332, Issue 8607, Pages 349 - 360, 13 August 1988, http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(88)92833-4/abstract

2. IOM Workshop Report of 2013, Large Simple Trials and Knowledge Generation in a Learning Health System, page 39, comments from Ken Getz, Director of Sponsored Research & Research Associate Professor, Tufts Center for the Study of Drug Development, http://www.nap.edu/openbook.php?record_id=18400&page=1

3. Clinical Trials Transformation Initiative LST experts meeting, May 2013 …http://www.ctti-clinicaltrials.org/what-we-do/investigational-plan/large-simple-trials/meetings

4. FDA Deputy Director, Office of New Drugs, CDER

5. Sensible approaches for reducing clinical trial costs, Eisenstein et al. Clin Trials 2008; http://www.academia.edu/986781/Sensible_approaches_for_reducing_clinical_trial_costs

6. EMA Reflection paper on risk based quality management in clinical trials, http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2011/08/WC500110059.pdf

7. Is the large simple trial design used for comparative, post-approval safety research? A review of a clinical trials registry and the published literature, Drug Saf. 2011 Oct 1 (doi: 10.2165/11593820-000000000-00000), http://www.ncbi.nlm.nih.gov/pubmed/21879776

8. Ibid.

9. EMA, Guidance for the format and content of the protocol of noninterventional post authorization safety studies; EMA (ENCePP) Guide on Methodological Standards in Pharmacoepidemiology (Rev2); ENCePP Checklist for Study Protocols (revised and accepted 2013

10. FDA’s Oversight of Clinical Investigations – A Risk-Based Approach to Monitoring, August 2013.

11. ENCePP Guide on Methodological Standards in Pharmacoepidemiology, http://www.encepp.eu/standards_and_guidances/methodologicalGuide.shtml; ENCePP Checklist for Study Protocols, http://www.encepp.eu/standards_and_guidances/; EMA Guideline on Conduct of Pharmacovigilance for Medicines Used by the Pediatric Population for studies conducted in children, http://www.ema.europa.eu/ema/index.jsp%3Fcurl%3Dpages/regulation/document_listing/document_listing_000087.jsp;  ISPE Guidelines for Good Pharmacoepidemiology Practices (ISPE GPP), http://www.pharmacoepi.org/resources/guidelines_08027.cfm: FDA-Best Practices for Conducting and Reporting Pharmacoepidemiologic Safety Studies Using Electronic Healthcare Data http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM243537.pdf

12. EMA Guideline on good pharmacovigilance practices (GVP), Module VIII – Post-authorisation safety studies (Rev 1) 19 April 2013, EMA/813938/2011 Rev 1* http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2012/06/WC500129137.pdf

13. Institute of Medicine Workshop 2013, Large Simple Trials and Knowledge Generation in a Learning Health System, http://www.nap.edu/openbook.php?record_id=18400

14. See ref# 12