Timothy Pratt, PhD

Yes, you read that headline right. Intrigued? Good, but I have misled you just a little, as this paper 

 about Clinical Trial Diversity, but is not so much “politically incorrect” as “politically agnostic”. What I hope to accomplish in this paper is to briefly define the problem, answer the question “why should I care?” (Hint—it has nothing to do with political correctness), and some actions you may wish to consider for your next trial or submission.

Recently, I was asked to sit on a panel at the Octane MedTech Innovation Forum in sunny Irvine California October 2021. I was certainly happy to escape the 40-degree Minnesota Autumn, but perhaps not so overjoyed at the topic—Diversity in Clinical Trials. In my current role, I increasingly see a push for a diverse population in In Vitro Diagnostic (IVD) trials, but little, if any action in Medical Device trials. With respect to drugs and biologics, my company at least has major initiatives, training, and task forces on the topic. Perhaps the difference between IVDs and devices may be a function of the size of the IVD trials (the last one we were involved in was 10,000 patients), and Device trials tend to be a small fraction of that—in the low hundreds, and sometimes less than 100—but then again, perhaps the IVD people have cottoned on to something device has yet to fully realize, let alone embrace. What’s that? I contend, as I did on the panel, that it is not about being politically correct or “woke”, but rather it should be all about the science. If we are admonished to “follow the science” in other aspects of life, why not in our clinical trials, as after all, is that not what we’re supposed to be doing? Besides that, there are compelling longstanding ethical reasons to pursue 

This is not a new concept. In 2002, nearly 20 years ago, The New England Journal of Medicine published an editorial by T.E. King MD whose opening lines read

”The rational use of a new drug or treatment should be based on the results of controlled clinical trials that are well designed, avoid bias, and include subjects representing the full range of patients who are likely to receive the treatment once it is marketed. In addition to age, sex, diet, underlying disease, and the concomitant use of other medications, race and genetic factors may play pivotal parts in the variability of subjects' responses to a medication.”

As King observed two decades ago, despite the efforts of various government agencies “it is uncertain…whether the participation of minority groups in clinical trials has increased”. As we will see below, the situation remains unacceptable, but there are concrete measures that can be taken to effect meaningful change.

Fundamentally, our clinical trials to date have not been representative of the populations in which the therapy is used. This state of affairs is fundamentally bad science—you cannot expect to see the same results in a diverse heterogenous population that were seen in a relatively homogenous population. It is very simple: if you wish to extrapolate the results of a trial to the entire population who will use the drug or device (or IVD for that matter), then the sample (clinical trial participants) must be representative of that population. To fail to do this is a fundamental sampling error that may lead manufacturers to conclude their product is safe and efficacious when in fact it may not be for the intended population. Much has been made in the past about how good results seen in approval trials are not mirrored when the therapy is released “into the wild” or the general population—and whilst some of that may be related to difference in performance at research sites, off-label use, patient non-compliance, etc., perhaps some of the disparity might just result from initial sampling error.

 Why? Simply because the inclusion and exclusion criteria (I/E) are constructed in such a fashion as to control variables and ensure a degree of homogeneity in the trial population. From a purely scientific perspective, there’s nothing wrong with this—it’s actually good scientific methodology—but it does mean the population studied is not the same as the broader population who may meet one or more of the exclusion criteria: from the perspective of generalizability to the entire population, many approvals trial I/E criteria actually create a sampling error situation.

If we consider published data from Moderna on the COVE COVID-19 study in the US (and this is one of the more diverse studies where a conscious effort was made to represent population racial makeup appropriately)), we can see that 63% of patients were Caucasian, 20% Hispanic, 10% Black, 4% Asian and 3% other.

 US Census Bureau 2019 estimates for the same population breakout nationally are 60.3% Caucasian, 18.5% Hispanic, 13.4% Black and 5.9% Asian—other, including Alaskan Natives, Pacific Islanders etc. account for 4.3%

 We can see in this simple example that Caucasians were over-represented in the COVE study, whilst Blacks and Asians were quite under-represented (by 34% and 47.5% respectively). US FDA’s numbers for new drugs or biologics seeking approval in 2019 puts the Caucasian participation rate at 73%, with most other groups under-represented.

 For specific therapy types, the situation is even more unbalanced – Nazha 

 report on nine advanced-solid-tumor phase III trials comprising 1711 patients—Caucasian participation ranged from 77.2% to 97.5% whilst Black participation ranged from 0-4%.

 These disturbing figures are in the face of a known 28% higher cancer-related mortality rate in Blacks compared to Caucasians.

 In another type of cancer, Caucasian females have a 131.8/100,000 incidence of breast cancer and Blacks 124.7, yet Blacks are 39.7% more likely to die from the disease, despite the lower incidence rate.

 It would seem that appropriate representation in clinical trials should be a priority to ensure therapies developed are effective in the populations that need them the most.

Other examples abound in a variety of therapeutic areas. Just recently a meta-analysis published in May 2021 of over 33,400 open-angle glaucoma clinical trial participant data showed 70.7% of the study population was Caucasian, 16.8% was Black, 3.4% was Hispanic/Latino, and 9.1% consisted of other races/ethnicities, including Asian, Native Hawaiian or Pacific Islander, American Indian or Alaska Native, and unreported.

 For the sake of brevity, however, given the wide play this issue has been receiving in various news media in recent times, perhaps we can just take it as read that there are significant problems in the makeup of many clinical trials in terms of representation of the population in whom the therapy will be employed.

Think about it for a moment—if you release a therapy into a population in whom it has not been adequately tested, doesn’t that make it unproven? Is that ethical? The absolute bedrock of everything we do as researchers must be founded in ethical behavior. Must. Surely there is no debate on this topic? Scientifically, we know there are problems with sampling error—this is especially true in genomics: as part of the research for the panel discussion I was involved in recently I discovered that overall, according to 

 of 5729 samples sequenced to develop therapies for conditions such as prostate and squamous cell lung cancer, 77% (n = 4389) were white, 12% (n = 660) were black, 3% (n = 173) were Asian, 3% (n = 149) were Hispanic, and less than 0.5% combined were from patients of Native Hawaiian, Pacific Islander, Alaskan Native, or American Indian decent.

 How can a manufacturer possibly assure the unsuspecting public that their proposed treatment will work adequately in Asians, Hispanics, or Pacific Islanders for example? The simple answer is they can’t – and that borders on the unethical. How can this be? As I reported in 2014—seven years ago! —12.9% of households speak Spanish at home in the US.

 You could argue that many Hispanics are actually racially Caucasian, and that has some degree of merit, but there are compelling other reasons to actively represent them in clinical trials, as will be discussed later.

The other reason you should care, if questionable ethics are insufficient to sway you (they can, and are, always debated to a degree) is that the US FDA cares, as do other Regulatory Agencies around the world. US FDA issued final guidance on the topic entitled 

Enhancing the Diversity of Clinical Trial Populations — Eligibility Criteria, Enrollment Practices, and Trial Designs Guidance for Industry November 2020

 so you know they’re serious about it, especially as guidances rapidly turn into expectations with FDA. Unfortunately, FDA’s guidance refers explicitly to drugs and biologics, and makes no mention of my own particular specialty, Medical Devices. Despite this shortcoming, the guidance has broad application in the Device field.

On that topic, devices, I have often heard manufacturers observe “I get clinical trial diversity when it comes to drugs, but how does that apply to devices or combination drug/device products? Anatomy and Physiology (A&P) is changed that much by race or ethnicity, is it?” To this position I respond as follows – 1) in order to make that claim (lack of meaningful difference in A&P) you need to have studied it, and per the basic sampling error problem described above that hasn’t happened, 2) Racial and cultural differences can make a huge difference to how a patient interacts with Software as a Medical Device or even cares for an implanted device, 3) cultural activities and foods can vary how a device functions (think for example about long periods of fasting and an insulin pump (Ramadan), and 4) because of differential access to healthcare related to income, mistrust, lack of physical clinics etc. some minorities, especially Blacks and Hispanics present, if they present at all, with much more advanced disease than may be allowed in a clinical trial. On the panel which prompted this paper, one of my co-panelists observed that Blacks presented with far more advanced aortic stenosis than Caucasians, and this made use of their cardiac valves much more difficult – the problem is that in order to get the product approved in the first place, inclusion/exclusion criteria in pivotal studies often excludes really advanced disease, so you really don’t know how it will work in that population. That particular problem is worthy of its own brief discussion.

FDA’s guidance on enhancing the diversity of clinical trial populations has a lot to say about broadening inclusion criteria and easing up on exclusionary criteria to increase minority participation. There remains, however, a major problem in terms of establishing performance goals for a therapy – relevant priors. We’ve all had the experience of going to US FDA and proposing a study design with a particular performance goal in mind, and having FDA come back with “paper X shows this outcome – that should be your goal”. Paper X however, and most likely the research papers you used to develop your own performance goals in the first place, is almost certainly derived from a non-diverse population – that is, after all, the fundamental problem under discussion. Bayesian statistics rely on appropriate relevant priors to be worthwhile, so, if your (or FDA’s) relevant priors are fundamentally flawed by sampling error, how can you then rely on them to develop meaningful performance goals for our product? If you enroll minorities who present with much more advanced disease and/or considerably more co-morbidities due to lack of access to healthcare would you not expect to see poorer outcomes than in a much healthier population? Of course!

Manufacturers want to get their therapy approved—many will benefit from it. FDA wants to ensure it is safe and effective. Relying on flawed non-representative historical data to set performance goals is not going to achieve either outcome in a diverse population. Rather, it exacerbates the situation as manufacturers push to get their therapy into the population where it is most likely to be “successful”—if the relevant prior was in a predominantly Caucasian population, then a new study demonstrating comparative performance should be done in the same population: that is pretty basic science. FDA pushes for a very high-performance bar without considering performance may be lower in a diverse population. And so, it goes on—what to do?

Specific action items that can make a difference

Three simple principles can dramatically change the quality of your next study. There are more things you can do to be sure but trying to “boil the ocean” all at once is usually frustrating and unproductive. Consider these elements and you’ll be well on your way - Design, Sites, and Engagement

Look for truly relevant priors, if they exist

Have the conversation with FDA/other Regulatory Agencies and the validity of relevant priors and negotiate appropriate performance goals based on Real World Evidence in combination with prior research. There’s a whole series of guidances on RWE now available.13

Prospectively plan to do subgroup analysis on Races/ethnicities and have specific enrollment targets pre-specified for those sub-groups (this will help bolster your argument on appropriate performance goal setting)

Also ensure your biostatistician accounts for the likely higher rate of co-morbidities in minority populations – this may require literature searching outside of the specific therapy area/disease state you’re actually studying

Review the FDA Guidance on enhancing Diversity in Clinical Trials to think through inclusion/exclusion criteria to open up the trial to more patients

Reduce follow-up frequency and burden as much as possible—many minority populations have large numbers of low-income members who may not be able to take time off work—or be able to afford to—ensure patient compensation for their time is adequate and/or limited access to private transportation is considered.

Various companies now offer in-home visits specifically in support of clinical trials: consider if some follow-up visits may be suitable for this approach

Plan to go into the communities where the populations you seek to enroll live. Lots of sponsors focus on Academic centers—you may wish to have some of those, but to really reach communities, you need to go into…communities

Work with a CRO with a network of diverse clinics and hospitals with whom they have existing relationships and demographic profiles of the population served

Be wary of smaller and mid-tier CROs who claim to have this on tap—in my experience many don’t, or their “database” is a spreadsheet with a hundred or two sites they have worked with before—I personally work with CROs who have detailed databases with literally thousands of sites with detailed information: ask to see this

Ensure that Investigators are identified that are part of the targetted race/ethnicity are available at the sites in the communities of interest

Consider a decentralized trial—COVID19 gave considerable impetus to the development of supporting technologies and many are now available

Develop a presence online and in social media disease support groups to raise awareness of your particular therapy

Identify and develop KOL/champions of the race/ethnicity targeted; develop explanatory videos to be deployed on your website and other online services/patient groups

Ensure your CRO has diverse project team members who can interact with minority sites, ideally in their language of choice

Ensure informed consent materials are available in the patient’s native tongue—this is basic ethics and required by ISO 14155:2020 5.8.2 (d), 21 CFR 50.20 and other regulations require it—yet here in the US with at least 12.9% of the population speaking Spanish at home some Sponsors still don’t want to provide Spanish language information and consent materials: that’s just wrong. Additionally, different cultures perceive and receive information in different ways, and “health information literacy” is especially poor in minority groups—graphics, video, and other technologies should be explored.

Hands up who wants to do questionable and flawed science! No takers? OK, then as researchers, let’s take representative population sampling seriously and employ 

 scientific methods instead. Is it easy? Well, it’s different, and change is often not easy. Is it necessary? Only if you actually take pride in your work and make people’s lives better with proven therapies.

is the Executive Director of Clinical Development for Syneos Health

King, TE, Racial Disparities in Clinical Trials | NEJM accessed 02 Nov 21

https://www.nytimes.com/2013/07/14/opinion/sunday/do-clinical-trials-work.html accessed 01 Nov 21

2020-COVE-Study-Enrollment-Completion-10.22.20.pdf (modernatx.com) accessed 01 Nov 2021

U.S. Census Bureau QuickFacts: United States accessed 01 Nov 2021. Note Caucasian % is expressed as not including Hispanics.

https://www.outsourcing-pharma.com/Article/2020/02/26/FDA-release-of-clinical-trials-participation-demographics accessed 01 Nov 21

Nazha B, Mishra n, Pentz R, and Owonikoko T, Enrollment of Racial Minorities in Clinical Trials: Old Problem Assumes New Urgency in the Age of Immunotherapy, American Society of Clinical Oncology Educational Book 2019 :39, 3-10

Aizer AA, Wilhite TJ, Chen MH, et al. Lack of reduction in racial disparities in cancer-specific mortality over a 20-year period. Cancer. 2014; 120:1532-1539.

https://seer.cancer.gov/statfacts/html/breast.html accessed 01 Nov 21

Allison K, Patel DG, Greene L. Racial and Ethnic Disparities in Primary Open-Angle Glaucoma Clinical Trials: A Systematic Review and Meta-analysis. JAMA Netw Open. 2021 May 3;4(5): e218348. doi: 10.1001/jamanetworkopen.2021.8348. PMID: 34003274; PMCID: PMC8132140.

Spratt DE, Chan T, Waldron L, Speers C, Feng FY, Ogunwobi OO, Osborne JR. Racial/Ethnic Disparities in Genomic Sequencing. JAMA Oncol. 2016 Aug 1;2(8):1070-4. doi: 10.1001/jamaoncol.2016.1854. PMID: 27366979; PMCID: PMC5123755.

https://www.appliedclinicaltrialsonline.com/view/informed-consent-observations-and-response-fdas-draft-guidance-july-2014 accessed 01 Nov 21

https://www.fda.gov/media/127712/download

Guidance for Industry (fda.gov) https://www.fda.gov/media/124795/download (RWE in Drugs & Biologics) https://www.fda.gov/media/99447/download (RWE in Medical Devices)

Articles

Some actions to consider for your next trial or submission.

Politically Incorrect Diversity in Clinical Trials

, Informed Consent Information Sheet: Guidance for IRBs, Clinical Investigators, and Sponsors (July 2014) close on September 15, 2014.

	To adequately cover the troubling, and at times sordid, history of how and why informed consent came to be a requirement in human research would require many pages, and is beyond the scope of this paper. It should suffice to say that obtaining informed consent is required ethically and legally in almost all forms of human research in almost all countries. The simple notion that one should 

 to what is done to oneself in terms of medical research is comprised of a number of fundamental elements, such as understanding risks and benefits, the right to not participate or withdraw and not have one’s care compromised as a result, and a number of other elements, but is at times difficult to implement, or at least assure. To assist in overcoming difficulties, recently the FDA issued a document entitled “Informed Consent Information Sheet: Guidance for IRBs, Clinical Investigators, and Sponsors (Draft Guidance, July 2014)”. Within the Draft Guidance, FDA has drawn particular attention to the language used in informed consents, and cited concerns over low levels of health literacy. This paper will discuss the informed consent process with specific attention to the phrase found in 21 CFR 50.20 “The information that is given to the subject or the representative shall be in language understandable to the subject or representative” and other similar elements of the Draft Guidance.

	FDA’s commentary on language/Discussion

	The Agency has drawn particular attention to defining what “understandable” language for subjects actually is.

 FDA’s views “understandable” in informed consent forms(ICFs) to mean that “information is presented to potential subjects in a language and at a level the subjects can comprehend (including an explanation of scientific and medical terms)”.

 While this verbiage is not markedly different from the guidance issued in 1998, it does remove the term “typical subject”, viz, “The IRB should ensure that technical and scientific terms are adequately explained or that common terms are substituted. The IRB should ensure that the informed consent document properly translates complex scientific concepts into simple concepts that the typical subject can read and comprehend.”

	It is not entirely clear what the Agency what hoping to achieve by removing the term “typical”-perhaps they view the term as too subjective, or, perhaps more likely, the aim was to cast the new definition in such a manner that it does not imply that the responsibility for understandability relies solely with the IRB. FDA goes on in the new Draft Guidance to draw attention to low levels of health literacy in the US population: in 2003 around 1/3 of the adult population, or 77 million people had basic or below basic health literacy. These classification levels of health literacy are characterized by the ability to locate easily identifiable information in short commonplace prose texts, following instructions in charts or forms, and locate easily identifiable quantitative information for simple operations (at most, one-step, and primarily addition) that is very concrete and familiar or where the arithmetic operation is specified or easily inferred.

 While FDA contends that “limited health literacy affects adults in all racial or ethnic groups” the 2003 National Assessment of Adult Literacy (NAAL) research cited does indicate dramatically higher rates of basic or below basic levels in Hispanics (66%), Blacks (60%) and American Indian/Alaskan Natives (48%) compared to Asian/Pacific Islanders (31%) and Whites (28%), indicating a marked disparity across racial and ethnic groups, at least at the time the research was conducted.

	The relevance of the NAAL health literacy research-conducted in 2003 and published in 2006 and its  applicability over a decade later in 2104 and beyond should be questioned; it remains, however, the newest citation on the Federal government’s health.gov website

 and is widely cited by many government agencies as the basis for action. It should be noted also that this is the only form of health literacy research quoted by the Agency in its Draft Guidance. The most recent wide-scale literacy research reported that is in any way analogous to the 2003 research cited by FDA is that conducted by the Program for the International Assessment of Adult Competencies (PIAAC) in 2012.

 Whilst the PIAAC work is not directly comparable, it does show that levels of literary and numeric skill in the Hispanic, Black and White segments of the adult population aged 16-65 have improved somewhat with rates of 43%, 35% & 10% respectively at levels 1 & 2 in literacy and 56%, 59% & 19% numeracy. It should be noted, however, that this research was focused on literacy in a technology-rich environment, and while similar, the definition for levels 1 & 2 do not correlate directly with the definitions of basic and below-basic discussed above: whilst level 1 had a good correlation to below basic, level two introduces more complex concepts, especially in the numeracy measure.

 Thus, as a working model, it may be reasonably assumed that somewhere between 19-66% of the adult population in the United States in the present day may experience some difficulty in comprehending this sentence, and most certainly the abstract concepts upon which it depends and ramifications attendant thereto.

	Both in the 1998 Guidance and in the 2014 Draft Guidance, FDA refers to language in the informed consent that is “at a level” that subjects can understand. ICH-E6 Guidelines for Good Clinical Practice (GCP) state in 4.8.6 that any written or oral communication be “as non-technical as practical and should be understandable to the subject”; ISO 14155:2011(E) is even more succinct, viz, per 4.7.2 e) an informed consent process shall “use native non-technical language that is understandable to the subject.” All of these are laudable goals. As a rule of thumb often promulgated, advice is usually given out that IC’s be at a reading level of “8th grade level or below”

 although many are now advocating still lower levels down to sixth grade,

 and one prospective comparative study found that “simplified forms, when used alone, do not significantly improve the comprehension of participants –at any level”.

 A comparison of relative reading scales is available in appendix B for review.

	The question arises as to what problem regulators are trying to solve in ensuring readability and understanding, and the obvious response is that non-medical professionals, and, large segments of the population in general, have difficulty in understanding such terminology. We should then ask “why do healthcare professionals use technical and scientific terms in the first place?” The answer is, of course, precision, that is, the ability to succinctly and precisely explain, unambiguously, exactly what concept is being conveyed. I am reminded of a conversation with my father whilst fixing a motorcycle together early in my medical career, when I was telling him about a jejunostomy I had witnessed being performed at Royal Adelaide Hospital. This caused him to launch into a diatribe about medical professionals using technical jargon to “lord it over mere mortals” and keep everyone else in the dark. My response was to ask him to hand me the nearby wooden handled 12 inch long tool with a 1.5 pound double ended hardened steel striking surface, one side of which was flat and circular in lateral aspect and the other of which was semi-circular and convex in shape in order that I might cause a transfer of kinetic energy through the rapid motion of the head of said tool against the 8 inch steel device I currently had attached by means of friction to a hexagonally shaped protuberance of a recalcitrant helically wound fixation device which had undergone an oxidative process such that it would not move without application of more energy than I could supply without the use of said wooden handled metal headed tool: silently, he handed me the hammer to loosen the rusty bolt.

	The purpose of the informed consent process, of which the consent form itself is but a part, is to 

. This is a worthy goal that all in healthcare should aspire to in order that patients can make informed decisions and choices about their healthcare, and, where necessary, participation (or non-participation) in health-related research. An example of the lack of precision that can occur in the quest for non-professional understanding can be found in paper used by Texas Tech University Health Services to guide the writing of informed consents

-in this document the authors advise writers to substitute “under the skin” for “intradermal” and yet a mere two lines later offer the same advice for the term “subcutaneous”; as any clinician knows, these two medical terms are hardly the same thing, although it could possibly be argued that to a patient the two procedures (assuming injection) are so similar as to appear the same. Perhaps more concerning, and within the same section as the previous example is the advice to rename “malignancy” to “cancer”, which is a term almost guaranteed to cause fear and alarm on the part of the patient, who may not be aware that many neoplasms (a term that certainly should be avoided in informed consents) are in fact benign, despite the fact they are cancers. A critic of the foregoing comments might observe that obviously even the “lower level” terms would need to be explained. That critical observation would be justified, but also support the argument under development here - the danger in attempting to convey complex and precise information at a lower level is the that simple but verbose statements must necessarily replace precision (witness my anecdote above and imagine the space it would take to actually write out the words at a 6th grade level instead of “hand me the hammer”-the summary is bad enough!).

	The situation goes from the sublime dozens-of-pages-long “consent form” to the dangerous when inattentiveness or inability to digest such lengthy documents precludes a true comprehension on the part of the patient, or worse still, lack of precision yields a vague and “fuzzy” message. As care and treatments become ever more complex, and regulatory oversight increases, the situation becomes worse, with concerns expressed by some that consent documents are becoming just too long for comprehension.

 Indeed there has been published evidence to suggest that if a consent form exceeds four pages in length, it is unlikely to be read.

 Others have placed the tipping point at a mere 1000 words.

 Recently however, in a cultural and medical environment not markedly dissimilar to the United States, it was found that consent forms had increased from a median of 7 pages to 11 pages during the period 2000 to2005, largely as the result of increasing regulatory oversight aimed at increasing patient safety.

 Perversely, patient safety may well have been negatively impacted if in fact patients never truly gave 

 consent as result of lengthy consent documents.

	In the quest to ensure that a potential study participant fully 

 what is being asked of them in research, we should acknowledge that yes, medical professionals use technical language, and yes, they do that because it is both precise and concise, but also that to the non-professional it can essentially be an almost foreign language. To attempt to drive down the “level” or “reading grade” yet further however, runs the risk of producing exactly the outcome that we seek to avoid-imprecision and verbosity leading to a lack of understanding. Further, we are increasingly relying upon IRBs and ECs to pass value judgments on readability for which they, as highly educated professionals, for the most part, may be ill-equipped to make, that is, acting in the role of communication and education specialists for large segments of the population whose level of literacy approaches the level of if not cognitive, then certainly functional, impairment. Indeed, the documents these IRBs are asked to review may themselves be the product of a flawed process. Despite recommendations by government agencies and IRBs alike to utilize products such as Microsoft Word to check for readability of informed consents,

 such an approach is not without its critics who maintain that the algorithm is flawed and affected by document format.

 Beyond that criticism, and the obvious problem of relying on an unvalidated algorithm, Microsoft itself freely advises that its readability measure functions for English text only.

 As the attentive reader will recall, the 2003 NAAL research FDA cites indicates 66% of Hispanics are at basic or below basic health literacy and that ISO 14155:2011(E) requires use of the subject’s native language: reliance on Microsoft Word to gauge readability of Spanish informed consent documents in the US is not possible. As an exercise, in the writing of this paper I wrote a small paragraph one might expect to find in an informed consent in order to informally test the readability assessment function of Microsoft Word 2013. In the first instance, I set out to write a simple description of a potential risk of participation for a research subject using as few polysyllabic words as possible and clearly connected concepts (common guidance given to writers of informed consent forms). In the second instance I attempted to reword the first description into the simpler language often found in consent form documents, and which will feel “familiar” to many readers. In neither case did I revise or edit my writing in order to achieve a different result once the text was written. Curiously, the first example, which is clearly unacceptable language, achieved a score of grade level 8.3, whilst the second, which I believe would pass muster with most IRBs, scored a grade level of 10.8, even when replacing “exsanguination” with “lose so much blood”. Please see Appendix C for details. Clearly, for the “foreign language” of medicine, a human interpreter is needed, beyond ink-on-a-page or electrons dancing through a CPU.

	With respect to “foreign languages”, there are astonishingly high levels of low health literacy in the Hispanic demographic group who make up some 17% of the population in 2013,

 per the NAAL and PIAAC research. This is an important point to consider, as, if the NAAL research is to inform FDA’s guidance, no fewer than 2 out of 3 Hispanic persons function day to day ant basic or below basic levels of health literacy. Given the emphasis on the consent form being given in language that is understandable to the subject in FDA, ISO and ECH guidance/regulations, perhaps routine production of Informed Consent Forms in Spanish should be required by FDA, especially as some 12.9% of households speak Spanish at home as of 2012.

 As a counter-point, it could be argued that the Hispanic population is highly regionalized to the Southwestern states, Florida, Puerto Rico and major urban centers such as New York and Chicago, with a marked paucity of presence in the Midwestern states

 (see appendix D), and thus the burden of producing Spanish consents only be undertaken for clinical studies conducted in those areas. In the same census document however, one can find Hispanic population growth rates for the period 2000-2010 showing marked expansion in geographic distribution in Midwestern and Southern states especially, with growth rates remaining static, or nearly so, in areas of high concentration in urban and Southwestern states. Assuming that geographic expansion has continued from 2010 to the present day, Hispanics as a percentage of the overall population may be much more generalized and dispersed than the urban and Southwestern concentration data initially suggests. Routine production of Spanish ICFs – the problems of actual literacy notwithstanding for the present – is, however, not a decision to be taken lightly. Clinical trials are costly and complex beasts already, and adding this requirement would increase that burden significantly. For example, in a recent 274 randomized patient US & EU study (250 enrolled at time of review) it was found that of 173 with ethnicity data only 2 were identified as Hispanic, and the cost burden to add a Spanish ICF to the 24 sites that did not have one would have approached $72,000 for translation and IRB review costs, to say nothing of the staff overhead in managing the files, monitoring, etc.

 Even if a Spanish ICF were available, it would only go part way toward ensuring Hispanic subjects truly understood the research in which they are being asked to participate-no less than Whites or any other group, they need an interpreter for the foreign language of medicine.

	Traditionally, the role of interpreter has fallen to the physician or other healthcare professional, and, unfortunately, in the case of the informed consent process, FDA acknowledges that many have either abrogated that responsibility or regard 

 consent as “synonymous with obtaining a subject’s signature on the consent form”.

 Beyond just obtaining a signature, in the same discussion section FDA goes on to say that informed consent involves providing a potential subject with adequate information to make an informed decision, and facilitating their comprehension, as well as allowing for any questions to be raised. It is this latter that is of most interest, as it presupposes the role of expert that healthcare professionals are expected to play, and by implication hints at the role of translator/interpreter they could, and I will contend, 

	There is no person better positioned to communicate effectively to the patient the true meaning and nuances of a clinical trial and its alternatives than a fully trained clinical investigator, or a Registered Nurse (RN), if that person is equally trained on the study in question and has expertise in the given therapy area (this latter is not necessarily true of all research coordinators (RC), hence the differentiation between RN and RC). If a picture is worth a thousand words, then a conversation with a clinical expert who will be conducting the study must be at least as valuable, and perhaps more so. The regulations under 21 CFR 50 

 for an oral consent process in 50.27 (b) (2) - except where the IRB makes an exception under 21 CFR 109 (c) – and a short form written consent stating that the elements of the consent were given is to be signed by the patient, the investigator and a disinterested witness,

 and the latter two parties only sign an IRB approved copy of a written summary of what is to be said.

	The thrust and emphasis of 21 CFR 50 however, is toward “long form” or traditional informed consent forms: this thinking can be seen in the reference to the 46 FR 8949 preamble where the information presented orally must “be of the same quantity and quality of information as when the long form is used”.

 This fundamental mindset is also reflected in FDA’s opening summary comments on page 3, “the consent 

 must contain information to allow the subject to make an informed decision” (emphasis added). If the long form consent is 15-20 pages long, which is not uncommon, FDA seems to be firmly indicating that the 

. It is difficult how to conceive how this could be achieved without a verbatim reading of the long form, and while there is no published research on patient/subject attention span and comprehension such as that cited above for lengthening printed documents, one suspects the would be a similar correlation. The position of this author, however, is a little different from that of the Agency. Given the preceding evidence demonstrating the difficulties of reliance on written information – that rather than permitting an oral consent process, and that consent essentially being a reading of the long form consent information, FDA rather mandates an oral consent process, and returns responsibility for the relationship between physician/healthcare provider and patient back to those individuals. In this fashion, a tailored approach can be adopted and those who require very detailed, very simplistic and very thorough explanation of particular items of concern to them can receive it. Conversely, those who may require or desire a less time-consuming approach that still covers the essential elements can receive that – it is only through a patient-provider relationship and knowledge of the individual patient’s needs that such a judgment can be appropriately made. To give the impression that in order to combat decreased literacy levels further reductions in reading levels are necessary, rather than adopting an approach whereby patient understanding can be enhanced, is unlikely to be a successful strategy.This is especially sowhen the very act of decreasing the “reading level” may render it impossible to convey the information required – a fact acknowledged by some leading IRBs

 and increase ICF length yet further, producing the opposite of the desired result. FDA has expressed interest in discussing alternatives with interested parties who may want to employ “new technologies”

 which one assumes may involve web-based presentations and animated/graphical/dynamic information. Web presentations of this nature would be at variance with current Draft Guidance, however, despite their probable efficacy, as they would not form the same “quantity and quality” as a “long form”, even though these are the types of data presentation most readily consumed by those at the “below basic” level of health literacy (see appendix A).

	An approach that requires trained investigators or other healthcare professionals to conduct an informed consent process orally is not without its concerns. Beyond the time required of both a physician or RN, and a witness to administer such a consent process, the question arises as to the capability of those professionals to undertake that work. Communication skills vary widely between different individuals, as is common knowledge, and need not be proven. A component of training in delivering informed consents orally would need to be put in place for each study, both for basic communication skills, and for the study specifics. At a minimum, that training should include a check-for-understanding where the person giving the oral information asks the prospective subject to explain back to them in their own words what they understand about the study with its risks benefits and their rights and responsibilities are. Documentation that an adequate explanation of the clinical study has been given to the patient, along with the opportunity to ask (and have answered) any questions that might arise would need to be robust. Such documentation may need to include a time component to provide some measure of quality (it is unlikely an explanation of any degree of acceptable quality could take place in 2 minutes, whereas 10 minutes may possibly be adequate) or a written simple test, especially if graphics or animations are used in support of the oral presentation. It is possible that is this day and age of near ubiquitous smartphones with video recording capabilities that a recording of each session may be a mechanism by which quality assurance can take place-although this may raise legal issues with which many would prefer not to deal. Nonetheless, these are relatively minor hurdles to overcome to ensure patients who are prospective subjects can make truly informed decisions about their participation in clinical research studies.

	Whilst the actions listed above may seem burdensome, and indeed they do increase workload and expense, our fundamental duty as researchers must have its basis in good ethical behavior. If there are means by which we can enhance patient understanding of what it means to be a subject, then we should actively pursue and consider adopting those means, as indeed the Agency is attempting with this latest Draft Guidance. It may well be that adoption of the recommendations in this paper are actually cost-neutral or perhaps even positive. It is likely that in a discussion process, a healthcare professional can make a far better determination of a patient’s suitability for participation, their commitment to meeting their visit and treatment obligations, and their propensity to dropout or become lost to follow-up, than a process where a prospective subject merely reads a form, is given the opportunity to ask questions, and then signs. Whether termed “attrition”, “lost-to-follow-up”, or simply “dropout”, rates of subject withdrawal from clinical studies can be high, sometimes exceeding 30% in Phase III trials.

 With recent costs per patient reported at over $48,000

 in Phase IIIb trials, and the incalculable costs in terms of human suffering resulting from delay to market of a prolonged clinical trial due to subject attrition, an informed consent process that concomitantly informs and decreases attrition rates must be seen as highly attractive.

	None of the forgoing argumentation should be taken to be advocating increasing the complexity of informed consent documents. Neither should anything contained herein be taken as advocating the withholding of information from patients. It appears obvious that conveying medical information to non-medical professionals is difficult in writing, and that large proportions of the population are not well prepared to receive it, no matter what the written format or reading level. That being so, the necessary conclusion is that a new approach is required that is not reliant upon an ever-lengthening paper document.

	FDA’s Draft Guidance on Informed Consent is the first made by the Agency since 1998. The focus of this paper has been on a singular aspect of that guidance, that is, low levels of health literacy, which, in specific groups, can reach as high as 66% in the 2003 NAAL research FDA has cited as the basis for its concern. Attempts to drive the “reading level” of consent documents yet lower may prove to be ineffective at assisting patients to make informed choices about becoming subjects due to the difficulty in explaining complex terms in simple language and the very long forms that result. FDA should mandate that an oral presentation of the proposed research be given to each patient, in their native tongue, by healthcare providers with expertise in the therapy area and the study. The oral consent given to each patient should be able to be tailored to that patient’s needs as long as certain fundamentals are covered, and not necessarily replicate the quantity of information in the “long form” consent document. In order to avoid any coercion, FDA should move away from a “recommendation” that a witness to oral presentations be a disinterested or impartial person and mandate that it be so. This will place a not-insignificant burden on research sites, and may preclude some smaller sites with limited staff from participation in studies, and this ramification should be carefully considered. FDA should identify and make available free of charge, a validated online readability analysis tool for the use of investigators, sponsors and IRBs, ideally one which can accommodate non-English languages.

	FDA should consider investigating the need for and viability of non-English Informed Consent Forms in the United States, bearing mind not only the added expense and complexity, but also the burden on IRBs, who may not (and in all likelihood do not) have a makeup conducive to full participation of all board members in reviewing a document written in languages other than English.

	The new approaches to enhancing the quality of truly informed consent process recommended here may be just what the doctor ordered.

Timothy Pratt, PhD, MS, MA, Grad Cert Mgmt, BN, RN, RPN, CCRN, NASPExAM/AP Testamur, Senior Principal Medical Research Manager, NAMSA

	Kutner, M., Greenberg, E., Jin,Y., and Paulsen, C. (2006). The Health Literacy of America’s Adults: Results From the 2003 National Assessment of Adult Literacy (NCES 2006–483).U.S. Department of Education. Washington, DC: National Center for Education Statistics, page 5.

	Goodman, M., Finnegan, R., Mohadjer, L., Krenzke, T., and Hogan, J. (2013). Literacy, Numeracy, and Problem Solving in Technology-Rich Environments Among U.S. Adults: Results from the Program for the International Assessment of Adult Competencies 2012: First Look (NCES 2014-008). U.S. Department of Education. Washington, DC: National Center for Education Statistics. Pp. B-3, B-7

	Paasche-Orlow MK, Taylor HA, & Brancati FL, Readbility Standards For Informed Consent Forms as Compared with Actual Readability, 

	Appendix C-Informal Test of Microsoft Word 2013 Readability Assessment

	Esophageal varices are a distinctly possible morbid outcome of this treatment. Part of this research involves use of anti-clotting agents. If varices occur, they can lead to exsanguination due to those agents. If that happens, cardiac function may cease and death would occur.

	MS Word 2013 Flesch-Kincaid Grade Level 8.3

	As a result of your participation in this research study, it is quite possible that enlarged blood vessels might happen in the tube that connects your throat and your stomach. Another part of this research requires that you are given drugs that cause your blood to be thin which means it will not clot easily. If enlarged blood vessels do form and they begin to bleed, it is possible that you could lose so much blood that your heart would stop and you would die.

	MS Word 2013 Flesch-Kincaid Grade Level 10.8
	 

	Appendix D Geographic Distribution of US & Puerto Rico Hispanic Population by 2010 Census Data

	Ennis SR, Rios-Vargas M, and Albert NG, The Hispanic Population: 2010 Census Briefs, US Department of Commerce, Economics and Statistics Administration, US Census Bureau

	1. Throughout this paper the term “subjects” or “patients” should also be read to include their “or their legally authorized representatives”

	2. Informed Consent Information Sheet: Guidance for IRBs, Clinical Investigators, and Sponsors (Draft Guidance, July 2014), p. 5

http://www.fda.gov/RegulatoryInformation/Guidances/ucm126431.htm

	4. Kutner, M., Greenberg, E., Jin,Y., and Paulsen, C. (2006). The Health Literacy of America’s Adults: Results From the 2003 National Assessment of Adult Literacy (NCES 2006–483).U.S. Department of Education. Washington, DC: National Center for Education Statistics.

http://www.health.gov/communication/literacy/quickguide/factsliteracy.htm

	6. Goodman, M., Finnegan, R., Mohadjer, L., Krenzke, T., and Hogan, J. (2013). Literacy, Numeracy, and Problem Solving in Technology-Rich Environments Among U.S. Adults: Results from the Program for the International Assessment of Adult Competencies 2012: First Look (NCES 2014-008). U.S. Department of Education. Washington, DC: National Center for Education Statistics.

	7. See appendix A for NAAL and PIAAC definitions discussed.

http://www.hopkinsmedicine.org/institutional_review_board/guidelines_policies/guidelines/informed_consent_ii.html

http://www.cancer.gov/clinicaltrials/conducting/simplification-of-informed-consent-docs/page2

https://oprs.usc.edu/files/2013/04/Informed-Consent-Booklet-4.4.13.pdf

http://www.research.uci.edu/compliance/human-research-protections/researchers/drafting-consent-form.html

http://flpublichealthethics.net/index.php/eng/researchers/improving_informed_consent#note2

	13. Davis TC, Holcombe RF, Berkel HJ, Pramanik S, and Divers SG, Informed Consent For Clinical Trials: a Comparative Study of Standard versus Simplified Forms, 

	14. Winslow EH & Hagan P, Making Research Consent Forms More Readable, 1999, Revised 2003, 

https://www.ttuhsc.edu/research/hrpo/irb/files/informed_consent_article.pdf

http://www.cancer.gov/clinicaltrials/conducting/simplification-of-informed-consent-docs/page1/AllPages#fn123

	16. Pandiva A, Readability and comprehensibility of informed consent forms for Clinical trials. 

	17. Albala L, Doyle M, Appelbaum PS. The evolution of consent forms for research: a quarter century of changes. 

	18. Sharp SM, Consent documents for oncology trials: does anybody read these things? 

	19. Beardsley E, Jefford M, Mileshkin L, Longer consent forms for clinical trials compromise patient understanding: so why are they lengthening? 

 Isles AF, Understood consent versus informed consent: A new paradigm for obtaining consent for pediatric research studies, 

https://www.nhlbi.nih.gov/crg/pdf-docs/ghchs_readability_toolkit.pdf

http://www.maine.gov/dhhs/mecdc/irb/IRB%20intranet/readguid.pdf

http://www.marywood.edu/irb/consent-form-readability-tips.html

http://www.lau.edu.lb/irb/guidance-documents/informed-consent.php

http://www.appliedclinicaltrialsonline.com/appliedclinicaltrials/Pick+of+the+Month/Informed-Consent-Reading-and-Understanding-Are-Not/ArticleLong/Article/detail/90594

http://office.microsoft.com/en-us/word-help/test-your-document-s-readability-HP010354286.aspx

http://quickfacts.census.gov/qfd/states/00000.html

http://www.infoplease.com/spot/hhmcensus1.html

 accessed 29 July 2014. Note that the 12.9% includes persons aged 5 and older and includes those who self-identify as speaking English well.

http://www.census.gov/prod/cen2010/briefs/c2010br-04.pdf

	29. Visor J, Director of Clinical Services, NAMSA, Personal Communication, 29 July 2014

	30. Informed Consent Draft Guidance July 2014, p. 2

	31. FDA regulations do not specify who the witness may be, but while “FDA's regulations are silent as to who can serve as the witness. There is consensus, however, that the witness needs to be impartial…” 

https://firstclinical.com/fda-gcp/?show=2008/RE%20ShortFormAndWitnesses&format=fulllist

 (accessed 22 July 2014) which reflects the thinking found in ICH-E6 GCP 4.8.9. The Draft Guidance July 2014 contains the phrase “FDA recommends that an impartial third party, not otherwise connected with the clinical investigation … serve as the witness.” p. 20

	35. National Research Council. (2010). The Prevention and Treatment of Missing Data in Clinical Trials. Panel on Handling Missing Data in Clinical Trials. Committee on National Statistics, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press. P. 39

http://www.cuttingedgeinfo.com/research/clinical-development/trial-operations/

 Clinical Development and Trial Operations (PH192) reports accessed 24 July 2014

It should suffice to say that obtaining informed consent is required ethically and legally in almost all forms of human research in almost all countries.

Informed Consent: Observations and Response to the FDA's Draft Guidance of July 2014

Until relatively recently, taking your company's clinical research into the realm of electronic data capture (EDC) meant going with a full service vendor with a price tag generally starting at $175,000 to $200,000 for a 250 patient, 50 site trial over 24 to 36 months, and as high as your project was complex. If you wanted to conduct smaller scale research these were often just not economically feasible-the same pricing as just mentioned might be provided for a 50 patient, five site trial.

	Of course, the option has been available from some of the larger EDC companies to "do it yourself" (DIY) in an attempt to decrease costs, but these were often only less expensive if you were doing large volumes of research and could afford to have skilled "build" specialists trained and on staff. The net result is that smaller companies either remained in a paper world with all its attendant woes, or were compelled by cost-containment needs to select markedly sub-par EDC systems almost as problematic as the old paper model.

	Of late however, perhaps over the past three to four years, viable DIY EDC options have begun to emerge onto the marketplace that make what has become a commonplace technology in larger organizations more accessible economically to smaller enterprise. Finding, assessing, and selecting DIY EDC though, is an activity that is by no means as simple as it might at first appear. Having lived through this exact process very recently, this article seeks to share experiences in the hope they may prove useful to others in similar positions.

	The company for whom I was evaluating systems, a small venture capital backed medical device company, had previously conducted a number of studies on paper using as a backend a well-known and relatively inexpensive CDMS. The largest and most recent of the studies conducted required a considerable amount of data cleaning after the enrollment period closed-largely because data errors had been introduced due to the nature of a dynamic non-point-of-capture-checked data acquisition. The burden placed on the company in cleaning up this rather complex trial dataset was very significant, and contributed in no small part to the decision to investigate DIY EDC systems.

	Given the upcoming projects, which varied from moderate to quite small in nature, full-service EDC was not seen to be a viable option. I was requested, given my background in EDC/CTMS system design and implementation, to source, evaluate, and recommend a DIY option. Cost was not a primary consideration, but neither was it immaterial, as one might expect in a small organization.

	With years of experience and more than a few systems under my belt, I foresaw little difficulty in conducting this project-get a few demos, jot down a few notes, make a recommendation, and sign contracts. Much to my surprise and chagrin, my assumptions were proved wrong almost immediately.

	It became obvious, fairly quickly, that problems with the initial methodology as conceived were legion. These included:

		The simple task of developing a vendor list was not easy.

		Scheduling and sitting through system demonstrations is a time consuming laborious process.

		The disparity between systems and capabilities makes non-systematic comparison inherently an "apples to oranges" exercise and hence invalid.

		The variation in pricing models made cross-comparisons difficult (this may be a deliberate ploy on the part of the vendors).

		Specialist knowledge is required to get past minutiae and understand true capabilities (or lack thereof).

		Sales reps have a habit of showing you what is great about their system and glossing over weaknesses-this can be hard to detect without an understanding of what particular system characteristics actually mean in the real world, and especially so outside of a structured demonstration.

	Many, perhaps even all, of the issues listed above exist for a company of any size. In a small company however, the problems, or their impact, become exacerbated, mostly due to far more limited resources in terms of personnel, diversity of expertise/experience, and time.

	Developing a list of candidates for assessment was addressed through looking at resources such as ClinPage (Who's in the news? Who are the advertisers?), lists of conference exhibitors (SCDM, Partnerships in Clinical trials, DIA etc.) and most usefully the 

 Annual Buyer's Guide. Using search engines such as Google, Yahoo, Bing, and others proved less than useful with an awful lot of CROs and non-clinical research organizations showing up in results.

	Other online resources include the various groups on LinkedIn that focus on EDC and CTMS-like ClinPage, these are useful for uncovering vendors that might be suitable-caution should be exercised with the postings/discussion though, as many contributors are vendor employees. There are a couple of website resources that list multiple vendors with profiles of products, but they tend to be largely derived from the vendors themselves, not any third party review, and out of date in many cases.

	Considering scheduling, making effective use of the most precious non-renewal resource there is-time-is critical. Quick Internet reviews of corporate websites and industry articles was employed to pare the candidate list down to 25 or so. Scheduling demos and reviewing that many would take months, so further filtering was conducted using simple criteria:

 More than a few have become defunct or been acquired over recent months and years.

Are they in danger of going out of business? 

Have they been delisted from the stock exchange? Have they sustained losses? Have they been shedding staff? Are they on the market for sale?

Many don't offer this (or don't have a good one) but would love to try and sell you a full-service offering.

Are they a CRO that happens to have and EDC system or a technology company?

 Sometimes it makes sense to use a CRO with and EDC system, but for most DIY solutions it's probably best to stick with pure technology companies (they have an incentive to make their product good so you don't call them-whereas hourly billing companies love phone calls).

What do your colleagues think? What have you heard about town? What does your CRO (if you have one) think? 

Utilize your network to seek out colleagues who have used different systems and get their unbiased opinions on ease of doing business with the vendor, ease of use of the product, problems, issues, etc. Your CRO can be invaluable at this point too. This criteria should be used to exclude vendors from consideration, not select them-your needs are different than anyone else's.

Options have emerged that make DIY EDC technology more accessible to smaller organizations.

Timothy Pratt, PhD

Articles

Politically Incorrect Diversity in Clinical Trials

Informed Consent: Observations and Response to the FDA's Draft Guidance of July 2014

DIY EDC Evaluation

Risk-based Approach to Monitoring

When is EDC not ePRO?

EDC: Caught in a Trap