Maxim Belotserkovskiy, MD

Safety surveillance is one of the core objectives of every clinical trial. It is based on registration and assessment of adverse drug reactions (ADRs), defined as noxious and unintended responses to a medicinal product related to any dose.

 Not all ADRs are attributed to the study drug and one of the key characteristics of every ADR is causal relationship, or causality-whether the event under question is obviously related to the study drug, or at least such relationship cannot be ruled out. All available information about ADRs observed with the drug is captured in the Investigator’s Brochure, and once the drug is approved for use-in the package insert. This makes assessment of causal relationship a key factor, as underestimation may subject potential patients to an unexpected and unnecessary risk, while overestimation might narrow indications and therapeutic use, or lead to dose modification (e.g., decreased dosage), which may lead to decreased efficacy.

	The WHO-UMC assessment scale is meant as a practical tool for the assessment of case reports. It is a combined assessment taking into account the clinical-pharmacological aspects of the case history and the quality of the documentation of the observation.

 Other algorithms are more complicated. Such scales have proved their validity, but have a general disadvantage-they are based on individuals’ medical judgment and thus subjective.  Experts may evaluate similar ADRs differently. Such disagreements have been reported previously and ranged from 51% to 95%.

 These reports mainly compared judgments made by physicians or experts without analysis of possible reasons for disagreements. Herein, we compared the assessments of 50 ADRs made by four experts from a clinical research organization (CRO medical monitors) and clinical trial investigators, analyzing the reasons for different evaluations.

	Four drug safety experts, who were not medical oncologists, but had experience monitoring oncology trials independently, analyzed 50 consecutive (in terms of time of registration) blinded serious adverse events (SAE) from three Phase II-III clinical trials conducted in 2014-2015. The studies involved in patients with locally-advanced (inoperable) or metastatic non-small cell lung cancer (NSCLC). One of three trials was a placebo-controlled one and two studies were open-label. All patients, including those from the placebo arm, received one or two chemotherapeutic agents. All patients from the open-label studies received an investigational drug as a part of their chemotherapeutic regimen. Among the 37 cases from the placebo-controlled trial, 17 of the patients (46%) received the study drug and chemotherapeutic agent and 20 (54%) were administered the placebo and chemotherapeutic agent. The information for the SAEs was summarized in narratives, which included demography data, medical history, concomitant medications, study treatment with dates (the name of the study drug was blinded), the course of the AE, and the list of expected AEs for the particular study drug. The investigator’s assessment was deleted from the narrative, so the experts were unaware whether or not the AE was assessed by the investigator as an ADR.

	For each drug-event pair, the questionnaire asked the experts to assess causal relationship based on the following criteria:

		Time to onset: incompatible/compatible/highly suggestive

		Dechallenge: positive/negative/not available

		Rechallenge: positive/negative/not available

		Previous information on the drug and adverse experience: yes/no

	Experts assessed each criterion and also made an overall causality assessment of “related” or “not related.” Events were considered related if a relationship cannot be ruled out. They also provided the reasons for their evaluation. At the time of evaluation, the experts were not aware of the investigator’s and sponsor’s assessments.

	Experts were CRO medical monitors. Their medical background included internal medicine, infectious diseases, pediatric, and intensive care. None of them were medical oncologists, but they received trainings in different aspects of oncology and were medical monitors in various oncology clinical studies. Three experts (experts A, B, and C) had > 10 years of experience as medical monitors. Expert D had six months of experience as a medical monitor. All four have solid experience in clinical practice and expert D has 20 years of experience as a clinical investigator.

	Cohen’s kappa agreement coefficient was used to assess agreement of each expert with the original reporting investigator. Contingency tables illustrating the agreement were also created. The results (near-zero values, CI covers zero in all cases) indicate that the investigator assessment is not related to that of any of 

the experts. This analysis was done for overall assessment only (as we do not have the detailed criteria data for the investigators, just the overall causality assessment). Frequencies of concordant (same for all four experts) and discordant (split into 2 vs. 2 and 3 vs. 1 categories) evaluations were summarized for overall assessment, as well as for the individual Items. Three vs. one assessments were further analyzed to identify the expert who disagreed with the other three. Frequencies of being the disagreeing expert were summarized for the overall causality assessment, as well as for individual criteria. Chi-square test was then used to test the hypothesis that all the experts have equal chances of disagreeing with the three others.

	Adverse events were coded in accordance with 

. The most common system-organ-classes (SOC) were respiratory, thoracic, and mediastinal disorders – 11 (22%), and general disorders and administration

 site conditions – 10 (20%). The proportion of all SOCs is presented in Table 1 at right.

	The disease under study – NSCLC, could explain this distribution of AEs, where the most frequent and expected adverse events are related to respiratory and cardiac systems. Another contributing factor is that we assessed serious AEs, which most likely are more clinically significant and, thus, more likely to involve vital organs and functions.

	The rate of agreement within the group of experts varied according to the criterion under evaluation (see Table 2 below).

	All four experts agreed on causality assessment in 32% of cases and, as expected, the pattern of assessment was different for the causality criteria. Maximal agreement was seen for expectedness (reaction previously reported) and risk factors. Interestingly, assessment of expectedness criterion was based on the data from the Investigator’s Brochure, where all previously registered and expected AEs were enlisted, but nevertheless, in 12% of cases assessment was not unanimous. Assessment of dechallenge and rechallenge was not informative and in the vast majority of cases was recorded as “not applicable.” Only one AE occurred during infusion of the medication and in this case it was possible to assess a dechallenge effect, which was positive, according to all four experts.

	There were several AEs where three experts were in agreement, while the fourth had a different opinion. We analyzed the pattern of such disagreement of one expert with three others on a specific AE (see Table 3 below).

	Statistically significant differences were seen in evaluation of time to onset, alternative cause, and overall causality assessment.

	We did not analyze agreement between the experts and investigators in assessment of separate causality criteria, as only the overall causality assessment was available from the clinical investigators. Overall agreement between all four experts 

and the investigators in assessment of AEs was 32% (16 cases), however, agreement was much higher for experts A, B, and C, with >10 years of experience in the capacity of medical monitor, at 70%-74%, while for expert D it was only 44% (See breakdown in Table 4).

	Based on the above, expert D seems to be more liberal in attributing causality as likely related to study drug. This may partially be explained by a difference in perspective, since this expert has been a clinical investigator.

	Causality assessments made by the sponsors concurred with those of the investigators in 48 cases out of 50 (96%). In two cases when it was different, the experts’ assessment was the same as the sponsor’s and in the other, two experts concurred with the sponsor and two with the investigator.  

	After unblinding of treatment assignments from placebo-controlled trials, it turned out that in four cases out of 20 (20%) where placebo had been given, the investigators assessed causality as “related.” Interexpert agreement in evaluating placebo group also was not unanimous and AEs were assessed as “related” in 1 (expert A), 4 (expert B), 5 (expert C), and 14 (expert D) cases.

	More than 30 drugs have been recalled from the US market since the 1970s, some in use for several decades. According to the FDA, “A drug is removed from the market when its risks outweigh its benefits. A drug is usually taken off the market, because of safety issues that cannot be corrected, such as when it is discovered that the drug can cause serious side effects that were not known at the time of approval.”

 Darvocet (propoxyphene), an opioid pain reliever, was recalled in 2010 after being on the market for 55 years, due to serious cardiotoxicity; over 2,000 deaths were reported between 1981 and 1999.

 Duract (bronfenac), a non-steroidal anti-inflammatory drug, was withdrawn in 1998 after being on the market for a few months, due to significant hepatotoxicity; four deaths resulted and eight patients required liver transplants.

 It seems evident that the more comprehensive the information about possible ADRs is obtained during clinical development, the more likely the drug will remain on the market. Thus a careful assessment of AEs and particularly causality are of great importance.

	Methods used for assessment of causality can be grouped into three categories: expert judgement, probabilistic approaches, and algorithms or scales.

 Despite numerous methods, there is still no “gold standard” and, to a large extent, assessment is based on physicians’ evaluation. In clinical trials, a probabilistic approach is used-most frequently the WHO-UMC system and its modifications.

 Early studies showed poor agreement between experts in assessment of causality of ADRs. Several assessment criteria are addressed in a “yes-no” fashion: timely relationship, alternative explanation, dechallenge, and rechallenge.

	There are six possible outcomes for causality assessments in the WHO-UMC system: certain, probable, possible, unlikely, unclassified, and unassessable. “Certain” has a unique requirement: the “event must be definitive pharmacologically or phenomenologically, i.e., an objective and specific medical disorder or a recognized pharmacological phenomenon.” An example is “grey baby syndrome,” which is a rare but serious side effect that occurs in newborn infants (especially premature babies) following the intravenous administration of the antimicrobial chloramphenicol due to the lack of a hepatic enzyme to metabolize the drug. This criterion is applicable only in cases of already known, previously observed, specific reactions. Such knowledge is not common in clinical trials, especially at early stages of clinical development. However, certain types of reactions specific for a class of medications could meet this criterion during a clinical trial.

	For regulatory purposes, the six WHO-UMC causality categories are collapsed into just two categories-related and not related.  Certain, probable, and possible map to “related,” and unlikely, unclassified, and unassessable make up “unrelated.” There are certain difficulties in distinguishing between probable and possible and frequently investigators do not consider them. Accordingly, we decided to simplify the analysis and classify AEs as related or not-related. We also modified assessment criteria and added to traditional time to onset, dechallenge/rechallenge, and alternative cause-such criteria as risk factors and previous information on the drug. Evaluation of temporal relationship is based on a general assumption that it takes between five and six half-lives for a medication to be eliminated from the body; consequently, it is generally accepted that one can exclude a temporal relationship with a medication, if the time since the last administration is ≥ 5 x T

 However, this approach is frequently not observed in clinical practice and is a matter of approximation. Another limitation is possible variations of pharmacokinetic parameters, including half-life, due to pathologic conditions and drug-drug interactions. 

 showed that patients with severe or acute respiratory disorders generally use multiple drugs and have increased susceptibility to ADRs.

	Positive dechallenge and rechallenge theoretically could be considered “a gold standard” of causality, but in practice their utility is very limited. Dechallenge could be mainly used for infusion reactions, when stopping drug may, or may not, lead to a quick resolution of clinical signs and symptoms. It is not useful when an ADR occurs some length of time after a drug administration-sometimes days or weeks. Dechallenge also may not be applicable in single-dose studies or long-lasting reactions, like hepatotoxicity or congenital anomalies. A rechallenge rarely occurs due to ethical concerns. An analysis of the performance of the CIOMS scale in the Spanish DILI (drug-induced liver injury) Registry showed that rechallenge data were absent in >95% of all cases.

 For drugs that are administered with a certain periodicity, repetition of the ADR after one of the subsequent administrations may reflect a positive re-challenge. At the same time, there are two main limitations: the incidence of even drug-related reactions is not 100%, thus if seen after one administration, the ADR will not necessarily repeat after the next one, and some drugs are given once weekly or even more rarely (e.g., chemotherapeutic medications) and this criterion could be assessed only retrospectively. In our study, only one ADR out of 50 was assessed for dechallenge and it was a psychiatric disorder that occurred during drug infusion and resolved after the drug was discontinued. As the relationship was obvious, the drug was not resumed and thus rechallenge was not applicable.  

	Under the category of “previous information on the drug and adverse experience,” we meant information about the drug that was available at the time of the ADR. For investigational drugs, investigators receive this information from a periodically updated Investigator’s Brochure. Usually it is updated annually and for many shorter studies updates occur after study completion. Another source of ADR-related information was providing investigators with individual case safety reports (ICSRs) delivered as MedWatch 3500A or CIOMS-I forms during the study conduct. However, these forms are being distributed for the events that have already been considered SUSARs (suspected unexpected serious adverse reactions). Those that were considered not related are not provided to investigators.

	Interestingly, in 20% of patients who later appeared to receive placebo, causality was assessed as related by the investigators, and discordance between the experts was also high. A detailed analysis of this phenomenon is beyond the scope of this article, but we can theorize, that in placebo-controlled trials, evaluation of AEs should be made with more caution, as there is a chance that a patient was receiving a placebo instead of an active drug, and unblinded safety data received during the study can’t adequately characterize the safety profile of a drug.

	Divergences between experts in ADRs causality assessment have been reported previously

 compared judgment of five experts using VAS score and confirmed marked interexpert disagreement (kappa=0.20).

., agreement between three clinical pharmacologists in assessing ADRs was 50% and complete agreement between them and the treating physicians was slightly lower, in 47% of cases.

 reported the rate of agreement between experts with kappa indices of the causality criteria ranged from 0.12 to 0.38.

 four experts rated 50 case reports first using only general guidelines and showed poor agreement.

 Influence of subjective judgement was shown by 

 who compared physicians’ opinion with the scores obtained by the causality assessment method.

 They showed that physicians more frequently assessed causality as “likely” and “very likely” related and complete agreement between physicians and causality assessment method was achieved in only 6% of cases. Previously, we showed discordance in causality assessment between investigators and a retrospective evaluation with Naranjo algorithm.

	In our present study, an expert with great experience as investigator and less experience as medical monitor, showed significantly lower agreement with investigators than three other experts, in whom the rate of agreement with investigators was 70%-74%. The actual agreement is illustrated by kappa scores, and they are all low, meaning investigators and experts are probably using different approaches. The high percentage of concordant records does not indicate agreement, but it indicates that both the expert and the investigator are considering the majority of events as “related.” The criteria of time to onset and alternative cause of the event were the most frequent reasons for disagreement between the experts. On the other hand, criterion of previously reported ADR, which was based on the factual information (Investigator’s Brochure), caused very few discordant evaluations. The overall agreement on causality assessment was low, at 32%.

	Consensus between experts is difficult to obtain when evaluations are made based on personal judgements of unspecified criteria. Developing methods of standardized assessments could solve this problem. In an attempt to minimize the subjective component, several algorithms for evaluation of ADR causality have been proposed: the Naranjo criteria, the Kramer algorithm, the Jones’ algorithm, the Yale algorithm, and several others.

 However, existing algorithms also show significant disagreement in assessing causality of the same ADRs with the most frequently seen discordance in evaluating timing of event, dechallenge, and alternative cause.

 The most frequently used causality assessment algorithm is the Naranjo tool, which is simple and convenient, but its validity has been variably assessed; some studies demonstrated good 

 while others questioned its reliability.

	Considering the low validity and reproducibility of general methods for assessing ADRs, several groups attempted to develop algorithms for evaluating certain types of ADRs. Several algorithms have been suggested for causality assessment of drug-induced liver injury (DILI): Maria and Victorino, DDW-J, and CIOMS-RUCAM (Roussel Uclaf Causality Assessment Method).

 At the same time, causality assessment in different pathological conditions (e.g., lung cancer and Crohn’s disease) is evaluated using the same general approaches. Both patient characteristics (e.g., age, comorbidities, and concomitant treatment) and disease characteristics (e.g., tumor burden, specific laboratory parameters) may influence incidence and severity of ADRs. At present, none of the existing causality assessment tools takes any of them into account.

	 Our study confirmed that the overall agreement between clinical investigators and drug safety experts is low. Disagreements between experts may be due to different clinical background, perspective, and expertise. At the same time, we observed disagreement in assessment of such objective criterion as previously reported reactions. Our findings concluded that assessment of AEs’ causality is subjective and influenced by individual judgement of investigator/expert, and, thus, does not reflect a safety profile of the drug. It is reasonable that safety assessments are made on an aggregate data.

, MD, PhD, is Director of Medical Monitoring and Consulting, PSI CRO, United States; 

 is Group Leader, Biostatistics, PSI CRO, Russia; 

, MD, is Director of Medical Monitoring, PSI Pharma Support America; 

, is Director of Medical Monitoring and Consulting, PSI Pharma Support America; 

, MD, is Medical Officer, PSI CRO, Argentina; 

, MD, is Senior Director, Medical Affairs, PSI CRO Deutschland, a subsidiary of PSI CRO AG

	1. Glossary of terms used in Pharmacovigilance. Assessed from: 

http://www.who-umc.org/graphics/28401.pdf

	2. The use of the WHO-UMC system for standardized case causality assessment. Assessed from: 

http://www.who-umc.org/graphics/26649.pdf

	3. Belhekar Makesh, Taur Santosh, and Munshi Renuka. A study of agreement between the Naranjo algorithm and WHO-UMC criteria for causality assessment of adverse drug reactions // 

, 2014; 46 (1): 117-120, doi: 10.4103/0253-7613.125192

	4. Macedo Ana Filipa, Marques Francisco batel, Ribeiro Carlos Fontes, and Teixeria Frederico. Causality assessment of adverse drug reactions: comparison of the results obtained from published decisional algorithms and from the evaluations of an expert panel //

, 2005; 14: 885-890, doi: 10.1002/pds.1138

	5. Son M, Lee Y, Jung H, et al. Comparison of the Naranjo and WHO-uppsala monitoring center criteria for causality assessment of adverse drug reactions //

	6. How does FDA decide when a drug is not safe enough to stay on the market? 

AboutFDA/Transparency/Basics/ucm194984.htm

	7. FDA Drug Safety Communication: FDA recommends against the continued use of propoxyphene. 

http://www.fda.gov/Drugs/DrugSafety/ucm234338.htm

	8. Hunter Ellen, Johnson Philip, Tanner Gordon et al. Bromfenac (Duract)-associated hepatic failure requiring liver transplantation. 

, 1999; 94: 2299-2301, doi: 10.1111/j.1572-0241.1999.01321.x

	9. Agbabiaka TB, Savovic J, Ernst E. Methods for causality assessment of adverse drug reactions: a systematic review. 

	10. Khan Amer, Adil Mir, Nematullah K. et al. Causality assessment of adverse drug reaction in pulmonology department of a tertiary care hospital. 

. 2015; 6: 84-88, doi: 10.4103/0976-0105.160744

	11. Andrade Raul, Robles Mercedes, Lucena M Isabel. Rechallenge in drug-induced liver injury: the attractive hazard. Expert 

	12. Arimone Yannick, Begaud B, Miremont-Salame G. et al. Agreement of expert judgment in causality assessment of adverse drug reactions. 

	13. Karch F, Smith C, Kerzner B. et al. Adverse drug reactions – a matter of opinion. Clin Pharmacol Ther. 1976; 19: 489-492

	14. Arimone Yannick, Miremont-Salame Ghada, Haramburu Francoise. et al. Inter-expert agreement of seven criteria in causality assessment of adverse drug reactions. 

. 2007; 64: 482-488, doi: 10.1111/j:1365-2125.2007.02937.x

	15. Louik Carol, Lacouture Peter, Mitchell Allen et al. A study of adverse reaction algorithms in a drug surveillance program. 

	16. Miremont G, Haramburu F, Begaud B. et al. Adverse drug reactions: physicians’ opinions versus a causality assessment method. 

	17. Kosov Maxim, Riefler John, Belotserkovsiy Maxim. Facing adversity. 

	18. Srinivasan R, Ramya G. Adverse drug reaction – causality assessment. 

	19. Pere J, Begaud B, HaramburuF, Albin H. Computerized comparison of six adverse drug reaction assessment procedures. 

	20. Kane-Gill S, Forsberg E, Verrico Margaret, Handler S. Comparison of three pharmacovigilance algorithms in the ICU settings: a retrospective and prospective evaluation of ADRs. Drug Saf. 2012; 35: 645-653, doi: 10.2165/11599730-000000000-00000

	21. Kane-Gill Sandra, Krisci Levent, and Pathak Dev. Are the Naranjo criteria reliable and valid for determination of adverse drug reactions in the intensive care unit? Annals of Pharmacotherapy. 2005; 1823-1827

	22. Garcia-Cortes Miren, Stephens Camilla, Lucena M. et al. Causality assessment methods in drug induced liver injury: strengths and weaknesses. 

	23. Regev Arie, Seeff Leonard, Merz Michael et al. Causality assessment for suspected DILI during clinical phases of drug development. 2014; 37: 47-56, doi: 10.1007/s40264-014-0185-4

Articles

Study measures the differing judgment levels between clinical investigators and drug safety experts.

Interexpert Agreement on Adverse Events’ Evaluation

	Cohen’s kappa agreement coefficient was used to assess agreement of each expert with the original reporting investigator. Contingency tables illustrating the agreement were also created. The results (near-zero values, CI covers zero in all cases) indicate that the investigator assessment is not related to that of any of the experts. This analysis was done for overall assessment only (as we do not have the detailed criteria data for the investigators, just the overall causality assessment). Frequencies of concordant (same for all four experts) and discordant (split into 2 vs. 2 and 3 vs. 1 categories) evaluations were summarized for overall assessment, as well as for the individual Items. Three vs. one assessments were further analyzed to identify the expert who disagreed with the other three. Frequencies of being the disagreeing expert were summarized for the overall causality assessment, as well as for individual criteria. Chi-square test was then used to test the hypothesis that all the experts have equal chances of disagreeing with the three others.

 most common system-organ-classes (SOC) were respiratory, thoracic, and mediastinal disorders – 11 (22%), and general disorders and administration site conditions – 10 (20%). The proportion of all SOCs is presented in Table 1 at right.

	We did not analyze agreement between the experts and investigators in assessment of separate causality criteria, as only the overall causality assessment was available from the clinical investigators. Overall agreement between all four experts and the investigators in assessment of AEs was 32% (16 cases), however, agreement was much higher for experts A, B, and C, with >10 years of experience in the capacity of medical monitor, at 70%-74%, while for expert D it was only 44% (See breakdown in Table 4 below).

 The most frequently used causality assessment algorithm is the Naranjo tool, which is simple and convenient, but its validity has been variably assessed; some studies demonstrated good agreement,

, is Group Leader, Biostatistics, PSI CRO, Russia; 

, is Director Medical Monitoring and Consulting, PSI Pharma Support America; 

, MD, is Senior Director Medical Affairs, PSI CRO Deutschland, a subsidiary of PSI CRO AG

Drug safety surveillance, a core focus of clinical trials, can be influenced by subjective judgement, as this analysis of differing expert assessments of adverse drug reactions-and the reasons why-shows.

There are several reasons for the increasing number of clinical trials globally. Some of these factors are aging of the population, the high prevalence of chronic diseases, the embrace by physicians and payers of evidence-based medicine, and the patients´ common belief that they will benefit from the medical technologies and treatment. Patients volunteer into clinical trials to gain early access to medication, treatment, and medical care.

 In some countries, the participation in clinical trial, is still the only possibility to be treated.

			Approximately 340 new drugs and biologics came to the market between 2000 and 2011.

  On the other hand, only 2 out of 10 marketed drugs return revenues that match or exceed R&D costs.

  There are approximately 3,200 NCEs in development

 and the number is constantly on the rise.  Economic difficulties and financial crises have led to an increased need for precision in the planning of CTs to minimize risks and eventual losses.

			Currently, 66% of clinical trials are conducted in North America, Western Europe and Australia.  The U.S. hosts 49% of clinical trial sites

 and generates approximately 40% of clinical data.  However, other regions have arisen in recent years.

  The main reasons behind this globalization trend are the availability of highly skilled professionals in new research markets,

 lower costs, faster patient recruitment that allows for earlier drug application approvals and concentrations of huge populations for future drug sales.

			These new countries, with growing economies, increase healthcare investments and new markets spring up.  As they adopt  International Conference on Harmonization (ICH) good clinical practice (GCP) guidelines, clinical trials follow.  

			Just as the number and regional distribution of studies shift, so too does the number and placement of studies by Phase. 

Changes in the Global Clinical Trial Landscape

			We adopted the geographical regions as classified by clinicaltrials.gov

				East Asia (China, Hong Kong, Korea, Mongolia, Taiwan)

				North America (NA - Canada, USA, Mexico)

				North Asia (post Soviet countries, incl. Russia and Ukraine)

				Pacifica (including Australia and New Zealand)

				Southeast Asia (including Indonesia, Malaysia, Philippines, Singapore, Thailand and Vietnam)

			Table 1: # of Newly Registered IND CTs per region: 2006 vs. 2012 (Absolute Numbers & % Relative Change per Region)

			Between 2006 and 2012, the absolute number of clinical trials registered annually has grown by 39%.  We observe this increase in every part of the world except in Central America and Pacifica, where the number dropped about 15%.

  The biggest relative change between 2006 and 2012 of newly registered clinical trials is observed in Japan (+94%), East Asia (+217%), South Asia (+45%), Southeast Asia (+42%) and the Middle East (+40%).  North America demonstrated a rise of +36% and Europe, a +32% increase in 2012 compared to 2006.  It is safe to say that a Northern Hemisphere focus has been gradually shifting towards the East, i.e. from North America and Europe to Asia.  This has to do with economic development allowing countries to invest more in healthcare

 and the increase in populations, which make these countries excellent targets for future growth of pharmaceutical sales.

			On the other hand, quite recently, some of these countries exhibit levels of instability. Lack of GCP standards, ethical misconduct and even frauds are still big concerns in these regions. India, for example, has no specific law pertaining to scientific fraud.

			Percentual substitution of CTs per region.

			Analyses of the changes in number and distribution of clinical trials from 2006 – 2012 unveil some curious developments.  To understand the true growth of clinical trials in a region, we tie the relative change of newly registered clinical trials within a region (in %) to the overall, absolute number of registered IND studies per year globally (100%).  This calculation gives a percentual substitution rate per region and tells where real growth has occurred.

			Table 2: Percentual substitution rates by regions in 2006 and 2012.

			Figure 1: Relative number versus percentual substitution per region in 2006 and 2012.

			Table 2 and Figure 1 demonstrate that the growth of the relative number of clinical trials in 2006-2012 was registered everywhere (blue bars), except Central America and Pacifica.  However, expressed in percentual substitution (red bars), the data actually demonstrate declines for almost all regions.  For example, there was a 32% growth of the number of studies in Europe.  On the other hand, in light of the total number of newly registered trials around the world, it turns into a 5% decline.  Similarly, a relative 36% growth in NA actually represents a 2.4% decline in percentual substitution.

			Markedly, the share of East Asia, South Asia, Southwest Asia, Japan and the Middle East is growing, especially in East Asia.

			Working on a detailed analysis of Europe, we made an intriguing discovery.  We split Europe into three large entities: Western Europe (WE), Central and Eastern European members of the EU (CEE-EU) and the non-European states in Central and Eastern Europe (CEE-non-EU).   The last group is not discussed in this manuscript.

			In 2013, Western Europe participated in 86% of the total number of studies in Europe and approximately 34% globally.  Treatment methods are similar to those of the US and thus, logically, a complement to US sites.

			Table 3: Numbers of newly registered studies in WE countries in 2006 and 2012.

			Figure 2: Percentual Substitution of Clinical Trials in WE countries: (2006 at baseline vs. 2012)

			Figure 2a: Percentual Substitution of Clinical Trials in WE countries: (2006 at baseline vs. 2012) continued

			Table 3 and Figures 2 & 2a show, by percentual substitution, the minimal growth of clinical trials in 2006-2012 in WE.  i.e. approximately 1.5% overall.  Countries like France, UK, Sweden and Spain enjoyed an increase in the number of clinical trials.  However, countries like Denmark, Germany, Greece, the Netherlands, and Switzerland remained stagnant while Portugal, Austria, Finland and the remainder have declined.

			This is an interesting observation and most probably is connected to the financial crisis of 2008-2012. Some countries still haven’t fully recovered. In some other countries the approval procedure is becoming  more complex through years. For example in Germany, there is a need to have BfS (The Bundesamt für Strahlenschutz, Federal Office for Radiation Protection) approval

 for some studies (extended procedure since 2011). Such approval can take around 6 months and represents unwelcome delay in the study start-up.

			This region participates in about 13% of clinical trials conducted in Europe and in about 5% of those conducted globally.  However, this number was as high as 7.5% in 2006.  In relative terms, the decline here goes as far as 28% over the last six years.  It has been observed in all countries of this region, especially Poland, Croatia and the Baltic states (Table 4 and Figure 3).  This is a most surprising finding since the CEE-EU region is considered to be especially well positioned for clinical trials 8.

			Table 4: Numbers of newly registered studies in CEE-EU countries in 2006 and 2012.

Examining why newly registered trials have fallen in Central and Eastern Europe, relative to other regions.

Decline of Trials in CEE: Fluctuation or Trend?

	The current practice of waiver on immediate reporting of cases of death due to the main disease progression may jeopardize data quality from discrepancies in the overall number of deaths registered and documented across the lifecycle reporting of a clinical trial. This article will describe the practical aspects of reporting deaths due to the main disease progression to ensure the data quality of the drug development program.

	The International Conference on Harmonisation (ICH) E2A Guideline titled “Clinical Safety Data Management: Definitions and Standards for Expedited Reporting”

 defines a serious adverse event (or reaction) (SAE) as any untoward medical occurrence that at any dose results in death, is life-threatening, requires inpatient hospitalization or prolongation of existing hospitalization, results in persistent or significant disability/incapacity, or is a congenital anomaly/birth defect.

	Serious adverse events are critically important for the identification of safety problems that might be significant enough to lead to important changes in the way the medicinal product is developed (e.g., change in dose, population, needed monitoring, consent forms). Such reactions should be expeditiously reported to regulatory authorities if they are suspected to be related to the medicinal product and unexpected as per the applicable product information.

	Reporting of death-an SAE in its most severe form-is the No. 1 postulate of drug safety or pharmacovigilance (PVG) as per ICH E2A. However, there are a number of trials in which death of the study subjects is an expected event and likely due to the main disease progression (e.g., late stage of cancer, coma). Reporting of disease-related mortality as single events is generally uninformative, 

and neither contributes meaningfully to the developing safety profile of an investigational drug nor to human subject protection. Thus, the sponsor and regulatory authority rather frequently reach the agreement that such cases will be aggregated and compared to cases in a control group and submitted only if the event occurs more frequently in the drug treatment group.

 Many times, the study protocol or the investigator’s brochure identify cases of death due to the main disease progression as not requiring expedited reporting. Investigators will not complete and submit an SAE report form for such cases unless they are considered related to the study treatment by the investigator.

	On the other hand, at the end of the trial, the sponsor should summarize the study results in the clinical study report (CSR). The CSR is the document-above all other study information-that should include all comprehensive information of all SAEs (including all deaths) from each particular clinical trial and is reported to regulatory authorities by the study sponsor. In the ICH E3 Guideline for Industry titled “Structure and Content of Clinical Study Reports,”

 section 12.3.2 states that “there should be brief narratives describing each death, each other serious adverse event, and those of the other significant adverse events that are judged to be of special interest because of clinical importance.”

	At this stage, if there was a waiver on expedited reporting of deaths due to the main disease progression, discrepancies may occur, such as:

		Death is an SAE in accordance with ICH E2A, and without any exception, information about all cases of death should be listed in the CSR (including cases of death that do not require immediate reporting as per the study protocol or the investigator’s brochure). However, not all cases of death have been captured in the safety database, as the cases of death due to the main disease progression were not reported within the trial expeditiously. We understand that our opponents may refer to the statement “Events (of death) that were clearly unrelated to the test drug/investigational product may be omitted or described very briefly” in section 12.3.2 of ICH E3. Nevertheless, the necessity to have all safety information on hand can arise at any stage of the investigational medicinal product life cycle based on analysis of aggregated data performed both for individual studies (if there are enough events to be informative) and across all studies.

 The larger sample size provides the best chance of detecting safety signals.

		Comparison of disease-related mortality is performed during the interim and/or final analysis of statistical data derived from the clinical study database. In circumstances where the mortality rate in the study treatment group is higher than in the control group, the case would be considered a suspected unexpected serious adverse reaction (SUSAR) and investigational new drug (IND) safety reports should be prepared and submitted to regulatory authorities in narrative format for aggregated data in an expedited way. The individual cases that were analyzed should also be submitted at the same time (e.g., completed FDA Form 3500A for each case).

 However, as already discussed, not all cases of death were captured in the safety database if information about deaths due to the main disease progression were not collected within the trial.

	It is the responsibility of the study sponsor to maintain the safety database in order to monitor the safety of medicinal products and to detect any change to their risk-benefit balance (including all events of death); however, information regarding deaths due to the main disease progression will not be comprehensively captured in the safety database. The lack of clarity and consistency in reporting of deaths in clinical trials can also lead to discordance in deaths documented across reports of a trial.

	As a contract research organization (CRO) and safety provider, we have already faced situations several times when it was very difficult to collect sufficient information about the cases of death due to the main disease progression that had not required the immediate reporting as per study protocol. Such information should be collected before the CSR finalization and sometimes after the study database lock (i.e., probably a long time after the occurrence). For queries, the investigators had to recall the information of patients who discontinued months (sometimes years) ago, which was rather time-consuming and annoying for the investigators.

	From our point of view, the following steps are necessary to improve the process of cooperation between the different parties responsible for collection, evaluation, benefit/risk assessment, and regulatory reporting of deaths due to the main disease progression that are not reported by the investigator in an expedited way:

		Cross-functional operation procedure on collection of such occurrences within clinical trials should be developed by any CRO or the study sponsor (if they run clinical trials on their own).
		 

		Safety management plan should match with, or contain a reference to, safety data reconciliation guidelines. Both documents should be finalized before the first patient in.
		 

		Data management team should provide PVG team with the listings of deaths on a regular basis (e.g., monthly or quarterly). The PVG team should review the listings and reconcile them with previously reported SAE cases of death on an ongoing basis.
		 

		Additional information should be requested from the clinical/site team for all cases of death that were not reported by the investigators on an expedited basis to avoid delayed queries after the end of the trial.
		 

		Brief narratives are compiled for all cases of death either by the PVG team or by medical writers in accordance with the safety management plan.

	Such simple steps can eliminate the possibility of losing data pertaining to disease-related mortality and could significantly improve the quality of CSRs and lifecycle documentation for the investigational product.

	Because the regulations require reporting certain adverse events in the aggregate rather than as individual cases, it is important for sponsors to plan the process of collection and evaluation of such safety data in the very beginning of the investigational product life cycle. It is crucial to remember that brief narratives describing each death should be provided in CSRs and in IND safety reports in situations where the mortality rate in the study treatment group is higher than in the control group. For cases of death that are specifically defined in the protocol as the study endpoints, and that are neither collected in the safety database nor on the adverse event pages of the case report form, it also could be useful to design an additional case report form, so that the investigator could provide a brief narrative for the case in the pre-specified format.

	A participant death is a serious event in a clinical trial and needs to be unambiguously reported either in IND or aggregated safety reports. Waiver on immediate reporting of cases of death due to the main disease progression can be a challenge in situations where the process of collection and evaluation of all cases of deaths was not carefully planned by the study sponsor in the very beginning of the investigational product life cycle. Failure to report all cases of death in the safety database leads to discrepancies in the number of deaths registered and documented across reports (the development safety update report, CSR) of a trial thereafter, affecting the quality of the reports. Incomplete reporting of deaths may overemphasize health benefits when benefits and harms of the investigational medicinal product are summarized in the large sample size or may influence the timelines of expedited reporting in circumstances where the mortality rate in the study treatment group is higher than in the control group.

	1. International Conference on Harmonisation.  “Clinical Safety Data Management: Definitions and Standards for Expedited Reporting.” E2A, Step 4. October 1994. 

http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/

	2. United States Food and Drug Administration. “Final Rule: Investigational New Drug Safety Reporting Requirements for Human Drug and Biological Products and Safety Reporting Requirements for Bioavailability and Bioequivalence Studies in Humans.” 21 CFR Parts 312 and 320.  October 2013. 

http://www.fda.gov/Drugs/DevelopmentApprovalProcess/HowDrugsareDeveloped

andApproved/ApprovalApplications/InvestigationalNewDrugINDApplication/ucm226358.htm

	3. European Parliament. “Approximation of the laws, regulations and administrative provisions of the Member States relating to the implementation of good clinical practice in the conduct of clinical trials on medicinal products for human use.” Directive 2001/20/EC. April 4, 2001. 

http://ec.europa.eu/health/files/eudralex/vol-1/dir_2001_20/dir_2001_20_en.pdf

	4. European Commission. “Detailed guidance on the collection, verification and presentation of adverse event/reaction reports arising from clinical trials on medicinal products for human use (‘CT-3’).” 2011/C 172/01 Article 16(1). 

http://ec.europa.eu/health/files/eudralex/vol-10/2011_c172_01/2011_c172_01_en.pdf

	5. International Conference on Harmonisation.  “Structure and Content of Clinical Study Report.” E3, Step 4. November 1995. 

	6. United States Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research (CDER), and Center for Biologics Evaluation and Research (CBER). “Guidance for Industry and Investigators: Safety Reporting Requirements for INDs and BA/BE studies.” December 2012. 

http://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/

	7. International Conference on Harmonisation. “Statistical Principles for Clinical Trials.” E9, Step 4. February 1998. 

http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines

	8. Earley Amy, Joseph Lau, and Katrin Uhlig. “Haphazard reporting of deaths in clinical trials: a review of cases of ClinicalTrials.gov records and matched publications–a cross-sectional study.” 

2013;3:e001963. Accessed June 1, 2015. doi:10.1136/bmjopen-2012-001963.

, MD, is Medical Officer/Pharmacovigilance Physician, PSI CRO South Africa (Pty) Ltd; 

 is Director, Medical Writing, PSI Pharma Support America Inc.; 

, MD, is Senior Director, Medical Affairs, PSI CRO Deutschland GmbH; 

, MD, is Manager, Pharmacovigilance Unit, PSI Pharma Support Intl

Examining the practical significance of reporting patient deaths due to main disease progression.

Death Waivers an Incomplete Picture of Drug Safety

	The International Conference on Harmonization (ICH) E2A Guideline titled “Clinical Safety Data Management: Definitions and Standards for Expedited Reporting”

	Reporting of death-a SAE in its most severe form-is the No. 1 postulate of drug safety or pharmacovigilance (PVG) as per ICH E2A. However, there are a number of trials in which death of the study subjects is an expected event and likely due to the main disease progression (e.g., late stage of cancer, coma). Reporting of disease-related mortality as single events is generally uninformative, and neither contributes meaningfully to the developing safety profile of an investigational drug nor to human subject protection. Thus, the sponsor and regulatory authority rather frequently reach the agreement that such cases will be aggregated and compared to cases in a control group and submitted only if the event occurs more frequently in the drug treatment group.

	A participant death is a serious event in a clinical trial and needs to be unambiguously reported either as IND or aggregated safety reports. Waiver on immediate reporting of cases of death due to the main disease progression can be a challenge in situations where the process of collection and evaluation of all cases of deaths was not carefully planned by the study sponsor in the very beginning of the investigational product life cycle. Failure to report all cases of death in the safety database leads to discrepancies in the number of deaths registered and documented across reports (the development safety update report, CSR) of a trial thereafter, affecting the quality of the reports. Incomplete reporting of deaths may overemphasize health benefits when benefits and harms of the investigational medicinal product are summarized in the large sample size or may influence the timelines of expedited reporting in circumstances where the mortality rate in the study treatment group is higher than in the control group.

Maxim Belotserkovskiy, MD

Articles

Interexpert Agreement on Adverse Events’ Evaluation

Interexpert Agreement on Adverse Events’ Evaluation

Decline of Trials in CEE: Fluctuation or Trend?

Death Waivers an Incomplete Picture of Drug Safety

Death Waivers Draw an Incomplete Picture of Drug Safety