Blinded independent central review (BICR) is the process by which radiographic exams and selected clinical data, performed as part of a clinical trial protocol, are submitted to a central location for independent review. The Food & Drug Administration (FDA) advocates BICR of radiographic exams for registrational oncology studies when the primary study endpoint is based on tumor measurements, such as progression-free survival (PFS), time to progression (TTP) or objective response rate (ORR).

  Current FDA guidance recommends multiple independent reviewers evaluating each subject.

  One consequence of a multiple reviewer paradigm is the potential for discordance between readers on the outcome of the subjects. Discordance between readers is adjudicated by a third reader to determine the final outcome. Adjudication rates concern sponsors and regulators because the reasons are poorly understood and there are few published metrics regarding the cause of discordance between BICR radiologists. This article discusses the causes of discordance between BICR readers, outlines the frequency of contributing factors, and suggests steps to mitigate preventable sources of discordance. This data was previously presented at the 2010 American Society of Clinical Oncology (ASCO) Annual Meeting.

Based on BICR data from 40 oncology clinical trials in 12 indications including 12,299 subjects, it was determined that two readers did not agree 23 percent of the time when determining the best overall response, and 31 percent of the time when determining the date of progression. In a subset analysis, data from only breast cancer clinical trials involving 876 subjects was blinded and pooled to identify cases wherein two primary readers were discordant in outcome. Discordance was defined as a difference in the overall best response, response date, or date of progression assessed between the two reviewers. There were 459 incidences of discordance identified. The radiographs were reviewed to determine the cause of discordance between readers (and to establish if the discordance resulted from a justifiable interpretation difference where neither reader was incorrect in their assessment, or if it resulted from an assessment error by one reader). The authors acknowledge there may have been bias introduced into this process, as the "correctness" of the interpretation was judged by different radiologists from the same facility as the original reviewers.  

Table 1. Differences between readers in lesion selection was the leading cause of discordance. 

 We identified the reasons for discordance including differences in lesion selection (37%), the perception of new lesions (30%), the perception of non-target disease progression (13%), image quality or missing image data (11%), differences in lesion measurements (9%), and missing clinical information (<1%). In total, 77% of discordant cases were deemed to be the result of justifiable perception differences between the two readers, while 23 percent of discordant cases were thought to be due to an interpretive error made by one of the readers. The distribution of reasons for discordance and the classification of justifiable perception differences versus interpretive errors is shown in Table 1 and Figure 1.  

Figure 1. The causes of discordance and classificationof justifiable versus interpretive error.  

When radiologists function as independent reviewers, they can potentially identify different target and non-target lesions that each considers representative of the subject's overall extent of disease. There is also the potential for the reviewers to identify the same lesions but classify them differently between target and non-target lesions. In a study by Hopper et al., two experienced radiologists identified the same lesions only 28 percent of the time.

  Since lesions do not respond or progress at the same rate, differences between reviewers with regard to response or progression of lesions can be observed. For example, in a RECIST (Response Evaluation Criteria in Solid Tumors) study, a percent change in the sum of the lesion diameters of target lesions may indicate an 18 percent increase (stable disease) by one reader and a 20 percent increase (progressive disease) by the second reader. In this example, it is likely both readers are correct and the results differ because the lesions chosen by Reader 1 changed at a different rate than the lesions chosen by Reader 2. Nonetheless, the outcomes are discordant, resulting in adjudication. There are multiple other examples where adjudication is forced by attempting to classify radiologist performance into categorical variables of complete response, partial response, stable disease, and progressive disease. An example of discordance resulting from a justifiable difference in lesion selection is shown in Table 2.  

Table 2. Discordance resulted from a justifiabledifference between readers in lesion selection.  

In Table 2, there was discordance in the date of response between reviewers (R1 and R2) due to a justifiable difference in lesion selection. R2 chose more lung disease and R1 identified hilar adenopathy. Because the thoracic lesions, classified as target disease by R2, responded at a more rapid rate than the liver lesions, R2 confirmed a partial response (PR) one time point (TP) before R1. The best response and date of progression between reviewers were concordant, but a justifiable difference in number and classification of lesions resulted in a response date discordance by one time point.  

Figure 2. Discordance resulted from a justifiabledifference in the perception of new lesions. 

For registrational studies, current FDA guidance recommends the use of a sequential unblinding image presentation paradigm. In the sequential unblinding paradigm, each time point is read in sequence without access to future time points or knowledge of the number of time points available for review.

  At times, this paradigm forces the reviewer to make an assessment based on limited information, such as the assessment of progressive disease versus stable disease in the face of an equivocal new lesion. 

Figure 3. Discordance resulted from a justifiabledifference in the perception of non-target PD.  

Due to the partially subjective nature of radiographic assessments, enforcement of the sequential unblinding paradigm, the presence of image artifacts, benign intercurrent diseases, and occasional image quality issues, some radiologists will determine that a new radiographic finding truly represents a new metastatic lesion, and therefore identify unequivocal progression, earlier than others. An example of discordance resulting from a justifiable perception difference in the identification of new lesions is shown in Figure 2. Similarly, the threshold for assessing non-target disease progression may differ between reviewers as shown in Figure 3. Perception differences are unavoidable and are, in part, the rationale for the two-reviewer and adjudicator paradigm, where the adjudicator has the ability to review the outcomes and select the most correct assessment.  

Figure 4. Measurement differences and an interpretationdifference due to missing clinical data. 

In Figure 2, there was discordance in the date of progression based on a justifiable difference in the perception of new lesions resulting from image quality issues. At baseline (A), a poor quality color paper bone scan was received. At TP2 (B), a gray scale bone scan with slightly better resolution was received. At TP2, R2 identified increased uptake on the bone scan in three areas where CT correlation was not possible, and assessed progressive disease. R1 did not assess progression because, in their judgment, an adequate comparison with baseline could not be made. This represents a justifiable perception difference, as comparison between time points was difficult due to poor quality images and changes in scan technique.  

In Figure 3, there was discordance in the date of progression resulting from a justifiable difference in the perception of non-target progressive disease (PD). R1 assessed progressive disease at TP2 based on an increase in miliary disease (B). R2, however, did not believe that non-target disease progression was unequivocal until TP3 (C). This resulted in discordance of progression date by one time point.  

There is a component of inter-reader measurement variability that can contribute to outcome differences between radiologists. As discussed by Hopper et al., when the same lesions are chosen by two different reviewers, the difference in measurements can be up to 15%, and greatest for lesions with poorly defined margins.

  The potential for measurement variability depends on many factors including lesion size, margination, conspicuity, phase of contrast enhancement, and measurement technique (manual, semi-automated, fully automated), etc.  

Figure 4 represents a justifiable interpretation difference in lesion measurements as well as an interpretation difference resulting from a lack of clinical information. Both R1 and R2 identified the same liver lesion as part of the baseline target disease (A-B, assessed as lesion 001 by R1 and 002 by R2). R1 and R2 measured the lesion on different image slices at baseline and subsequent time points (TP3, shown in C-D). At TP4, the lesion appeared to have increased in size (E-F). R1 measured the lesion which resulted in progression of target disease. R2 indicated the lesion had changed in appearance and decreased in density, and local treatment with radiofrequency ablation was suspected. The lesion was therefore assessed as unevaluable by R2. This represents a justifiable interpretation difference, given the lack of clinical information.  

Given the global nature of large oncology studies and the limited technical equipment available in certain countries, independent reviewers may have to evaluate images they consider sub-optimal based on their local standards. Image variability can potentially impact discordance as indicated in Figure 5. Due to the potential for reviewers to identify different lesions at baseline, missing or poor quality image data may impact one reviewer's assessment but may have no bearing on the assessment made by the other reviewer. Missing or incomplete clinical data may affect the assessment made by one reviewer but not the other.  

Figure 5. Discordance resulted from an inconsistencyin scan technique across time points.  

In Figure 5, there was a justifiable discordance in response between readers based on inconsistent scan technique between assessment points. Both readers identified the same target lesion (002) in the liver, however, inconsistent contrast administration made comparison between assessment points difficult. At baseline (A), the liver was imaged long after contrast was administered, resulting in delayed scans. At TP2 (B), the liver was imaged without contrast, making comparison with the baseline exam difficult. At TP3 (C), the contrast timing was similar to the baseline; however, the images were reconstructed with a different filter, further complicating the assessment. TP4 (D) was performed correctly, with the liver imaged in the portal venous phase. At TP3 (C), R2 attempted to follow the lesion. R1 did not believe reliable measurements could be made, due to the inconsistency in technique across time points, and assessed the lesion as unevaluable. At TP4 (D), both reviewers measured the lesion and R2 confirmed a partial response (PR). R1 was unable to confirm PR due to the prior unevaluable time point, and therefore assigned a best response of stable disease (SD).  

Additional factors affecting adjudication rates

Additional factors which can contribute to discordance between reviewers include the number of adjudication variables, tumor type, drug efficacy, duration of treatment, the number of assessment points, the complexity of the assessment, the subjectivity of the assessment, the types of imaging exams required by the protocol, the precision of the response criteria, and the dating conventions that are followed for assessing the date of progression or response.

Implementing processes to mitigate discordance

 BICR reviewers should be qualified based on experience in the indication and response criteria specified in the clinical trial protocol and BICR Charter. As a minimum requirement, reviewers should hold board certification or its equivalent in a related medical specialty.

  New BICR reviewers should undergo a certification process including training on BICR processes and reading paradigms followed by the completion of qualification test cases. In addition, each reviewer must complete dedicated protocol training according to BICR standard operating procedures for each clinical trial they are assigned to. Lastly, reviewers should be continuously qualified at a corporate level through ongoing monitoring of reviewer metrics.   

BICR Charters should detail the response criteria used, all modifications to the criteria, application of the criteria, approaches to qualitative assessments, dating conventions for response and progressive disease, and algorithms for handling missing radiographic and clinical data. Limiting the number of adjudication variables to only those that affect the primary endpoint is also recommended. For example, if a study has a primary endpoint of PFS or TTP, the only adjudication variable should be the date of progression.  

 Dating conventions that assign a single time point date are recommended in order to prevent date discordance within the same time point. This would be in accordance with current FDA guidance, which advocates the concept of utilizing a single time point date for the assessment of progression dates.

  For example, if reader 1 identifies progression of target disease on a CT scan and reader 2 identifies new disease on a bone scan performed one day later, adjudication is required based on a difference in progression date by one day. If the BICR Charter outlines a convention for using a single time point date for the assessment of progression dates, the date of progression assessed by both readers would be concordant, negating the need for adjudication.  

Corrections in the sequential unblinding paradigm. 

While current FDA guidance indicates assessments should be locked after each successive time point, more recent feedback in public meetings attended by officers of the FDA indicates assessments can be overturned based on data which emerges after the initial assessment. Response assessments at prior time points can be corrected based on information presented subsequently as long as the process is defined in the BICR Charter, the process is driven by data which emerges after the initial assessment, and there are adequate audit trails that can substantiate the changes.

 Allowing for such corrections results in a data set more representative of the actual response of the subjects, while mitigating discordance associated with pseudo-progression, flare phenomenon of bone disease, and new radiographic findings that subsequently appear to be the result of benign intercurrent diseases. These issues are discussed in the updated RECIST 1.1 guidelines.

Best practices for qualitative assessments. 

Discordance often results from perception differences in qualitative assessments where detail and guidance are lacking in published response criteria. In our analysis, 43% of discordance was based on perception differences between readers in the identification of new lesions and the qualitative assessment of non-target disease progression. One way to mitigate these perception differences is to reach consensus within the reader pool on interpretation approaches and establish best practices for qualitative assessments. This can be accomplished through regular case reviews and training using data from past studies.  

As an example, the updated RECIST 1.1 guidelines provide additional detail and guidance regarding the assessment of non-target disease progression in patients with and without measurable disease.

  We presented data at ASCO in 2009 proposing additional rules for assessing non-target disease progression which can be used by the BICR as a supplement to RECIST 1.18. In addition, the RECIST 1.1 guidelines provide specific details regarding the handling of equivocal new lesions and an algorithm for using FDG-PET to identify new lesions.

  This additional detail has been particularly helpful in ensuring a consistent approach among readers. Additional areas where discordance can be mitigated through consensus of assessment approaches include determining the threshold for when subtle findings represent unequivocal new disease, instituting thresholds regarding how a new bone scan lesion is unequivocally confirmed by subsequent modalities, and establishing thresholds to identify when an imaging exam is of inadequate diagnostic quality. The rules governing these approaches should be outlined in BICR Charters. 

Sponsors should consider contracting directly with imaging facilities to ensure imaging exams are performed according to the scanning parameters outlined in the protocol or BICR imaging manual. Just as investigator payment is tied to receipt of case report form (CRF) pages, payment to imaging facilities could be tied to receipt of images at the BICR and resolution of any quality issues.

Educating and training the monitoring staff on the complexity of clinical trials with imaging endpoints is imperative for enforcing site compliance and reducing discordance. Understanding the impact that different quality issues have on response assessments, and the probability of resolution at the imaging facilities, will allow monitors to focus their efforts on critical open issues.

A well-controlled process for providing the independent review committee with clinical data relevant to BICR can reduce discordance rates. In oncology trials, clinical data such as prior radiation therapy, benign findings, cytology, local therapy, and physical exam findings often contribute to the assessment made by independent reviewers. As illustrated in Figure 5, missing clinical data can result in discordance between reviewers.   

It is important for clinical information to be collected in a manner that facilitates future BICR. The independent review committee can assist the Sponsor during CRF development to ensure that all clinical data relevant to the BICR is collected consistently and is organized in a manner that can be effectively used during BICR (e.g., on a per-time-point basis).

  A process for providing clean clinical data and managing updates to data previously assessed should be prospectively defined and agreed upon in a BICR Clinical Data Transfer Plan. 

The overall adjudication rate and the causes of discordance should be monitored throughout the trial. In addition, reader-specific metrics (such as adjudication acceptance and rejection rates, error rates, and inter- and intra-reader variability) should be monitored on an ongoing basis. If outliers or significant discrepancies are identified, investigation, re-training and/or other corrective actions must occur. Monitoring reader performance throughout the trial can identify potential issues early in the process and mitigate unnecessary discordance resulting from those issues. 

Some factors which cause discordance are process-driven or due to justifiable interpretation differences, as was observed in 77% of cases in our review. Examples include lesion selection, image quality issues, inter-reader measurement variability, perception of new lesions, and the assessment of non-target progression. Separate from these factors are reader assessment errors, such as those identified in 23% of cases. This finding compares favorably to reports in the literature regarding radiologist error rates.

  The BICR process of adjudication by a third reader identifies and mitigates these justifiable discordances and interpretive errors. Discordance can be further mitigated by qualifying and training reviewers, detailing response criteria and any modifications in BICR Charters, allowing for corrections in the sequential unblinding paradigm, establishing best practices for qualitative assessments, taking steps to increase site compliance, ensuring a well-controlled process for transferring clinical data to the BICR committee, and implementing processes to monitor reviewer metrics throughout the trial. 

 MS, is Director, Medical Affairs, email: 

 MD, is Chairman, Scientific Advisory Committee, and Chief Medical Officer Emeritus, 

 MD, is Diagnostic Radiologist at CoreLab Partners, Inc., 100 Overlook Center, Princeton, NJ 08540.

* To whom all correspondence should be addressed.

1. U.S. Department of Health and Human Services, Food and Drug Administration,

 Guidance for Industry: Developing medical Imaging Drug and Biologic Products Part III: Design, Analysis and Interpretation of Clinical Studies 

2. U.S. Department of Health and Human Services, Food and Drug Administration,

Articles

Feature Article

Understanding the causes and implementing processes to mitigate preventable sources of discordance.

Discordance Between BICR Readers

Due to advances in bioinformatics and pharmacogenetics, an abundance of molecules continue to move into early drug development. This is bringing greater complexity to studies, as well as the need to produce more robust study information. In order to achieve the go/no-go milestone with greater confidence, clinical development teams seek to conduct more rigorous early phase studies. Designed to demonstrate early signals of efficacy, Proof of Concept studies conducted in targeted patient populations are growing in interest. Assisting biopharmaceutical companies in making better and faster development decisions, these Proof of Concept studies clearly have benefits—helping to identify and select the most promising compounds earlier as well as decrease the failure rates in later stage trials.  

Within the entire development continuum and life cycle of a biopharmaceutical product, the need to take a promising compound from the bench through development includes the crucial Proof of Concept stage. In a typical pathway, once an IND is approved, the exploratory product needs to enter into early phase studies, including first in man and Phase IIa studies, then migrate into multi-national phase IIb studies in patients. In order to maximize efficiency during the process, sponsors must begin with the end in mind—with an understanding of how to mange both the foreseeable and unforeseeable to mitigate risk throughout clinical development. By focusing on innovative study design; clinical development planning; and successful regulatory strategies, a sponsor can ensure the best path to approval is taken.  

TOM GRILL/PHOTOGRAPHER'S CHOICE RF/GETTY IMAGES			

At the early phase, unique operational models for Proof of Concept studies can assist in reducing time and cost in achieving milestones. These operational models should not only include proper evaluation of protocol design but also considerations around incorporating approaches such as Phase I hybrid models combining Phase I sites with traditional investigator sites. Central to accelerating time to milestone while reducing Phase III failure risks is the way that trials are designed in the early phase to facilitate an unequivocal go/no-go decision. These designs are becoming more and more innovative in order to produce earlier, more effective decisions about compounds. 

In one case, for instance, a clinical study team worked to develop an innovative way to accelerate recruitment and regulatory submissions for an early phase development program. The design leveraged geographic time lines for regulatory submissions, while incorporating a combined protocol design to accelerate study time lines. The Phase I and Proof of Concept studies were part of a multi-site, global program. The goal was to recruit a limited number of volunteers for the Phase I first in man and comparative bioavailability studies, and a limited number of patients for the Phase IIa Proof of Concept study portion of the program.  

The initial protocol was designed to combine first in man single doses and a multiple ascending dose study into a single, combined protocol. The protocol was submitted for regulatory and ethical approvals in the United Kingdom to facilitate rapid start-up of the trial. During the multiple ascending dose trial, a second protocol was submitted using the data from the single dose groups to select the dose for a food effect study, which was approved and able to start in parallel with the later groups of the multiple ascending dose part of the study. At the same time, the Proof of Concept protocol was submitted for regulatory approval in South Africa for the study. This part of the program conducted in South Africa was able to benefit from rapid patient recruitment strategies and multi-site patient recruitment approaches to complete the Proof of Concept study well ahead of schedule. 

The entire early phase program required only three regulatory submissions. The target goals for recruitment were met, and the combined protocol approach contributed to exceeding targets for completing the study milestones. This innovative combined protocol enabled a decision about the compound to be made seven months earlier than is typical in the early phase. By conducting multi-site early phase studies, the program was completed 30% faster than typical studies. A typical early development program lasts 24 months from first in man to Proof of Concept, and in this case the study was completed in 17 months through a Phase I first in man and multiple dose combined protocol, a Phase I comparative bioavailability food effect study, and a Phase IIa Proof of Concept study in patients.  

Applying strategic recruitment approaches

In order to make better, faster decisions about their compounds, biopharmaceutical companies need to be able to apply more innovative and strategic recruitment approaches in first in man and Proof of Concept studies. To accelerate recruitment and more rapidly achieve milestone targets in the early phase, multi-site studies can be conducted in various locations worldwide. There should be a focus on rapid study start-up through the application of geographic strategies that will provide access to diverse patient populations and healthy volunteers.  

Early phase recruitment challenges today are combined with many emerging trends, such as personalized medicine. This was apparent in an early phase study that required genotyping of more than 10,000 potential participants to target a specific, well-defined study population of poor metabolizers of Cytochrome P450 2D6 (CYP2D6), an important enzyme involved in the metabolism of drugs. The pharmacogenetics-based study focused on evaluating a possible drug effect on cardiac repolarization to assess safety and tolerability of a neurological treatment, which has been demonstrated to show a 10-fold higher drug exposure in poor metabolizers of CYP2D6. The low prevalence of CYP2D6 poor metabolizers, complex inclusion criteria, and the long duration of the thorough QT/QTc study—with four periods each lasting 15 days—were identified as major challenges in the conduct of the study. However, the clinical conduct of the study was able to be completed in only nine months—from first to last volunteer visit. 

More than 10,000 potential study participants were genotyped to identify nearly 400 CYP2D6 poor metabolizers. Of these individuals, nearly 300 were screened for eligibility to achieve the goal of enrolling more than 130 CYP2D6 poor metabolizers for the study. Rapid and innovative recruitment strategies for early phase development enabled management of the large pool for genotyping and contributed to accelerating development time lines and quickly achieving milestone targets.  

This approach included a large team of recruitment specialists and call centers. Specialized recruitment campaigns were developed, which included community outreach projects, allowing recruitment of a vast pool of interested volunteers to invite for informed consent and subsequently test for eligibility. This study required close coordination between PK/PD Phase I specialists, internal cardiology specialists, and a dedicated cardiologist. A specialized trial design, state-of-the-art technologies, and in-depth scientific expertise were applied, and as a result, an accelerated and effective evaluation about the future of the study drug was generated. 

Maximizing partnerships in the early phase

Overall, to bring safe and effective treatments to market more efficiently, it is critical that biopharmaceutical companies continue to make better and faster go/no-go decisions. The industry needs to apply innovative approaches, including in the areas of specialized trial design and recruitment strategies, to be able to return solid, reproducible outcomes, which bode well for success in later-phase trials.  

The new era of the "learn/confirm" model is requiring robust strategic input from partners in early phase development. This input includes integrated clinical operations and regulatory insights to maximize the efficiency and effectiveness of what is learned about new compounds in development. More and more, biopharmaceutical companies are relying on their partners to bring greater levels of innovation to the design and execution of early phase studies. A partner should help obtain as much information as early as possible in the drug development process.  

Partner teams will need to work more closely together to determine the appropriate use of biomarkers and adaptive trial designs. These study designs will need to be supported not only by biostatistical expertise, but also eClinical technologies, the use of medical imaging as a surrogate endpoint, and the increased use of biomarkers. In fact, biomarker analyses are anticipated to become increasingly important in clinical trials to understand the biological activity and safety profile of candidate therapies. Effective partnerships in the early phase will focus on selecting relevant biomarkers for development programs, interpreting results, and determining implications for therapies. Key goals include generating reproducible and reliable data that can be interpreted for pharmacokinetic or pharmacodynamic purposes, and ultimately making earlier assessments of new compounds in development. 

With the need to obtain safety and efficacy data in patients sooner, early phase patient studies will continue to increase. Sponsors will look to partners to provide an enhanced level of dedicated support for Proof of Concept studies, combining regulatory strategies with drug development and clinical pharmacology aspects. Also, specific scientific and therapeutic expertise, to design and implement these studies, for new drug entities across an even broader range of indications will be seeked.  

Increasing the number of patients per site and reducing the overall number of sites in a study serves to decrease variability—improving study quality and reproducibility. As a result, partners will need to provide integrated and harmonized research units to handle complex global studies involving healthy volunteers as well as patient populations. In order to accelerate patient recruitment in early development, sponsors will work more with partners to select optimal regulatory environments across the globe, access multiple and varied patient populations, and take advantage of seasonal benefits as well as rapid time line potential. "Super Site" networks in particular will grow in importance to provide sponsors with high enrollment potential for their clinical studies. These networks combine a broad, global array of investigator and academic sites. The approach draws on dedicated recruitment specialists and relationships with local, hospital-based physicians. Call centers and databases allow for a high number of patients to be recruited in a rapid time frame. Through these networks the number of sites as well as the time and cost involved can be significantly reduced. 

Ultimately, to identify the strongest opportunities sooner and mitigate risk of phase III failures, sponsors will need to engage partner teams in early development that can couple scientific expertise with forward thinking regulatory and business planning. Overall, by incorporating more innovation into early phase trial design, sponsors will be able to accelerate better development decisions.  

, MB ChB, is Corporate Vice President and Worldwide Head, Early Phase, at Parexel International, 192 West Street, Waltham, MA 02451, email: 

Incorporating original study design accelerates better development decisions.

Innovative Early Phase Trial Design

Compliance with Good Clinical Practice (GCP) on safety data is important for the knowledge of the toxicity profile of a product. All adverse events (AEs) including abnormal laboratory findings should be documented, sorted, and analyzed to ensure the protection of patients. The main objective of this work was to identify and classify the hematological and blood chemistry data not documented as AEs in sponsor's adverse event and clinical laboratory databases of five clinical trials. 

From 458 screened laboratory parameters, 450 were unidentified as AEs and eight of them were documented as AEs in the adverse event database. The abnormal clinical laboratory parameters found in these databases were identified and classified as AEs, using the table of toxicity indicated in each protocol. Most of the events were found mild and moderate; while only three of them were life threatening. The most frequent AEs were anemia and increased levels of alkaline phosphatase. These results show that there are deficiencies in the recognition and classification of data as AEs by clinical investigators, causing underreporting in safety data.  

GCP is an international standard of ethical and scientific requirements for designing, conducting, recording, and reporting of clinical studies. It ensures compliance in human protection, safety, and well-being as well as credibility and reliability in documentation.

 Adverse events are some of the most important data generated in clinical trials for the protection of patients; safety assessment of a product is maintained after the sanitary registration. If exposure to the product has more risks than benefits, it should not be registered or it should be withdrawn from the market if it was already commercialized.

 Although the regulations stipulate that AEs must be documented, in several occasions due to the dynamics of work, there is not an adequate traceability of all safety data. It is necessary to conduct the monitoring and tracking; the data is documented in the medical record until the final report. Traceability is the ability to verify the history, location, or application of an item by means of documented recorded identification (traceability of adverse events, shown in figure 1).  

Figure 1. Every detected AE must be processed according to the proper protocol. 

Abnormal findings of the clinical laboratory are AEs as described by the International Conference on Harmonization (ICH), and can be temporarily associated with the use of a drug therapy or procedure without taking into account the relationship with the administered treatment.

  These laboratory values are less identified and documented as AEs. The purpose of this study is to determine the hematology and blood chemistry data that was not documented as AEs in databases. 

This was a retrospective study of five clinical trials already finished (same sponsor). The investigational product of trials had the same active ingredient, but the formulations and treatment schemes were different. The data was formed by 141 patients (12 of clinical trials 1, 34 of clinical trial 2, 25 of clinical trial 3, 42 of clinical trial 4, and 28 of clinical trial 5); 95% was diagnosed as non-small cell lung cancer, in advanced stages of their illness (stage III and IV). The implementation of these clinical trials was approved and supervised by the Ethics Review Committee of the hospitals involved and by the Regulatory Agency. Patients signed the informed consent before inclusion, giving effect to all the fundamental ethical principles of the Declaration of Helsinki. 

One of the most important objectives to be measured in the different studies was toxicity, as part of the evaluation; all evidence of AEs should be collected to comply with GCP. 

Hematological and blood chemistry toxicity was measured through clinical laboratory exams ordered to subjects as indicated in each protocol (table 1). In this study the values of the first six months of treatment were taken. Different laboratory indicators were evaluated, the parameters that were the most common in all five clinical trials were selected and analyzed, such as hemoglobin, leukocyte, platelets, alanine aminotranferease (ALAT) and aspartate aminotransferase (ASAT), creatinine, and alkaline phosphatase.  

Table 1. Blood collection times in the five trials. 

We analyzed the database designed to gather information about AEs, and only hematological and blood chemistry parameters were selected. The database collecting hematology and blood chemistry parameters were also analyzed, taking those parameters that were out of range to be identified and classified. 

Identification and classification of AE. 

The selection of AEs per patient took into account the baseline values. Only those identified as AEs, abnormal clinical laboratory findings that increased their value respect to baseline, were recorded. 

The table of toxicity from the Common Toxicity Criteria v2 established in the protocol of each clinical trial was taken as criteria to identify and classify the AE. The variables hematology and blood chemistry selected for the analysis were reviewed, and values out of range of reference were identified and classified as mild/grade 1, moderate/grade 2, severe/grade 3, and life-threatening/grade 4. 

The events were grouped according to these tables without taking into account the clinical trials. Charts were prepared to show the events and their relative frequency as well as the intensity of the events.  

The hematology and blood chemistry AEs that were documented in the adverse event database were compared with those found in the clinical laboratory's database for obtaining underreporting. 

Hematological and blood chemistry AE found in the database. 

In the adverse event database, eight AEs were documented: six episodes of anemia, five of them were classified as moderate, and one case was not classified. An increase of the value of ALAT in a patient was found that was not classified, and an increase in the value of creatinine in another patient, was classified as mild as shown in Figure 2, top.  

Figure 2. The distribution for hematology and blood chemistry’s indicators of adverse events detected in the database. 

Hematology and blood chemistry AEs found in the clinical laboratory database. 

458 AEs were detected according to toxicity tables. Figure 2, bottom, shows the hematological and blood chemistry disorders disaggregated by the indicators analyzed. 167 episodes of anemia (36.5% of all AE) were found. 130 events were related to increased values of alkaline phosphatase (28.4%), 51 events were related to AST (11%), 48 events were related to ALAT disorders (10.5%), 44 events were related with abnormal values of creatinine, 17 episodes were related to leukopenia (3.7%), and one episode was related to thrombocytopenia (0.22%). 

 Taking into account the toxicity tables, AEs were classified according to their intensity (figure 2). Anemia and high values of alkaline phosphatase were the most frequent events occurring in patients, most of them classified as mild and moderate; severe and life-threatening events were also found, but in a lower frequency. All episodes of leukopenia, the increase of platelets count, increased values of AST and creatinine were mild and moderate. Only two AEs related to ALAT values were classified as severe. 

Figure 3. Hematological and blood chemistry AE’s classified according to the grade of event intensity. 

Comparison between the documented events in the AE database and AEs in the database. 

In table 2 it is noted that the investigators had documented only six anemia episodes, corresponding to 3.6% of the 167 that occurred. They documented that increased values of the ALAT accounted for 2.1% of 48 events and increased creatinine values represented 2.3% of the 44 events. For the rest of the events any percentage was documented. These results showed the existence of an underreporting of 98.25% of AE data, researchers identified and documented only 1.75% of existing ones.  

Table 2. Difference between CL’s data documented as AE in AE’s DB and those found in the CL’s DB 

AEs are as old as the first remedies used in treating diseases. Its detection in any stages of clinical research is very important for the protection of humans. This investigation analyzed the database of five clinical trials and discrepancies in the documentation process of hematology and blood chemistry where AE were found. Three hypotheses have been assumed as responsible for this problem: clinicians do not identify the abnormal findings of clinical laboratories as AEs; little integration between the laboratory participant and the clinical staff in the clinical trials; and there is not a safety monitor to check the traceability of data of AEs. These situations could lead to underreporting of hematological disorders and blood chemistry AEs.  

The clinical investigator tends to document the results of clinical laboratories in the medial record of a patient and files the result issued by the laboratory. Neverthelesss,  he/she doesn't identify nor classify the event as AE and thus he/she does not transcribe the information in the AE forms. Therefore, the data is not documented in the database as an AE. This leads to an underreporting of AEs in final reports.

Patients included in analyzed clinical trials had cancer in advanced stages; it is usual for the oncologist that assists these kind of patients to find altered hematological

  and blood chemistry parameters, in addition to the treatment that they receive. This means, that clinicians often fail to identify the AE that may arise and take it as an event related to the disease and not related to the treatment. They do not realize that any event that happens to the patient during an investigation is an adverse event. This can be a warning of future adverse reactions when the product is in pharmacovigilance stage.  

The identified underreporting in hematology and blood chemistry data shows an insufficient working relationship between the clinical investigator and the clinical laboratory researcher. A qualified personnel that is trained to be an essential part of the scientific research team for the evaluation of new drugs is needed at each site. They must ensure not only an adequate handling of laboratory samples and the logistics related to this process within the hospital, but must go further and help to confirm or reject a diagnosis to establish a prognosis and monitor disease progression and treatment outcomes of patients.

 The investigator in the lab plays an important role within the scientific team, helping the clinical investigator to identify altered outcomes of laboratory parameters as AEs or to simply guide or alert the principal investigator about the possibility of an AE.  

Monitoring also plays an important role in the quality of the documentation of safety data of a product because it constitutes an external control to verify the traceability of data and the compliance with GCP. Often monitors have a high number of clinical trials to check. Most of them are multicenters, which goes against the quality of this activity, and sometimes important data is not monitored as planned. 

It is necessary to establish the figure of a safety monitor, whose profile should be directed to perform quality checks of all data. Since the AEs are detected, their traceability must be monitored as shown in figure 1, as well as the identification of them where it has not been already done. The monitor should let the clinician be aware of this situation. This monitoring should be conducted in each patient as it is indicated in the protocol. 

The abnormal findings of hematology and blood chemistry parameters are not documented as AEs in the analyzed database resulting in an underreporting in the safety data of the product under study. 

Barbara Wilkinson Brito,* Tania Crombet Ramos, Carmen Viada González, Olga Torres Gemeil, Liana Martinez Pérez, Mayelin Troche Concepción, Aliz Mathild Vega Rodriguez, 

Center of Molecular Immunology, 216 esq 15, Siboney, Atabey, Playa, Havana, Cuba. 

Hermanos Ameijeiras Hospital, Havana, Cuba. 

National Coordinating Center of Clinical Trials (CENCEC).

*To whom all correspondence should be addressed.

1. International Conference on Harmonisation: Guideline for Good Clinical Practice. ICH harmonised tripartite guideline. 1996, 1-57. 

http://www.ich.org/MediaServer.jser?@_ID=554&@_MODE=GLB

3. H. Silva, "Fases del Desarrollo de un Medicamento La Buena Práctica en la Investigación Clínica," Conferencia 6, Segmento 3 (2001). 

4. National Cancer Institute: Investigator's Handbook, 2002: Section 11, Adverse Events. 

http://ctep.cancer.gov/investigatorResources/docs/hndbk.pdf

5. C. Hudis and J. Gralow, "Cancer and Treatment-Related Anemia: Clinical Considerations," 

6. P. Gascón, "Evaluating Erythropoietic Agents for the Treatment of Anemia in the Oncology Setting," 

. 1ra Ed.. Ciencias Médica, Cuba, 2004. 

8. National Institutes of Health (NIH), Independent Safety Monitors Guidelines. National Institute of Allergy and Infectious Disease ATTAMENT III (2004).  

9. National Institutes of Health (NIH). Independent Safety Monitors Guidelines National Institute of Dental and Craniofacial Research (NIDCR), 21 May 2008.  

10. EMEA: Guideline on Data Monitoring Committees. Committees for Medicinal Products for Human Use. London, 2005. 

http://www.emea.europa.eu/pdfs/human/ewp/587203en.pdf

11. O. Scharf and D. Coleva, "Adverse Event Reporting in Publication Compared with Sponsor Database for Cancer Clinical Trials, 

Analysis of hematology and blood chemistry's AEs in sponsor's databases.

Supplements-09-01-2010

Discordance Between BICR Readers

Innovative Early Phase Trial Design

Documenting Adverse Events