RBM—An Update of Experiences Among European CRAs

INTRODUCTION In 2014 the German Federal Association of Contract Research Organizations (Bundesverband Medizinischer Auftragsinstitute, BVMA), supported by the European Clinical Research Organization Federation (EUCROF), conducted a survey among Clinical Research Associates (CRAs) in Europe to get a picture about their experience and assessment of risk-based monitoring (RBM) processes [1]. Two years later, in 2016, the survey was repeated in order to find out, how knowledge and practical experience with RBM have changed over that time period. This article compares the results of this second survey with replies from 2014 and analyzes the progress of RBM implementation. METHODS Both surveys included 21 questions from the areas working efficiency and site contact as well as data quality and patient safety. Although some questions were slightly modified in the 2016 survey for better clarity, answers and results of both surveys are well comparable. The 2016 questionnaire was performed as an internet-based online survey from August to October 2016 and was answered by 231 CRAs, whereas the 2014 survey had been distributed via email and replies from 180 CRAs had been collected from August 2014 until January 2015. This article shows the results of several selected questions and compares the answers given in 2014 to the responses in 2016. The total number of respondents of a survey question is labeled as “N” whereas “n” indicates the number of respondents for each answer option. Percentages for the proportion of “n” within “N” are shown as well. As not every CRA answered each survey question, the total number of respondents (N) may vary between questions. RESULTS Experience of CRAs Table 1 gives an overview about the work experience of participating CRAs. In 2014 as well as in 2016, respondents were monitoring mainly in Germany and more than half of the respondents had three or more years of experience as a CRA (71%, n=128, N=180 in 2014; 58%, n=134, N=230 in 2016). In both surveys, the vast majority of responding CRAs were familiar with the term RBM (74%, n=134, N=180 in 2014; 79%, n=181, N=229 in 2016) and about 40% of the respondents had already monitored studies using RBM (hereafter labeled as “RBM-experienced” CRAs). The percentage of RBM-experienced CRAs increased slightly from 36% in 2014 to 41% in 2016 (n=65, N=180 in 2014; n=94, N=231 in 2016). However, the 2016 survey revealed that 37% (n=34) of the RBM-experienced CRAs had monitored only 1 study with RBM so far, 55% (n=51) had monitored two - five studies, and only 9% (n=8) had monitored six or more studies using RBM (N=93; data not shown; question in the 2016 survey only). Among all respondents only the CRAs with RBM experience were asked to answer the RBM-specific survey questions concerning working efficiency and site contact as well as data quality and patient safety. Working efficiency and site contact There was no clear opinion whether RBM facilitates the work of a CRA, neither in 2014 nor in 2016, but overall only half of the CRAs experience the RBM model as a supportive tool. In 2014, 57% (n=35) of the RBM-experienced CRAs (N=61) stated that RBM facilitates their work, whereas in 2016 46% of CRAs (n=42, N=91) assessed RBM to ease a CRA’s activities. The following citations are representative replies from the 2016 survey that illustrate the CRAs' perception: “Despite RBM, complete patient source data have to be reviewed to have the overview and see the critical points (hidden SAEs etc.)” “If SAEs occurred all data have to be verified; it is even hard to understand a context if not all data are reviewed! It is more time consuming to check what is allowed to be verified rather than verifying all data of the subject!” Moreover, survey answers of both years indicate that cooperation with sites has become more difficult with the implementation of RBM. In 2014, more than half (55%, n=34) of the RBM-experienced CRAs (N=62) reported that RBM makes the communication and cooperation with sites “more difficult”. The 2016 survey confirmed this result, with 48% (n=39, N=82) stating that working with sites in RBM studies is “more difficult”. Only 13% of RBM-experienced CRAs (2014: n=8; 2016: n=11) say it is “easier” with RBM and about one third think that RBM has “no impact” on communication and cooperation with the sites (32%, n=20 in 2014; 39%, n=32 in 2016). Data quality and patient safety Figure 1 shows persisting skepticism regarding data quality in RBM studies-even slightly higher in the 2016 survey. In 2014, the majority (73%, n=43) of RBM-experienced CRAs (N=59) did not believe that RBM improves data quality. This number in fact increased in 2016, with 81% (n=66) of RBM-experienced CRAs (N=82) doubting that RBM improves data quality. Still, there is concern that (S)AEs and protocol deviations might be overlooked when applying RBM. In 2014, about 59% (n=37, N=63) of the RBM-experienced CRAs argued that (S)AEs might be overlooked with reduced source data verification (SDV) in comparison to traditional on-site SDV and this number was even higher in 2016 (64%, n=54, N=85). Furthermore, about 60% of respondents of both surveys are concerned that important protocol violations can easily be overlooked when applying RBM (61%, n=39, N=64 in 2014; 58%, n=48, N=83 in 2016). Figure 1: Data Quality and Patient Safety with RBM Figure 2 illustrates the skepticism of CRAs regarding the quality of site performance in RBM studies. In 2014, 61% of CRAs (n=105, N=173) assumed that only up to 20% of the sites would deliver good study data quality without any on-site monitoring and the 2016 survey revealed the same picture (62%, n=121, N=195). The following selected citations of individual survey responses are representative for the CRAs' opinion: • “The Investigators are motivated by the CRAs presence at the site […].” • “RBM needs very well trained site personnel which need less control by the monitor.” Figure 2: Site Performance with RBM CONCLUSION The results of the 2016 survey indicate that CRAs' experience and satisfaction with RBM have not changed much over the last two years. In 2016, there is still uncertainty and limited practical experience with RBM among the responding CRAs. The survey performed in 2016 reveals a persisting preference to traditional monitoring and skepticism about the ability to maintain high data quality with RBM. Regular contact to the sites including on-site visits is rated as very important, especially with regard to the motivation of site staff. For many respondents, quality of data is still linked to the extent of monitoring and SDV. The focus and the way clinical monitoring is performed as well as responsibilities of the CRA will change with new RBM technologies and regulatory requirements. However, CRAs will remain the most important “connector” between sites and sponsor. Therefore, the CRAs will play a key role in successful implementation of the new monitoring processes. RBM will only be a promising approach and gain more acceptance among CRAs if we do not impose the new procedures to the CRAs, but listen to them and include their input. Moreover, the challenges emerging from new RBM-specific tasks demand thorough training, both for CRAs and site staff. Despite the potential which centralized monitoring, analytical data or trend analyses offer for a risk-based quality approach, the human factor and the CRAs' perspective and experience remain invaluable. ACKNOWLEDGMENTS We would like to give special thanks to the participating CRAs and to the BVMA member CRO proinnovera GmbH, especially Nicole Baeumer, PhD, and Kathrin Weber, PhD, who contributed significantly to the creation of this manuscript. REFERENCES 1. Litterscheid, S., et al. "RBM Experiences Among European CRA," Appliedclinical trials, Ebook 4th Ed., February 2016, 8-11. KEYWORDS Risk-based monitoring, RBM, centralized monitoring, CRA, survey, acceptance, data quality, site performance AUTHORS Christine Künzl, PhD, is Head of Medical Writing, CRO proinnovera GmbH Germany; Yvonne Rollinger, PhD, is Managing Director, OmniComm Europe GmbH; Michael Sigmund, DVM, is Chief Executive Officer, SSS International Clinical Research GmbH; Veronika Kunert is Chief Operating Officer, CRO proinnovera GmbH Germany; Burkhard Breuer, PhD, is Chief Executive Officer, CRO proinnovera GmbH Germany