Michael Herschel, MD

Clinical project managers, monitors and assistants in the pharmaceutical industry involved in conducting clinical trials are experiencing increasing regulation. This seems to be more of a hindrance than a help in their daily work. As well as being expected to be aware of the manifold legal requirements and international regulations such as the German Drug Law (AMG), GCP Regulation, ICH GCP, various European directives, and FDA guidelines to boot, in the case of global trials, employees are also confronted with an array of sponsor-generated quality management documents which frequently seem to consist of a tangled web of rules and regulations. The result is employee dissatisfaction and demands for improvement of the content, a reduction in the number of SOPs, and improvement in the way the content is presented. This situation is due in no small measure to the fact that there has been little or no debate to date on what constitutes "adequate quality" of an SOP system. 

The literature has comparatively little to say about what an optimal SOP system can and should look like. Most authors limit themselves to very basic suggestions (e.g., short, meaningful sentences, clarity, use of illustrations)

  or alternatively very detailed instructions,

  which are ignoring basic elements of instructional design. GCP requires "the setting up and maintenance of quality control systems supported by written SOPs,

  but provides virtually no guidance on SOP system design.  

Only a few articles looked into the complex regulatory environment of the pharmaceutical industry

  and the resulting principles of writing effective SOPs.

The German Association of Research-Based Pharmaceutical Manufacturers—Verband Forschender Arzneimit-telhersteller (VFA)—appointed a taskforce from the Clinical Research and Quality Assurance subcommittee to look into the topic. The VFA is an organization of 45 research-based pharmaceutical companies whose studies, between them, make up a significant portion of clinical trial activity in Germany. Responses in this survey were obtained from several companies who belong to the leading pharmaceutical companies in Europe. The aim of the SOP Benchmarking project was to evaluate existing SOP systems on the basis of existing best practices and benchmarking, and to develop proposals for potential improvements. Existing SOP systems including the associated training were to be analyzed and improvement proposals were to be designed as appropriate on that basis. Another important issue was to determine whether specific SOP concepts were associated with higher "satisfaction" on the part of the parties concerned in their respective functions. The survey was conducted in late autumn of 2009 to determine the current state of affairs among VFA member companies, identify the mood/opinion of their employees with respect to the current situation, and present proposals as to what sponsors could do to change their SOP systems or remedy any deficiencies and inadequacies. 

The questions related only to SOP systems used to conduct Phase I to Phase IV clinical trials and did not cover health outcome studies or non-interventional studies.  

Two questionnaires were designed for the survey: Questionnaire 1 (Table 1) about the cost and benefit of SOPs looked at the existing structure and size/scope of the company's existing SOP system in terms of GCP, SOP management, and presentation of SOP content. The addressees were management and organizational employees (e.g., medical directors, directors of clinical investigation, and quality assurance directors).  

Questionnaire 2 was addressed at clinical research employees (i.e., monitors and project managers of the sponsor and freelancers/employees of CROs involved/in sourced by the sponsor for study projects). Questions were asked about handling and satisfaction with the existing SOP system/training methods, and respondents were asked to make proposals for possible future systems and methods. 

Replies from the two surveys were encrypted by VFA via company code so that it was possible to match management responses to employee responses, but without revealing the identity of individual companies and persons in the evaluation.  

A total of 18 companies took part in the survey, the majority of which is represented by a member in the already mentioned subcommittee. The results hence reflect relevant experience of SOP systems in the pharmaceutical industry. In nine cases the company was the local affiliate of a global corporation. Three to four responses per company were received from the employees invited to take part in the survey in the operative line departments (67 total); the majority were company project managers and/or clinical monitors with an average of 10 years of job experience.  

Estimation/user satisfaction with the currently established SOP system. 

Irrespective of their function or role in clinical research and time with the company, 19% of respondents were fully, 57% partly, 16% less, and 8% not at all satisfied with the current SOP system in their company. The main criticisms of existing SOP systems concerned the complexity and lack of clarity of individual documents/SOP systems, which made it more difficult for users to rapidly seek and find the relevant sections/instructions required for day-to-day work or in a specific on-site situation. In many cases, instructions concerning parts of processes (e.g., study initiation) are spread out among different sections of an SOP or even among a number of different documents (SOPs, instructions, appendices, etc.).  

SOPs with large numbers of pages did not necessarily receive poorer ratings. Instead, SOPs were criticized for being unclear and/or overloaded with abbreviations and jargon. Documents intended to be globally applicable were frequently mentioned in this context; respondents said global documents tend not to contain clear information/delimitations for each area (global/local functions). The associated need for additional material on local processes was seen to result in unreasonable multiplication of process descriptions. Users saw clear potential for improvement in implementing a level of document detail suited to the specific tasks and responsibilities, combined with a rapid and accurate search feature. Almost half of the respondents reported little or no personal involvement in the creation of SOPs. A mere 15% of those involved in document compilation said they were actual authors; most were involved only in the review/feedback process. Involvement in SOP compilation did however not correlate with level of satisfaction. 

Only 27% of respondents were happy with system size/scope and less than half (48%) were happy with the level of detail. Detailed evaluation showed that monitors tended to be more satisfied with lengthy SOPs than project managers. The same applied for satisfaction in relation to time spent working in the company. The SOP system was increasingly felt to be too lengthy with increasing time spent working in study management. In contrast to size/scope, there were no major differences in satisfaction between monitors and project managers in respect of level of detail. Here again, however, there was a trend toward dissatisfaction with overly detailed descriptions as a function of time spent working for the company (> 10 years of job experience).  

Twenty five percent of respondents said existing SOPs were not coherent and comprehensive. This opinion was more commonly expressed by project managers than by monitors. There was no correlation with time spent with the company. Thirty five percent of respondents said the SOPs were not clearly structured, monitors more so than project managers. Systems with fewer pages (in reference to monitoring) also tended to be rated more favorably.  

SOPs were primarily found to be useful (approximately 85%), irrespective of duration of job experience, even more so by monitors than by project managers.  

Are SOPs more restrictive than the legislative framework? 

About two-thirds of respondents said internal company SOP systems were more restrictive than applicable law. Project managers tended to find them even more restrictive than monitors.  

There was a trend for a negative rating of all the sub-elements of scope/size, level of detail, comprehensibility, clarity, usefulness, and restrictiveness from employees who were dissatisfied with the SOP system overall. 

Existing SOP systems in clinical research. 

The global and/or local SOP systems for conducting trials that exist alongside legislative requirements differ greatly between companies. The survey shows that there is neither a uniform hierarchy/structure in respect to SOP documents, nor in terms of nomenclature of document types. To name just a few, the terms employed include SOP, policy, operating instruction, and working instruction. There is also a plethora of different annexes such as flow charts, tables, job aids, completion guides, etc. The reported estimated length of SOP systems ranged from a few pages to complex SOP constructions with an estimated number of up to 8,500 pages. 

Comparison of job areas and processes with a largely pre-dictated scope of regulation (e.g., monitoring, SAE reporting, audit conduct, and collaboration with third parties/CROs) gives a similar picture. Again, there are huge differences between SOP systems. For monitoring (site selection/pre study visits, study conduct, study completion, and close out visit), the scale ranged from one to 35 SOPs (Figure 1) and from 10 to 700 pages. The mean number of SOPs a study manager/project manager in clinical research had to be familiar with was approximately 50 SOPs/480 pages.  

Increase and overregulation of processes—potential for reductions. 

Whether and how subjective dissatisfaction in recent years correlates with SOP proliferation is evident from the following statements from management staff. About 40% said there had been no change regarding the estimated number of global or local SOPs. Thirty percent had an increase of less than a 10%; 30% of more than 10% increase per year. None had a reduction in the past 12 months. Respondents said processes were overregulated (not including those relating to clinical trials but not originating in clinical research) in about 5% (median) of all processes described.  

Few ways of streamlining processes or eliminating SOPs showed up. Half of the companies' management staff surveyed saw little or no potential for reductions (two SOPs on average). The vast majority was unable to identify SOPs that could be done without. In contrast, clinical operations staff felt that a median of 10% of SOPs they have to comply with in their day-to-day clinical trial performance were superfluous.  

Conversely, gap analysis revealed little or no need to extend the SOP system. Only two companies reported a specific need for regulation in particular areas. 

According to the respondents, corporate headquarters are responsible for SOP authorship in 60% of the surveyed companies. More than 70% of the surveyed companies involve the line departments and about 50% involve quality assurance in SOP compilation. One-quarter of the companies involve SOP writer specialists in drawing up SOPs. Affiliates and subsidiaries are involved in the generation of global SOPs in addition to corporate headquarters. Sixty five percent said they were included in discussions about new SOPs, and their feedback is adopted/integrated in documents in more than 70% of cases. The expertise of SOP authors was largely rated favorably; only 10% of respondents said SOP authors tended to be far removed from reality. 

In about 70% of companies, additional local processes and SOPs are generated to supplement the corporate regulations in order to reflect and implement local/regional needs and legislative requirements.  

Three-quarters of the surveyed monitors and project managers said they sometimes had to deviate from an SOP in their day-to-day work. The main reasons cited for necessary deviations were "unfeasibility," as well as (lack of) time, effort, and cost. About 45% of employees said they never deviated from an SOP. Almost the same number (about 42%) said they deviated one to two times per year, and only a few said they had to deviate more than three times or five times per year.  

Deviations resulting from a discrepancy between global SOPs and local legislation are handled in different ways. About 50% of companies draw up local modifications. About 30% put national laws ahead of company instructions and implement this policy without any formal modification/amendment of the global SOP. 

Respondents said one reason for violations and dissatisfaction among those working with SOP systems was the compilation process itself or the generation period/revision cycle. About 40% of management employees said that new or revised SOPs were often implemented unexpectedly and without "prior warning" by corporate headquarters. The SOP generation period from first draft through feedback periods to approval and release lasts 10 weeks on average. Lead times from announcement to enforcement of an SOP were four weeks on average. 

In addition to regular review of SOPs at intervals of two to three years, there are other important reasons for revision. In more than 80% of reported cases, a change in legislation was the reason for an SOP revision or creation of a new SOP. A similar situation applied (almost 90%) with respect to structural changes within companies that entailed adaptation of processes and responsibilities.  

Two-thirds of the companies surveyed said learning SOP content was primarily the job of the individuals concerned. Nonetheless, formal learning was deemed the best method of learning (80% of companies that said this was mostly or occasionally the case). Two-thirds of companies rejected the prospect of ad-hoc training provided only to meet an acute need and if a particular process was specifically required. 

Seventy five percent of employees also said self-study was one of the most important ways of learning SOPs. Even more respondents (88%), however, said face-to-face training was the best learning method. Less than half of respondents (43%) said e-learning was the best method. However, acceptance of this method increased if it was linked with review of learning outcomes (58%). 

Despite significant variation among the surveyed clinical research management, self-study accounted for the major part of training effort, accounting for a median of 70% of total time required, followed by e-learning (median of 40%) and face-to-face/classroom training (median of 15%). Training with formal review of learning outcomes accounted only for a median of 7.5% of training events.  

Regardless of training events, the surveyed companies said the SOP competency of employees is not reviewed systematically. One-third do not regularly check competency, 15% only in isolated cases, if the manager notices deficits or an audit identifies any deficiency. Fifteen percent of companies check their employees' knowledge of the main SOPs at least once a year. Training units are repeated every one to two years in 56% of companies, and at intervals longer than two years in another third. 

The open-end answers clearly show that learning modalities cannot be viewed in isolation from the complexity or novelty of content, from the overall structure of the training system, or from the individual employee's level of experience: "The best setting to choose depends greatly on the content of the SOPs." Respondents also explicitly suggested combination with other learning modalities: self-study and online training for induction, face-to-face training to deepen understanding and for discussion. 

An average of 11 days per year is devoted to training, but this figure varied significantly among the respondents (Figure 2). The three-peak distribution shows peaks at five, 10, and 20 days. Monitors and project managers were asked which quantity of training they would consider to be the most appropriate. The answer was approximately nine days, again with significant variation. The breakdown here again shows systematic differences between the relevant user groups.  

In respect of cooperation with CROs, it emerged that sponsor SOPs were used as a basis for study conduct in approximately two-thirds of cases. CRO SOPs were used in less than one-third of cases. 

The baseline hypothesis (i.e., that SOP systems in GCP as they now exist would be rated inadequate by corporations and users) is not strongly supported. Although analysis of survey outcomes reveals deficiencies in the system and a need for improvement and change in some areas, the benefit of SOPs is nonetheless accepted. Nor did the survey results confirm the often cited suspected widespread overregulation of SOP systems.  

Not every SOP system is the same, and not every SOP is equally suitable for every organization. What works for one company need not be the best solution for another.  

Authors of SOPs tread a thin line in general, because user expectations and needs are as diverse as the spectrum of SOPs in existence. The needs of career starters differ from those of employees who have worked in the company or in clinical research for longer. Differences are also apparent in terms of the roles and responsibilities of respondents, for instance between monitors and project managers. Globally operating companies face the challenge of implementing global policies into local practice in a way that accommodates local legal requirements and best practice but at the same time does not violate global SOPs. 

The various aspects are looked at in greater depth in the following: 

SOPs are described overall as too lengthy. The subjective feeling of many respondents seems to be that "less is more." 

Many respondents cited the difficulty in finding individual process steps within an SOP system as being a key problem. "Details of a single process need to be searched for in multiple SOPs." "Search features in the electronic SOP repository are ineffective." "Sometimes it is not easy to find the right SOP you need for a particular process." 

The size/scope of SOP systems evidently differs greatly between companies indicating that SOP systems can display significant differences and may not always be directly comparable. The total number of SOPs and in particular the estimated number of pages—8,500 for one company—showed a wide range of variation. The only explanation is that SOPs may be written as no-frills "instructions" (who does what and how), as well as guidance (e.g., every single document, be it a simple letter to an investigating site, trial protocol template, or CRO communication plan). The latter policy may lead to a situation where important information and instructions for employees are hidden in a multiplicity of annexes/working instructions/templates to SOPs, which employees have to read and wade through even though they may not be immediately relevant to the specific (sub) task in hand. The former bears the risk of heterogeneous implementations. 

As already mentioned at the outset, existing regulations do not dictate the depth of regulation in SOPs. ICH GCP section 5.1.1 merely states that sponsors are obliged to implement an effective quality management system with written SOPs. Thus, the decision on depth of regulation is a task of the company.  

Articles

Survey appraises the use of SOPs in clinical research from the sponsors' point of view.

Michael Herschel, MD

Articles

SOPs in Clinical Research