Evelyn Jäger, PhD

 unanimously request effective and efficient quality management systems (QMS). However, 

 demonstrate the need for continuous improvement.

	Therefore, development of a strategic organizational structure focusing on the subject of quality management has become a top priority in many pharmaceutical companies. Since clinical quality assurance (CQA) plays an integral role within the QMS, this makes it all the more important that up-to-date benchmarking information on CQA systems is evaluated.

 provided an overview of CQA best practices within the pharmaceutical industry 10 years ago. A recently conducted comprehensive literature review did not reveal relevant updates on the subject. Therefore, a follow-up benchmarking survey was conducted to assess current practices. The survey among global pharmaceutical companies of different sizes focused on the organization, resources, strategies, and activities of CQA.

	In this article, we critically review the changes over the past 10 years and provide recommendations on how to shape the future.

	Benchmarking information regarding organizational structure, capacities, and strategies is difficult to obtain in a highly competitive environment. In 2004, we managed to receive information from 19 global companies. In our current survey, we received feedback from 12 global pharma companies, which also reflects mergers that took place in the past 10 years.

	Organizational structures of the participating companies differ substantially. Compared to the results in 2004, there is a shift of CQA reporting to a higher management level. Also, the current results are showing no instances where CQA reports into the clinical development team. This finding perfectly reflects the necessity of independent CQA units in the spirit of 

 (refer to section 2.13, 5.1, and 5.19). This is also in alignment with guidelines such as 

Guideline on Good Pharmacovigilance Practices (GVP) – Module IV

 where a harmonized and independent “quality unit” is requested.

	Ten years ago, the results showed a clear relationship between R&D budget and CQA capacity. This reflected the obvious relationship between R&D activities and the required quality management (QM) support. We did not find this relationship replicated in the present results. Whereas, in 2004 most elements of QM support were provided by centralized CQA, today this is much more heterogeneous. In some companies, quality and compliance functions are implemented in a decentralized model within business units with less increase of CQA staff. In addition, the much higher level of outsourcing of clinical research activities, either including QM support (fully outsourced models) or keeping QM support in-house, contribute to the heterogeneity seen in the current survey.

	In 2004, CQA capacity was spent relatively homogenously; besides the core activity of auditing (~70%) consultancy, SOPs, and training accounted for about 5% to 10% each. Today, auditing, including audit conduct and management of contracted audits, still reflects the highest share of CQA capacity, but has decreased to about 50%. Whereas, capacity spent for consultancy and training remained the same, it increased from about 5% to 10% for SOP management.

	Today, CQA departments take over a broader variety of tasks. In particular, inspection management; non-compliance investigations, including root cause analysis; and involvement in process improvement projects are now typical activities of CQA and account for 5% to 10% each. This matches the decrease of capacity spent for auditing. Another new activity is vendor oversight other than audits (e.g., vendor selection) by CQA, which is performed by more than half of the pharmaceutical companies surveyed.

Figure 1: Distribution of CQA activities in 2014 displayed by minimum and maximum values, upper and lower quartile, and median (%). Count [n] reflects the number of companies (total n =12) performing a certain activity.

	The CQA of one company performs quality control for other departments. In the view of the authors, this is not in line with the independence of CQA and furthermore undermines the accountability of the individual departments in regards to Quality Control steps.

	Outsourcing of certain tasks is very common in pharmaceutical companies. The number of organizations outsourcing CQA activities increased from 63% in 2004 to 75% in 2014.

	Investigator site audits are outsourced most frequently. The in-house capacity necessary for handling and supervising outsourced audits is still about 30%. The outsourced audit activities are budgeted for by CQA in about 75% of the companies. All this did not change in the past 10 years.

	Although outsourcing of vendor and system audits increased, it is still a lesscommon practice. Because vendor audits are primarily performed by internal auditors, this helps assure the company’s own quality standards are fostered and knowledge about the vendors is maintained within the company.

	The majority of companies outsource audits either to specialized audit CROs or individual independent auditors. In this benchmarking survey, 25% of the companies stated that they outsource audit activities to the CRO that is also in charge of conducting/monitoring the respective trial. This increased from about 5% in 2004 and might be a consequence of full package outsourcing. Due to potential conflict of interest, the authors strongly recommend to separate out CQA outsourcing that better reflects ICH GCP 5.19.2: “The sponsor should appoint individuals who are independent of the clinical trials/systems to conduct audits.”

	In the current survey, we also explored outsourcing of activities other than auditing. Except for SOP management and vendor oversight, all other activities are outsourced to some extent, most prominently internal training and training/education of CQA staff. This provides the advantage of getting input from third parties (e.g., external trainers).

	In contrast, outsourcing of SOP management and vendor oversight is not commonly done, as it requires that external consultants are contracted on a long-term basis. This is conducted as such so the consultants can acquire sufficient knowledge of the company’s processes and responsibilities in order to provide beneficial support.

	Today, nearly all companies (with one exception) establish an annual audit program. Out of these, the great majority apply a risk-based approach, most of them using a formal scoring system for risk factors.

	A variety of risk factors are taken into consideration for establishing the audit program and the most common risk factors named are as follows:

				Trial type/importance (pivotal, dose-finding, proof-of-concept, supporting safety, bio-equivalence, post-approval, etc.)

				Special populations /vulnerable subjects

				Novelty of design, indication, or technology

				Registration intent (e.g., first registration [NCE] vs. line extension vs. publication)

				Previous compliance history (monitoring / site management)

				Product portfolio (importance/properties of projects)

				Workload/volume of activities (e.g., number of PV cases or Phase I trials)

				Prior compliance record (audit and inspection history)

	Based on the audit strategy, audit coverage for trials, systems, and vendors (CROs) is determined. While risk factors already played a role in 2004, today the use of a risk-based strategy is much more pronounced and formalized with explicitly defined criteria, scoring rules, and associated tools.

	Regarding trial audits, about one-third of the companies focus their audit activity on pivotal trials; about one-third state they have all trials from Phase I to IV in scope; the remaining companies determine audit coverage according to the risk level assigned to the trials. One company does not target trials, instead focusing on systems and, therefore, certain studies within the scope of their system audits. Compared to 2004, the sample of audited sites per trial selected for audits has decreased: typically, it now lies in the range of 5% to 10%, whereas 10 years ago, samples were in the range of 10% to 20%.

	Regarding document audits, about half of the companies do not perform them at all; the other half select clinically important documents (protocol, IB, TMF, trial report, submission dossier)-mostly on a sample basis. There is a clear decrease in selecting documents for audits in comparison to the practices in 2004.

	For system and vendor audits, the approach is relatively homogeneous across companies. Risk levels assigned to systems/vendors guide the selection for audit and also determine the audit cycle times, which range from two to five years.

	The following is a typical response regarding system audits: 

“Selected based on risk. Various risk rating tools employed. Cycle time determined by risk, typically every three to five years.”

	Another response regarding vendor audits: 

“Volume and importance of work. Time since last audit with identified issues. Preferred providers are audited every two years.”

	In contrast to 2004, risk-based approaches are now widely applied to trial, vendor, and system audits.

	The number of trial, system, and vendor audits performed per year and audit type varies considerably across companies (e.g., for investigator site audits the number varies from nine to 600). The number of investigator site audits is nearly perfectly correlated to the number of trials (r=0.99).

	The majority of audits still focus on trials (investigator sites and trial documents) but their proportion decreased in favor of system and vendor audits. This clear trend is in line with the reduction of sample size for investigator site audits reported above.

Figure 2: Distribution of trial, vendor and system audits compared 2004 (n = 3310, total audits performed across 19 companies) and 2014 (n = 2592 total audits performed in 12 companies)

	The shift from trial to vendor and system audits surely reflects the increase of outsourcing and the use of additional technologies. In our view, however, the main driver of this change is the paradigm shift towards a holistic QM system with focus on processes and systems. This interpretation is also supported by the regulatory agencies’ views (e.g., the 

Report on the Pilot EMA-FDA GCP Initiative

’s conclusion: “To focus the joint inspections on sponsors and CROs instead of investigator sites in order to work towards developing a truly harmonized quality-systems approach to sponsor/CRO inspections.”

	The trend seen is very likely to continue to achieve a healthy balance between focusing the audit strategy on processes and systems supported by trial audits (

ISO9001:2008 – Guidance in the concept and use of the process approach for management systems

	In addition to audits, we asked for the number of pre-inspection visits performed per year. Again, this varies considerably across companies (range 0 and 20 visits).

	Resource assumptions for the different audit types have been collected and analyzed. In general, more capacity is spent on system and vendor than on trial audits. This is also reflected in the median number of man-days spent for an audit:

	Compared to 2004, the capacity spent for an investigator site audit increased by about 50% (2004: 8 man-days, 2014: 12 man-days).

	In summary, compared to practices in place 10 years ago:

		Risk-based approaches are currently more formalized and applied to all types of audits

		More capacity is allocated to individual audits, but fewer sites tend to be audited per trial

		Fewer document audits tend to be performed; it is rather the exception than the rule

		Finally, the proportion of system and vendor audits clearly increased reflecting a possible paradigm shift in the audit approach

	The process of audit reporting is quite standardized amongst companies and did not significantly change compared to 2004:

		Evaluation of significance of audit findings (e.g., critical, major, minor/other)

		Content-related categorization (e.g., study medication, monitoring, etc.)

		QC-steps for reporting of audit findings (e.g., peer review of audit reports)

Articles

A follow-up survey on clinical quality assurance (CQA) assesses current industry best practices compared to a decade ago, and provides recommendations on this key component of quality management for the future.

Evelyn Jäger, PhD

Articles

Critical Appraisal of Clinical Quality Assurance Based on Benchmarking Results: Past - Present - Future