Workload Measurement Instrument

It is increasingly evident to all parties involved in the sponsorship and management of clinical trials that a better understanding of the time, staff, and financial resources required to conduct clinical trials is necessary. The increasing use and complexity of multi-modality treatment regimes, the rising costs of clinical trials, the emphasis on the efficient use of available resources and adherence to Good Clinical Practice (GCP), and increasing regulatory requirements and demands for quality assurance/control, have resulted in an increased focus on workload issues. Failure to address these issues has resulted in research staff using unproven methods or a simple estimation for determining workload, leading, in some cases, to unrealistic expectations, excessive workload, or inefficient use of resources.

CHAD BAKER/GETTY IMAGES

The EORTC Clinical Research Coordinators Group recognized the importance of addressing this problem based on understanding the tasks involved, the time requirements, and the resources necessary to effectively and efficiently conduct clinical research. The development of a workload measurement instrument (WMI) was seen as a means of providing a tool by which individuals could estimate more accurately the time and resources required to participate in clinical trials. The initial project was supported by the EORTC and funded by a grant from Fonds Cancer (FOCA).

The WMI was developed in seven stages and the first stages: development, validation and revision of a draft checklist of trial related activities, drafting of the WMI, a feasibility pilot study, and analysis and revision of the trial related tasks incorporated into the four modules previously outlined in an Applied Clinical Trials article that appeared in June 2008, titled "Workload Measurement."¹ Coincidental to the development of the WMI, Jacqueline Briggs had developed a complexity tool for assessing the complexity of a study prospectively.² It was agreed that, in the longer term, a joint workload and complexity project would aim to independently validate and link together the WMI and the complexity tools.

This article describes the seventh stage of development of the WMI; a prospective study using the WMI to collect data for the purpose of validating the four WMI modules.

Materials and methods

The preparatory phase identified the specific actions necessary and the financial and staff resources required to complete the prospective study. The Steering Committee members presented the prospective study outline to a meeting of National Cancer Research Network Managers and identified the networks that would be willing to collaborate. Expression of interest questionnaires were sent out to all UK network managers and to research staff in several European centers to identify research staff and potential trials for inclusion in the prospective study. Fifteen UK networks and a center in Ireland initially agreed to participate in the study and, of these, 12 networks returned completed modules as part of the prospective study.

The prospective study focused only on research related activity and not on what would be considered routine care. Workload was only recorded for research staff working on the trials and excluded investigators, pharmacists, day care staff, etc. These staff members were excluded as it was felt that it would be difficult to collect accurate workload data from such a wide range of staff and this was accepted as a limitation for the purposes of this study.

As the joint project had been extended to include complexity the four WMI modules (Module 1: planning stage; Module 2: implementation stage; Module 3: trial data management stage; Module 4: closure/final stage) were reviewed. In order to facilitate the validation of the complexity tool and create the necessary links between the two tools, it was agreed to divide Module 2 into two modules: 2a recruitment and treatment and 2b treatment and follow-up.

A data collection period of six months was agreed and the time frame for completion of data collection was October 2005 to May 2006. Prior to the commencement of the data collection period, three training days were held in Newcastle, London, Nottingham, and Wales to roll out the study to the interested participants. Forty-three research staff attended these training days. Information was provided individually for staff unable to attend the training days.

From the training days and returned questionnaires, a number of trials were selected on which the prospective study would focus. For modules 1 and 4 it was agreed that any trial in set up or at the closure stage could be included due to the small number of trials likely to be at these phases during the six month time frame. For modules 2 and 3, trials covering as many tumor types, treatment modalities, and study types were selected to ensure as inclusive a range of activities as possible. A total of 36 studies were identified and included in the prospective study.

Guidelines for the completion of each module were developed and circulated, together with the four modules and the list of identified trials, to 70 collaborators, covering 15 networks that had agreed to participate. Collaborators were asked to record the time taken, in minutes, for each individual task or subtask undertaken as identified within the modules. A subsection—other—was included for recording any additional activities not identified within the modules. A separate section recorded the hospital name, trial name and number, the date for data collection commencement and completion, the reason why data collection was ended, and whether more than one form had been completed. This allowed us to identify the total period of data collection and subsequent analysis of workload for a single trial visit or over the course of a trial period.

Data was entered into a database developed in conjunction with the NCRN IT department based on all the identified tasks and subtasks of the WMI modules. The NCRN agreed to provide staff in order to complete the data input and carry out some preliminary data analysis and a system for quality assurance of the data was agreed upon.

The Steering Group met regularly to review the progress of the prospective study, and as returned modules were received, to carry out the initial quality assurance procedures. All returned modules were double-checked to ensure accuracy, and when tasks or subtasks were entered in an incorrect section these were noted and revised prior to entry onto the database. Incorrect or unclear entries were deleted. Regular contact and feedback was maintained with the collaborators throughout the study by e-mail, newsletters including answers to frequently asked questions, and presentations at Cancer Research Network Managers' meetings.

Results

A total of 414 completed modules were returned from collaborators from 27 hospitals covering 12 UK Cancer Research Networks. Three networks or centers did not return completed modules due to merger of networks, shortage of staff, or excessive workload issues. Data was returned on 35 of the 36 studies identified for inclusion in the prospective study. Table 1 shows the modules returned.

Table 1. A total of 414 completed modules were returned from collaborators from 27 hospitals covering 12 UK Cancer Research Networks.

As part of the analysis process a series of assumptions were made. These included: research staff were experienced and trained and not involved in the actual delivery of treatment, research activity was additional to standard treatment, and hospital support services and standard equipment were available and accessible to research staff.

Cumulatively, 130 changes were made in the 74 subtasks (Table 2). These changes included transfer from one subtask to another where an entry was incorrect, deleting investigators, and separating out multiple patient visits into individual visits. The highest numbers of recording errors (9, 15, and 16) necessitating transfer related to the recording of administration activities in other subtasks rather than administration activities.

Table 2. Cumulatively, 130 changes were made.

Following extensive quality assurance procedures all tasks and subtasks within Modules 1, 2, and 3 were validated as relevant and appropriate with no tasks or subtasks removed. All subtasks were recorded at least once with a very wide range within each module. For Module 1, the most frequently completed subtasks were those relating to the preparation and submission of documentation to Ethics and Research and Development departments. Conversely the lowest recorded tasks related to organizing and attending meetings other than in-house meetings for information, training, and trial activation.

For Module 2a, the most frequently completed subtasks related to screening for eligibility, informed consent, and sample preparation. This was consistent with the expectations and personal experience of the project group. The least completed subtasks in Module 2a related to liaising and verifying radiotherapy dose reductions and missed treatment which was anticipated given the very low numbers of radiotherapy related trials included in the study.

For Module 2b and 3, the most frequently completed subtasks related to case report form completion, photocopying and sending trial documents, the preparation and submission of trial amendments, and updating study documents following amendments (Table 3).

Table 3. For Module 2b and 3, the most frequently completed subtasks related to case report form completion, photocopying and sending trial documents, and the preparation and submission of trial amendments.

Two major revisions were identified as necessary. The first change related to the recognition of the importance given to administration and communication by the collaborators. The section "other" consistently included additional administration and communication and it was agreed that these two activities should be included as subtasks for all main tasks in the final modules. In total, 110 transfers were made to administration main tasks and 230 to administration subtask activities; 38 to communications main task, with 100 to communication subtask activities. The second change was the recombination of Module 2a and 2b with specific sections relating to consent, treatment, and follow-up incorporated—confusion had arisen in completion of these as two separate modules.

Recommendations

Modules 1, 2, and 3 of the workload measurement instrument have been validated and are now complete. These modules are available in their entirety online in PDF format at appliedclinicaltrialsonline.com as part of the online version of this article. Module 4 has been revised in accordance but would benefit from validation as part of a specific prospective study as trials reach closure stage.

It is acknowledged that there were limitations to the prospective study that included the short data collection window of six months, the exclusion of principal investigators and other associated staff from the data collection exercise, compounded by the difficulty associated with specific local hospital issues all had potential to confound the data collected. It was accepted that these factors would contribute to workload but would not necessarily alter the tasks and subtasks identified for inclusion in the modules and, therefore, such confounders were not considered significant for the purpose of the WMI validation. For hospitals or departments undertaking a workload analysis in the future it would be appropriate to take these confounders into account.

This WMI was developed to be a generic tool and can be used locally, nationally, or internationally to review the trial workload of an individual researcher, team, hospital, or network of centers. It should inform discussions at all levels and allow centers considering implementing new or additional clinical trials to accurately evaluate the associated workload and to allocate staff and resources accordingly.

Mary Coffey* is Head of Discipline at Division of Radiation Therapy, School of Medicine, Trinity Centre for Health Science, St. James' Hospital, Dublin 8, Ireland, e-mail: mcoffey@tcd.ie. Julie Berridge is Clinical Trials Team Manager at Mid Trent Cancer Research Network, Nottingham City Hospital, United Kingdom. James Lyddiard is Head of Cancer Trials at University College London Hospitals, NHS Foundation Trust. Jacqueline Briggs is Stroke Research Network Manager at Peninsula Stroke Research Network, Royal Devon and Exeter Foundation NHS Trust, Exeter, United Kingdom.

Module Links

1. EORTC Module 1.

2. EORTC Module 2

3. EORTC Module 3

4. EORTC Module 4

References

1. Julie Berridge and Mary Coffey, "Workload Measurement," Applied Clinical Trials, June 2008, 98-101.

2. Jacqueline Briggs, "Real World Workload Needs: Developing a process and management tool for scoring complexity in cancer clinical trials," Applied Clinical Trials: Oncology & Clinical Trials in the 21st Century, supplement, May 2008, 22-24.