Yoong Kai Shen

The International Conference on Harmonization- Good Clinical Practice (ICH-GCP) is an international ethical and scientific quality standard that ensures the rights, safety, and well-being of clinical trial subjects are protected and that the clinical trial data generated are credible.

	Malaysia’s GCP guideline was developed by the National Pharmaceutical Regulatory Agency (NPRA) in 1999. This guideline adopts the basic principles outlined by ICH-GCP with some modifications to adapt to local conditions.2 The object of the Malaysia’s GCP guideline is to ensure that drug-related trials in the country are conducted in accordance with international ethical and scientific standards.

 Although there is currently no legislative requirement for GCP training, mechanisms were put in place to ensure that Investigators are trained on GCP. For example, it is mandatory to submit a copy of the principal investigator’s GCP certificate when obtaining regulatory approval from NPRA to conduct a clinical trial.

 In addition, all investigators must submit their GCP certificates for the ethics committee’s approval in Malaysia.

 Since the first edition of the Malaysian guidelines for GCP to the current fourth edition, approximately 12,000 clinicians have been GCP certified.

	It is recommended for GCP-certified personnel to be retrained on GCP guidelines every three years in order to stay current with updated regulations, standards, and guidelines.

 To that end, Clinical Research Malaysia (CRM), a research organization founded by the Malaysia Ministry of Health (MOH), has taken the initiative to conduct GCP refresher workshops for clinicians, nurses, allied health professionals, and study coordinators who have been certified with Malaysian GCP. The objective of the workshop was to provide a review on the principles of GCP, to improve participants’ skills and knowledge on managing the conduct of a clinical trial, and to share recommendations on how to conduct a GCP-compliant clinical study.

Since the first workshop on Oct. 4, 2016, CRM has conducted 31 GCP refresher workshops throughout Malaysia to about 450 participants at no cost. The majority of the participants are clinicians, nurses, and allied health professionals from government hospitals who are involved in clinical trials. Over the course of the conduct of the workshop, modifications have been made based on participants’ feedback to improve the quality and learning outcome of the workshop. A summary of the changes in the workshop agenda is listed in Table 1. As shown in the table, the pre- and post-workshop assessment test was introduced March 16, 2017, in order to evaluate the effectiveness of the workshop on participants’ knowledge regarding GCP. In this article, we analyze the pre- and post- workshop assessment score for workshops conducted since March 16, 2017.

	Before the start of the workshop, participants were required to complete 20 multiple-choice GCP-related questions (pre-workshop assessment). The questionnaires were then collected before the start of the workshop. When concluded, participants were then required to complete the same set of questions (post-workshop assessment). Participants were given 30 minutes to complete the questionnaire.

	The average score of the pre- and post- assessment results from participants of the same workshop was then calculated, which constitute one set of data points.

	The average score of pre- and post- workshop assessment from each workshop conducted since the introduction of the assessment on March 16, 2017, were compiled. Of the 28 workshops conducted since then, five workshops’ data could not be retrieved, hence, only 23 sets of data point were analyzed.

	The compiled average score of pre- and post- workshop assessment were then compared using Microsoft Excel’s T-test, assuming unequal variance to evaluate if there is any significant improvement in the average assessment score after the workshop.

Table 2 shows the compiled data from 23 workshops. Four hundred and twenty participants completed the pre-workshop assessment, and 434 participants completed the post-workshop assessment. On average, there is 20% improvement in the assessment score before and after the workshop.

	As shown in Table 3, the p-value is less than 0.05, suggesting that the improvement in the score is significant after participants attended the GCP refresher workshop.

	Training of investigators and allied health professions involved in clinical trials on the regulations and standards that govern clinical trials is important to improve the quality of studies and ensure maximum safety for study subjects. A survey among clinical researchers in Saudi Arabia on their ICH-GCP knowledge found that there is a poor understanding of investigators’ responsibilities on informed consent, and prompt reporting to the ethics committee.

 Hence, refreshing investigators’ awareness of their responsibilities could help to improve the performance of investigators in conducting clinical trials, particularly in these important ethical aspects.

	The GCP refresher workshop conducted by CRM fulfills its objective to provide revision of the principles of GCP to participants, with more than an average of 20% improvement in assessment score after the workshop.

	The factors that potentially contribute to the success of the CRM GCP refresher workshop in improving participant’s scores are explored ahead.

	Case studies were included for informed consent, adverse event reporting, and protocol deviations, for discussion.

	Participants were given real-life scenario case studies and they were required to answer guided questions in groups. The study has shown that case studies are more effective than textbook reading at promoting the learning of key concepts and comprehension of the relevance of concepts to real-life scenarios.

	Several methods were used to make the workshop interactive, such as games, quizzes, videos, group activities, and group discussion. Active engagement of participants helps to retain information.

	Trainers would also actively engage participants by asking questions and encouraging participants to share their experiences and practices. Regulations of clinical research can be understood more easily when participants can reflect the principles based upon their own experience.

	Participants were given an evaluation form at the end of the workshop to provide their feedback and opinion. Based on their feedback, modifications were made to improve the conduct of the workshop.

	The first GCP refresher workshop was a half-day (four-hour) workshop. Participants reflected that the workshop felt rushed and suggested to make it a full-day course. Subsequent workshops were then conducted one full day (eight hours) to ensure participants can learn at a better pace, contributing a better learning outcome.

	Based on early feedback, participants would like more activities, especially for safety reporting and protocol deviations. Modifications based on this feedback include the shift from a lecture-style workshop to an activity-based workshop. Presentation slides were also revised to be more concise to retain attention.

	Following feedback from participants, current edition Malaysian GCP guideline booklets were provided for participants to refer to during the workshop.

	It is important for investigators and allied health professionals who are involved in clinical studies to be trained on the latest GCP guidelines. GCP refresher workshops conducted by CRM were effective in refreshing participant knowledge on GCP guidelines. The success of the workshop could be attributed to providing case studies for discussion and interactive activities, as well as taking participant feedback into consideration. We would recommend other governing bodies to support the conduct of similar interactive workshops in their jurisdiction and to consider participant feedback for continuous improvement.

		Good Clinical Practice Network, n.d. ICH GCP Good Clinical Practice. Retrieved 05Nov2019 from: 

		National Pharmaceutical Regulatory Agency, 2018. Malaysian Guideline for Good Clinical Practice 4th Edition. Introduction to Malaysian Guidelines For GCP, Page 8.	

		National Pharmaceutical Regulatory Agency, 2017. Malaysian Guideline for Application of Clinical Trial Import Licence and Clinical Trial Exemption. Retrieved Nov. 5, 2019 from: 

http://www.nccr.gov.my/view_file.cfm?fileid=33

		Society Of Clinical Research Professionals Malaysia, 2016. A Guide to Conducting Clinical Trials in Malaysia, First Edition. Retrieved Nov. 5, 2019 from: 

http://www.clinicalresearch.my/wp-content/uploads/2017/03/A-Guide-to-Conduct-Clinical-Trials-in-Malaysia.pdf

		National Pharmaceutical Regulatory Agency, 2018. Malaysian Guideline for Good Clinical Practice 4th Edition. Forward to the Fourth Edition, Page 4.	

		National Institutes of Health, n.d. Good Clinical Practice Training. Retrieved 05Nov2019 from: 

https://grants.nih.gov/policy/clinical-trials/good-clinical-training.htm

		Al-Nomay NS, 2015. Compliance with ICH-GCP Guidelines among the Saudi Health Care Professionals: Should Saudi Arabia Conduct Widespread ICH-GCP Training? 

s. 2015; 1(2): 75-82. Retrieved Nov. 5, 2019 from: 

https://pdfs.semanticscholar.org/0112/2b2e7c0fdd762172251e807c273fcb4a399a.pdf?_ga=2.112527492.1640531090.1574928919-1845614418.1571207552

		Bonney KM, 2015. Case study teaching method improves student performance and perceptions of learning gains. 

. 2015;16(1):21–28. Retrieved Nov. 5, 2019 from: 

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4416499/

		Robin L. Bachelor Patrick M. Vaughan Connie M. Wall, 2012. Exploring the Effects of Active Learning on Retaining Essential Concepts in Secondary and Junior High Classrooms. Retrieved Nov. 5, 2019 from: 

https://files.eric.ed.gov/fulltext/ED531546.pdf

		Halkoaho A, Matveinen M, Leinonen V, Luoto K, Keränen T., 2013. Education of research ethics for clinical investigators with Moodle tool. BMC Med Ethics. 2013;14:53. Published 2Dec 12, 2013. Retrieved Nov. 5, 2019 from: 

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3867673/

 is Head of Clinical Operations, both from Clinical Research Malaysia

Articles

Survey results from Clinical Research Malaysia’s GCP refresher workshop.

Is Retraining on Good Clinical Practice Guidelines Effective?

	The International Conference on Harmonization-Good Clinical Practice (ICH-GCP) is an international ethical and scientific quality standard that ensures the rights, safety and well-being of clinical trial subjects are protected and that the clinical trial data generated are credible.

	Malaysia GCP Guideline was developed by National Pharmaceutical Regulatory Agency (NPRA) in 1999. This guideline adopts the basic principles outlined by ICH-GCP with some modifications to adapt to local conditions.

The object of the Malaysia GCP Guideline is to ensure that drug-related trials in Malaysia are conducted in accordance with international ethical and scientific standards.

 Although there is currently no legislative requirement for GCP training, mechanisms were in place to ensure that investigators are trained on GCP. For example, it is mandatory to submit a copy of Principal Investigator’s GCP certificate when obtaining regulatory approval from NPRA to conduct a clinical trial.

 In addition, all investigators must submit their GCP certificates for ethics committee’s approval in Malaysia.

 Since the first edition of Malaysian GCP to the current 4th edition, approximately 12,000 clinicians have been GCP certified.

	It is recommended for GCP certified personnel to be retrained on GCP guidelines every three years in order to stay up to date with updated regulations, standards, and guidelines.

 With this, Clinical Research Malaysia (CRM), a research organization founded by Malaysia Ministry of Health (MOH), has taken the initiative to conduct GCP Refresher Workshop for clinicians, nurses, allied health professionals and study coordinators who have been certified with Malaysian GCP. The objective of the workshop was to provide a review on the principles of GCP, to improve participants’ skills and knowledge on managing the conduct of a clinical trial, as well as to share recommendations on how to conduct a GCP-compliant clinical trial. 

	Since the first workshop in October 4, 2016, CRM has conducted 31 GCP refresher workshops throughout Malaysia to approximately 450 participants at no cost. The majority of the participants are clinicians, nurses, allied health professionals from government hospitals who are involved in clinical trials. Over the course of the conduct of the workshop, modifications have been made based on participants’ feedback to improve the quality and the learning outcome of the workshop. A summary of the changes in the workshop agenda is listed in Table 1.

Table 1: Changes in presented topics from 2016 to present

	As shown in Table 1, pre- and post- workshop assessment test was introduced in March 16, 2017 in order to evaluate the effectiveness of the workshop on participants’ knowledge on GCP.

	In this paper, we analyzed the pre- and post- workshop assessment score for workshops conducted since March 16, 2017. 

	Before the start of the workshop, participants were required to complete 20 multiple-choice GCP-related questions (pre-workshop assessment). The questionnaires were then collected before the start of the workshop. After the workshop, participants are then required to complete the same set of questions (post-workshop assessment). Participants were given 30 minutes to complete the questionnaire. 

	The average score of the pre- and post- assessment results from participants of the same workshop will then be calculated. This will constitute one set of data point.  

	The average score of pre- and post- workshop assessment from each workshop conducted since the introduction of the assessment in March 16, 2017 were compiled. Of the 28 workshops conducted since then, five workshops’ data could not be retrieved, hence only 23 sets of data point were analyzed.

	The compiled average score of pre- and post- workshop were then compared using Microsoft Excel’s t-test assuming unequal variance to evaluate if there is any significant improvement in the average assessment score after the workshop.

	Table 2 shows the compiled data from 23 workshops. 420 participants completed the pre-workshop assessment, and 434 participants completed the post-workshop assessment. There is on average, 20% improvement in the assessment score before and after the workshop.  

Table 2: Compilation of the average pre- and post- workshop assessment score.  

assuming unequal variances comparing the pre- and post- workshop average scores.

	As shown in table 3, the p-value is less than 0.05, suggesting that there the improvement in the score is significant after participants attended the GCP Refresher Workshop. 

	Training of investigators and allied health professions involved in clinical trials on the regulations and standards that govern clinical trials is important to improve the quality of studies and ensure maximum safety for the study subjects. A survey amongst clinical researchers in Saudi Arabia on their ICH-GCP knowledge found that its poor understanding of investigator’s responsibilities on informed consent, and prompt reporting to the ethics committee.

 Hence, refreshing investigators’ awareness of their responsibilities could help to improves the performance of investigators in conducting clinical trials particularly in these important ethical aspects.

	The GCP refresher workshop conducted by CRM fulfills its objective to provide revision of the principles of GCP to participants, with more an average of 20% improvement in assessment score after the workshop. 

	The factors that potentially contribute to the success of the CRM GCP Refresher workshop in improving participant’s score was discussed below:

	Case studies were included for “Informed Consent”, “Adverse Event Reporting” and “Protocol Deviations” for discussion. Participants were given real-life scenario case studies and they were required to answer guided questions in groups. Study have shown that case studies are more effective than textbook reading at promoting learning of key concepts and comprehension of the relevance of concepts to real life scenario.

	Several methods were used to make the workshop interactive such as games, quizzes, videos, group activities and group discussion. Active engagement of participants helps in the retain of information.

	Trainers would also actively engage participants by asking questions and encouraging participants to share their experience and practices. Regulations of clinical research can be understood more easily when participants can reflect the principles upon their own experience.

	Participants were given evaluation forms at the end of the workshop to provide their feedback and opinion of the workshop. Based on their feedback, modifications were made to improve the conduct of the workshop: 


		The first GCP Refresher Workshop is a half-day (4 hours) workshop. Participants reflected that the workshop felt rushed and suggested to make it a full-day course. Subsequent workshops were then conducted one full day (8 hours) to ensure participants can learn at a better pace, contributing a better learning outcome. 


		One of the earlier feed-back received is to include more activities especially for safety reporting and protocol deviations. Modifications based on this feedback is the shift from a lecture-style workshop to activity-based workshop. Presentation slides were also revised to be more concise to retain attention. 


		Following a feedback from participants, the current edition of the Malaysian GCP Guideline booklets were provided for participants to refer to during the workshop. 

	It is important for investigators and allied health professions who are involved in clinical studies to be trained on the latest GCP guideline. The GCP Refresher Workshop conducted by CRM is effective in refreshing participants’ knowledge on the GCP guideline. The success of the workshop could be attributed to providing case studies for discussion, interactive activities as well as taking participants’ feedbacks into consideration. We would recommend other governing bodies to support the conduct of similar interactive workshops in their jurisdiction; and to take participants’ feedback for continuous improvement. 

 is the Head of Clinical Operations, both of Clinical Research Malaysia.

		Good Clinical Practice Network, n.d. ICH GCP Good Clinical Practice. Retrieved 05Nov2019 from: 

		National Pharmaceutical Regulatory Agency, 2018. Malaysian Guideline for Good Clinical Practice 4th Edition. Introduction to Malaysian Guidelines For GCP, Page 8. 	

		National Pharmaceutical Regulatory Agency, 2017. Malaysian Guideline for Application of Clinical Trial Import Licence and Clinical Trial Exemption. Retrieved 05Nov2019 from: 

		Society Of Clinical Research Professionals Malaysia, 2016. A Guide to Conducting Clinical Trials in Malaysia, First Edition. Retrieved 05Nov2019 from: 

		National Pharmaceutical Regulatory Agency, 2018. Malaysian Guideline for Good Clinical Practice 4th Edition. Forward to the Fourth Edition, Page 4. 	

, n.d. Good Clinical Practice Training. Retrieved 05Nov2019 from: 

		Al-Nomay NS, 2015. Compliance with ICH-GCP Guidelines among the Saudi Health Care Professionals: Should Saudi Arabia Conduct Widespread ICH-GCP Training? J Public Health Dev Ctries. 2015; 1(2): 75-82. Retrieved 05Nov2019 from: 

		Bonney KM, 2015. Case study teaching method improves student performance and perceptions of learning gains. J Microbiol Biol Educ. 2015;16(1):21–28. Retrieved 05Nov2019 from: 

		Robin L. Bachelor Patrick M. Vaughan Connie M. Wall, 2012. Exploring the Effects of Active Learning on Retaining Essential Concepts in Secondary and Junior High Classrooms. Retrieved 05Nov2019 from: 

		Halkoaho A, Matveinen M, Leinonen V, Luoto K, Keränen T., 2013. Education of research ethics for clinical investigators with Moodle tool. BMC Med Ethics. 2013;14:53. Published 2013 Dec 12. Retrieved 05Nov2019 from: 

Yoong Kai Shen and Joanne Yeoh, both of Clinical Research Malaysia, analyze the pre- and post- workshop assessment score for GCP workshops conducted since 16 March 2017. 

Yoong Kai Shen

Articles

Is Retraining on Good Clinical Practice Guidelines Effective?

Is GCP Refresher Training Effective—Perspective from Malaysia