Evaluation of a simulation-based hospital pharmacy training package for pharmacy students

Medication-related problems (MRPs) that include medication errors, adverse effects, medicine interactions and non-adherence are a major burden on health care systems. In Australia, it has been estimated that more than 190,000 Australian hospital admissions per year are attributable to MRPs, with an associated cost of AUS$660 million (Roughead and Semple 2009). A proportion of MRPs are caused by medication errors which often arise during prescribing, dispensing or administration of medicines and contribute to patient harm or injury, and in some cases mortality (Runciman et al. 2003). Research by Roughead and Semple (2009), Roughead et al. (2016) indicates that approximately 50% of Australia’s hospital admissions due to MRPs are considered potentially preventable. Consequently, appropriate and correct management of medicines by health professionals plays a central role in facilitating better health outcomes (Aronson 2009; Roughead et al. 2016, 2013). However, it was highlighted by Stowasser et al. (2004) that the medication management pathway involves complex steps, with the potential for errors and interventions at various stages. It is therefore important that health services integrate medication management processes and that health professionals are adequately trained to follow these processes to reduce incidences of MRPs.

The Australian Commission on Safety and Quality in Health Care 2016 in Healthcare prioritised the need to improve the safety and quality of medicine usage and developed a number of medication safety tools and resources to support hospital and community practices in achieving better outcomes. These initiatives align with quality use of medicines (QUM) principles as specified in the Australian National Medicines Policy (Australian Government Department of Health and Ageing 2002; Stowasser et al. 2004). The continued appropriate use of these tools and resources by all health professionals involved in patients’ medication management remains an important strategy to ensure QUM and continuum of care within clinical practices and across healthcare settings. Hospital pharmacists play an important role in the integration and application of these clinical resources and the coordination of medication management services (Society of Hospital Pharmacists of Australia 2013). Published studies continue to support the role of hospital pharmacist services in improving medication safety, including continuum of care services (Hennen and Jorgenson 2014; Hilmer and Ogle 2006; Semple and Roughead 2009).

Pharmacy students need to understand medication management processes within a variety of settings, including community and hospital sectors. Indeed, the Australian Pharmacy Council Accreditation Standards for Pharmacy Programs (2012) states that it is important for hospital practice settings to be experienced by students during pharmacy degree programs. Exposing pharmacy students to hospital pharmacy practices provides an opportunity for students to develop a working knowledge of hospital procedures and an appreciation of the role of hospital pharmacists. Hospital pharmacy placements also facilitate the integration and application of theory knowledge. However, Australian universities increasingly find it challenging to organise hospital placements for all pharmacy students. Research by Fejzic et al. (2016) suggested a variety of reasons for this including course time constraints and high competition for limited hospital places. Pharmacy academics therefore need to explore alternative approaches to equip students to practise in hospital pharmacy settings.

Simulation is a generic term that refers to the replication of real world experiences. According to Gaba (2004) simulation-based education is an educational approach that uses simulation, in its many applications and modalities, to achieve educational goals through experiential learning. International literature emphasizes the need to utilize an assortment of technology-based learning environments to train and educate health professionals (Cook et al. 2011; Keppell et al. 2015) and specifically for pharmacists (Fejzic and Barker 2015; Smith and Benedict 2015). Multiple studies support embedding technology-enhanced, simulation-based education into pharmacy curricula, such as online learning that is enriched by pre-recorded role plays with actors or simulated participants, virtual patients, task simulations and/or interactive platforms that simulate a specific skill or competency (Fejzic and Barker 2015; Kirwin et al. 2013; Loke et al. 2011; Regan et al. 2014; Salter et al. 2014; Smith and Benedict 2015; Veettil and Rajiah 2016; Vyas et al. 2013).

A systematic review and meta-analysis by Cook and colleagues that compared technology-enhanced simulation-based education with no intervention concluded that greater effects were consistently seen in a number of outcome areas including participants’ skills, knowledge and behaviours, with simulation based education and training (Cook et al. 2011). However, within Australia, there is a lack of evidence on the development, implementation and evaluation of simulation-based hospital medication management processes in the training of pharmacy students. A need was hence identified to develop a simulation-based training package that was focused on hospital medication management processes to assist pharmacy students in the application of medication management tools and utilization of relevant resources.

The overall aim of this project was to develop, implement and evaluate a simulation-based training package to expose final year Bachelor of Pharmacy (BPharm), BPharm Honors and Graduate Entry Master of Pharmacy (MPharm) students to a patient’s hospital journey focusing on medication management and reconciliation processes. It was hypothesized that the exposure of pharmacy students to the simulated hospital journey training package would assist them in gaining an understanding of hospital medication management processes and procedures, how hospitals function, how the pharmacist works within the hospital and the importance of the continuum of care. The purpose of introducing the simulated-based training package in the curriculum was to better prepare students before hospital placements.

This project involved the development of the simulation-based training package, pre-testing of students’ knowledge and skills, utilization of the training package, post-testing of students’ knowledge and skills and obtaining students’ feedback. Mixed methodology (Dowding 2013) was used for the evaluation of the training package and the impact of the training on students’ knowledge and skills. Evaluation involved a written pre- and post-test to explore students’ knowledge of medication management processes and skills in the application of medication management processes (i.e. through the use of clinical tools), and an end-of-training survey to obtain quantitative and qualitative feedback on student satisfaction with the training package. The pre-and post-tests and surveys were validated by a pharmacy academic and two hospital pharmacists.

Curtin University Human Research Ethics Committee approval was granted to conduct this research (RDHS-63-16). The project was divided into the four phases with Phase 1 being the development of the training package, Phase 2 the pre-testing of students’ knowledge, skills and confidence, Phase 3 the students’ self-directed learning through completing the training package and Phase 4 the post-testing of students’ knowledge, skills, confidence and obtaining their feedback.

A total of 105 students (31 BPharm students who had completed a hospital placement, 48 BPharm and 17 MPharm students who had completed a non-hospital community placement, and 9 BPharm Honors students who had completed no placement) were invited to placement orientation workshops and were invited to participate in the study. Pre-test data from the 79 students who agreed to participate (75.2%) were analysed. These participants comprised 27 BPharm students who had completed a hospital placement, 36 BPharm and nine MPharm students who had completed a non-hospital community placement, and seven BPharm Honors students who had completed no placement. Pre- test scores were available for all 79 participants, but one of these responders did not complete the ranking questions (Section B). Of these 79 participants, 44 (55.7%) completed the post-test questions. These included 17 students who had completed a hospital placement and 27 students who had not (18 BPharm, four MPharm and five BPharm Honors students).

The change in score for the 44 participants who completed both the pre- and post- tests was statistically significantly greater than zero when using the one sample t-test to compare the mean change (p < .0001) or the signed rank test to compare the median change (p < .0001). Therefore, there seems to be strong evidence that the scores improved (by 11.2 points on average) following the training.

Because of the large proportion of participants who did not complete the post-test survey due to being off-campus to study for examinations, an independent samples t-test was performed to compare the baseline scores for those who did and did not complete the post-test. This analysis confirmed that these two groups performed similarly at baseline (mean scores: 29.5, 30.7 respectively; p = .61). This suggests no systematic bias in the post-test responders (i.e. students who completed the post-test performed no better or worse at baseline than those who did not do the post-test survey).

The comparison of the pre-and post-test confidence levels of the ranking statements about confidence with medication management roles and processes are summarised in Fig. 1.

The results show that the students who had completed a hospital placement gained significantly higher total scores at pre-test compared to the other students, while there were no significant pairwise differences between the remaining three groups. Analysis of the change in total scores showed that the hospital placement group showed the least improvement in score (probably because they were starting at a higher baseline), and this was a significantly smaller improvement than either the groups who completed community placements students or the Honors students. However, the improvement for the MPharm students was intermediate and not significantly different from any of the other groups. At post-test, there were no significant differences between groups overall (p = 0.14), showing that the groups had all converged by the end of the training.

A GLM was used to explore whether a change towards agreement with each of the confidence statements in section B was associated with a change in the students’ test scores (section A). It appeared that Question 2 (I am confident using a medication management plan) showed the greatest correlation with the improvement in marks: those whose agreement increased for this question obtained a score which was 13 marks higher than their pre-test score, while those whose Question 2 agreement was unchanged increased in score by 8.3 marks (p = .03). This suggests that greater confidence in using this plan was associated with a jump in the post-test score.

A second regression model (Mixed model) was used to explore whether the knowledge and skills test score was associated with the confidence score. This model accounted for the repeated measurements made on each person by treating the person as a random effect. The results showed that an increase of 1 point in the confidence score corresponded to an increase of 0.50 in the knowledge and skills test score (p < .0001). There was no influence of the student category type (1–4) on this relationship, which means that a similar improvement in test score with increasing section B score was noticed for all student groups.

The training package appeared to significantly increase test scores, and agreement with the statements which assessed confidence in dealing with a range of clinical pharmacy situations. There were differences at baseline between the four different student types with those who had completed a hospital placement scoring the highest. Following the training, all groups showed improvements and gained similar test scores. Assessment of their confidence according to the 16 ranking statements also improved markedly post-training.

International literature continues to reinforce the need for and the positive effects associated with the use of simulation-based methodologies as part of pharmacy educational programs (Kirwin et al. 2013; Loke et al. 2011; Murdoch et al. 2014; Regan et al. 2014; Ruehter et al. 2012; Salter et al. 2014; Smith and Benedict 2015; Veettil and Rajiah 2016; Vyas et al. 2013). However, there is limited information on Australian studies that involved the development, implementation, and evaluation of hospital-based simulation training packages within tertiary education. One Australian study (Fejzic et al. 2016) that focussed on pharmacy students’ communication skills involved supplementing the curriculum with simulation-based activities with positive enhancement of student confidence and professional practice skills. This study showed that the integration of a simulation-based training package into the curriculum of final year pharmacy students improved students’ knowledge base of hospital pharmacy processes as well as their confidence levels in feeling prepared for working in a hospital pharmacy environment.

The 44 students in our study who completed both pre-and post-test questions improved test scores on average by 11.2 marks with a t-test showing a statistically significant improvement in their knowledge and ability to apply medication management tools. An American study by Douglass et al. (2013) evaluated the impact of a simulation-based virtual patient program on third year pharmacy students’ competence in relation to their clinical skills, with post-test results indicating that 78% of the students agreed that the simulation improved their problem solving clinical skills. Kirwin et al. (2013) described subsequent effects on students’ confidence and performance abilities with the implementation and evaluation of a simulated hospital pharmacy module that reflected the day-to-day tasks of a hospital pharmacist. The pharmacy students participating in that study had prior exposure to hospital practice experience, however, it was reported that their confidence and comfort at completing hospital pharmacy tasks was low. Despite the prior experience, students completing their simulation module significantly improved their practice skills and confidence to complete tasks typical of a hospital pharmacist. The authors concluded that exposure to hospital information systems used within the module would be of benefit to pharmacy students prior to their placements. Another study incorporated patient simulated scenarios with other traditional educational methods to prepare medical students for practice with authors suggesting that this combination enhanced third and fourth year students’ critical evaluation skills in order to assess medicine safety (Karpa et al. 2015). Our hospital pharmacy training package will similarly be completed by all pharmacy students at Curtin University before exposure to their first hospital pharmacy placement.

Benefits of incorporating simulation-based activities into pharmacy curricula could result in reduced MRPs and associated adverse events experienced by individuals (Battaglia et al. 2012; Daupin et al. 2016; Metzger et al. 2015). Multiple studies have shown that pharmacists who are involved in medication management processes, such as medication reconciliation programs, improve patients’ clinical outcomes and assist in the reduction of hospital re-admissions (Mekonnen et al. 2016a, b; Steeb and Webster 2012). Our study has shown that the use of an online simulation-based training package which details the complex nature of medication management processes within a hospital setting helps to raise awareness and educate pharmacy students. This may enhance pharmacy students’ practice-readiness by developing their confidence to address MRPs, identify medication errors, and collaborate effectively within interprofessional teams.

The effectiveness of simulated learning in pharmacy is not limited to the domain of higher education. A descriptive cross-sectional study conducted in a 500-bed university hospital identified that a simulation exercise within a ‘medication errors room’ to test the hospital’s medication-use processes resulted in almost 68% of the staff participants that included medical, pharmacy and nursing health professionals, correctly identifying the medication-related errors (Daupin et al. 2016). The researchers emphasised that this type of simulation activity encourages good practice, raises awareness of medication safety and offers improvements in management systems. Another study which compared simulation-based training of nurses to traditional methods such as lecturing demonstrated that simulation-based activities provided a significant reduction of almost 25% in administration-related medication errors, whereas traditional formats such as a didactic lecture, had no effect on reducing medication administration errors for the same group (Ford et al. 2010).

The benefits seen by pharmacy students’ participation in the online medication management training program was enhanced by the inclusion of a simulated interprofessional communication activity. A recent study involving nurses, doctors and a pharmacist based on two medical wards in an acute hospital identified the interpersonal, authoritative and instructive styles used when communicating with patients about medication and highlighted the need for improved interprofessional collaboration in order to ensure safe, patient-centred care is employed during medication management (Liu et al. 2016). By using simulation-based learning, this complex and high level discourse could be explored in ways that engender empathy, respect and safety whilst allowing students the opportunity to practise and rehearse dialogue with the interprofessional team and the patient.

Online simulation-based learning is a beneficial method for engaging final-year pharmacy students in authentic, self-directed, flexible learning, which fills the gap between traditional didactic teaching methods and experiential learning. It has an integral role in helping to prepare students for fieldwork by empowering them to take more responsibility for their own learning, improving their knowledge and skills, and contributing to them being more career ready. The evaluation of the simulation-based training package showed improvements in students’ knowledge, application of clinical tools as well as their confidence in medication management processes. However, the sample size was small and hence the next step will be to conduct a full validation of the methodology and an evaluation of the implementation of the training package with a bigger cohort of students. This model will then provide a reproducible framework for other online simulation-based learning activities and could be applied within various professions and educational environments.