Learner Engagement During Virtual Simulation Sessions: A Pilot Study (1090-003670) (Research Abstract Professor Rounds: Group 6)
Start time: Friday, January 29, 2021, 11:30 AM End time: Friday, January 29, 2021, 12:30 PM Session Type: Research Abstracts (Completed Studies)
The Covid-19 pandemic has increased the need and appeal for virtual simulation-based education, but few studies have examined the vital engagement of learners among these techniques (1-3). Carolinas Simulation Center has a wide array of user groups and, to meet their needs, our educators developed several different styles of virtual simulation education. We hypothesize that measured learner engagement and engagement perceptions will vary depending on the style of virtual simulation in the following order of highest to lowest engagement 1) hybrid: some learners are in-person and others on Zoom, 2) observer: all learners are on Zoom and full debrief follows a video-recorded scenario, 3) vignette: all learners are on Zoom and with a flipped debrief model, specific teaching points are covered before and after watching short video-recorded vignettes, 4) demo: all learners are on Zoom and an instructional discussion follows a live procedural demonstration.
Sessions from 4 styles of virtual simulation (hybrid, observer, vignette, demo) were included in this study. There were 2-4 sessions per style and 12 or fewer learners per session. All sessions were audio-recorded. To established learner engagement, a checklist was completed by three reviewers while watching the sessions live and listening to the audio-recording. Session demographics are presented in Table #1. Measurements of communication included “Answers & Comments Made by Learners to Educators,” “Replies Made by Learners to Other Learners’ Answers & Comments,” “Percent Replies Made by Learners to Other Learners’ Answers & Comments,” and “Percent Learners with Cameras on.” Data were analyzed using t-tests, Mann-Whitney, ANOVA, and the Pearson Correlation tests. Survey responses were analyzed to compare learner self-perceived level of engagement to educator/Simulation Operations Specialist (SOS) perception of learner engagement using the Mann-Whitney test.
Comparing the 4 different styles of virtual simulation (Table 2), there were significantly (p<0.05) higher levels of “Answers & Comments Made by Learners to Educators per Hour” in the observer style compared to the vignette and hybrid styles. In addition, there were significantly (p<0.05) more “Replies Made by Learners to Other Learners’ Answers & Comments per Hour” in the hybrid style compared to the vignette style. Table 3 shows that learners with active cameras were strongly correlated with higher levels of communication in both the vignette and the observer styles. Furthermore, significantly fewer learners had an active camera in the vignette style when compared to both the hybrid and observer styles (Fig 1). Except for the vignette sessions, the self-perceived level of learner engagement matched the educator/SOSs’ perception of learner engagement. Educators using the vignette style believed the learners were significantly more engaged than the learners’ self-perceptions (Fig 2).
While our results did not show the anticipated ranking of engagement levels (hybrid, observer, vignette, demo), a significant variation in engagement was found. Based on these pilot findings, it appears that the observer style, instead of the hybrid, resulted in the highest engagement, while the vignette, instead of the demo, resulted in the lowest. Active cameras were correlated with higher engagement. Virtual learners participating in the hybrid sessions may have felt a disconnect or left out since their fellow learners were in-person, whereas each of the learners in the observer style were virtual. Except for the vignette sessions, learners in each group were previously acquainted, which possibly contributed to their low level of engagement. Although more data is required, these invaluable results can inform curriculum development and help determine the best format for maintaining learner engagement, as virtual simulation is essential for learners in today’s challenging world.