International Mixed Methods Study on the Usability of a 2nd Generation VR Sterile Catheter Insertion Game (1090-003790) (Research Abstract Oral: Procedure)
Start time: Monday, January 25, 2021, 8:00 AM End time: Monday, January 25, 2021, 9:00 AM Session Type: Research Abstracts (Completed Studies)
Research Questions - What is the usability of the 2nd Generation Sterile Urinary Catheter Insertion Game (VR SUCIG)? - What are user reactions to the 2nd Generation Sterile Urinary Catheter Insertion Game? Background: Psychomotor skills teaching remains understudied in nursing education. Learning and retention of psychomotor skills in health care is essential. A VR SUGIG incorporating haptics was designed using Bauman’s layered-learning model as a framework.(1, 2) The game was created to address the need for students to practice sterile catheter insertion in a step-wise manner and to point out contamination errors. Testing of the 2nd version of this game was conducted with 10 schools of nursing from the United States and Australia. Three-hundred nursing students and 46 nursing faculty and healthcare professionals completed Usability and User Reaction surveys. Bauman's Layered Learning model guided the research approach. (1,2)
Subsequent to IRB clearance a sample of 300 nursing students, and 46 nursing faculty and healthcare professionals were recruited. Nine universities in the United States and one from Australia participated. Participants were recruited, consented and offered game tutorials. Each was allowed up to 30 minutes to play the game, then complete a Usability Survey (3) and User Reaction Survey (4, 5). The System Usability Scale (SUS) measures the perceived usability of a product and has a tau-equivalent reliability (coefficient alpha) of .92. The 10 item scale measures effectiveness of computer applications. An average score is 50.9 (0-100 range) (SD 13.8).(5) The User Reaction Scale designed by Butt et al. (4) elicits user responses to perceived impact of a game on learning. Users respond to Likert style questions on a five-point scale anchored by strongly agree to strongly disagree. Tau-equivalent reliability (coefficient alpha) of the URS in the Butt et al. (4) study was .83.
Descriptive statistics were calculated: means (standard deviations) for quantitative data and frequencies and percentages (relative frequencies) for qualitative data. The system usability score (SUS) for the student cohort was 57 (SD 17.3). The SUS for the faculty/professional cohort was 47 (SD 25.6). The overall SUS 54 (SD 18.9). The variation between faculty and student SUS score p = .0176. The User Reaction Scores: Students and faculty/professionals thought the game was fun, challenging and engaging, user reactions demonstrated greater readiness for adoption in the student group. The faculty/professional cohort had challenges learning the technical aspects of the game. The last question in the URS asked participants to share suggestions for improvement. Responses included; loving the game, great for learning, great for remembering steps, frustration with technical glitches, unfamiliar with VR function. Students appreciated step-wise practice and seeing the germs.
This paper reports the results of a usability and user reaction survey of the 2nd generation of the VR-SUCIG, completed by 300 nursing students, and a 2nd cohort of 46 nursing faculty and professionals. The system usability score in the student group was 57 (SD 17.3) The system usability score in the faculty professional group was in the medium-low range 47 (SD 25.6), and user reactions were mixed. Future studies should include faculty/professional feedback. However, adding a detailed orientation to VR gaming technology, purpose of the game and purpose of a usability study are essential for faculty and professionals. Researchers may also glean valuable information about faculty perceptions if questions about the pedagogical value of the game are included. Future faculty need to consider benefits of moving from a digitally naïve state to becoming masters of technology that students are embracing. The game needs further refinement before marketability will be successful.