Rapid Transition to Tele-OSCE Assessment: COVID-19 Effects on Clinical Skills Summative Assessment Using an Online Platform (1090-004021) (Research Abstract Oral: Virtual/Augmented Reality)
Start time: Monday, January 25, 2021, 9:30 AM End time: Monday, January 25, 2021, 10:30 AM Session Type: Research Abstracts (Completed Studies)
Content Category:
Researcher
Hypothesis:
Objective Structured Clinical Examination (OSCE) is an assessment tool commonly used in Medical schools for academic progression and as a confirmation of minimal clinical skill (CS) competency. Social distancing policies and mandatory quarantine due to the COVID 19 pandemic meant that in-person OSCE was no longer a safe option. Medical schools around the world who utilize this tool were forced to urgently adjust their curriculum. Several Medical Schools have adapted TeleOSCE model as a temporary solution. We will share our observations during this transition, which took place in the middle of a two-week OSCE CS assessment at Ross University School of Medicine. The purpose of this study is to retroactively analyze and compare if student assessment of CS competency differed between in-person OSCE and TeleOSCE. Our study demonstrates minimal differences in the scoring of the subcomponents (CIS, SEP, and ICE) and their respective competencies in preparation for the USMLE Step 2 CS.
Methods:
There were two cohort of a total of 273 students with similar academic background. The first cohort consisted of 175 students with 2.87 GPA. This group was assessed during the in-person OSCE during week 7 of the clerkship. The second cohort consisted of 98 students with 2.91 GPA. This group received the same mix of OSCE cases, represented by the same SPs and graded by the same clinical skills faculty. A one-way ANOVA was applied to test within-subject effects of the means between each group in the categories of communication and interpersonal skills (CIS), spoken english proficiency (SEP), and the integrated clinical encounter (ICE) which consisted of the physical exam and the patient note, where F test of significance, P=0.05, was applied to test the null hypothesis that there were no differences within-subject. Assumption of normality of the distributions of scores and homogeneity of variances (using Levene’s test statistic) was tested with an alpha of 0.1.
Results:
Results of ANOVA indicated that the means between each group in the categories of communication and interpersonal skills (CIS), spoken english proficiency (SEP), and the integrated clinical encounter (ICE) were not statistically significantly different. However, for physical exam, results of ANOVA indicated statistically significant [F(1,271) = 235.32, p=.00] differences for each group. Students in the OSCE group (mean 49.81, SD±.10) performed lower on physical exam than students in the TeleOSCE group (mean 70.64, SD±.11).
Conclusions:
Our initial data demonstrates that summative assessment of CSs is not significantly different between OSCE and TeleOSCE for areas that involve data gathering, data interpretation, and data reporting. This was demonstrated by showing no difference in CIS, SEP, and Patient note scores. The notable exception to this finding was a significant difference with the assessment of physical exam skills. TeleOSCE has a set of new skills that are valid in the telehealth environment. These skills require an adaptable training that is different from the traditional OSCE, and medical schools may be missing an opportunity to leverage valid components of telehealth models for CS assessments. As a result, medical students are scoring higher in the Physical Exam shown in the results above. Further study is needed to find out if medical students are meeting their CS competency requirements and demonstrating proficiency during times of restricted person to person interaction in medical education.