IMSH Delivers Sessions - Scroll down to "ADD to your Briefcase"
Crowdsourcing Authoring for Healthcare Competence Assessment Tool (1090-004422) (Research Abstract Professor Rounds: Group 1)
Start time: Tuesday, January 26, 2021, 2:00 PM End time: Tuesday, January 26, 2021, 3:00 PM Session Type: Research Abstracts (Completed Studies) Cost: $0.00
Content Category: Researcher
Hypothesis:
The current OSCE development method is complex and costly, and many institutions lack human and financial resources to develop high-quality assessments. Crowdsource authoring has been highly successful in both the medical field and non-medical field. The aim of this project was to create a platform in which experts from around the world can easily interact and exchange materials, ideas and opinions related to competency assessment tool development. The Crowdsource Authoring Assessment Tool (CAAT) is an online collaboration system that allows the creation and sharing of assessment tools that can be easily edited and customized to match local needs and conditions.This research project analyzed data collected from recognized international experts in the field of competency assessment after participating in the CAAT system beta test.
Methods:
Fifty experts were asked to edit a urinary catheterization checklist using CAAT online system, after which participants completed a Technology Acceptance Model (TAM) questionnaire. The scale consisted of 14 items evaluating four domains of TAM.The results were analyzed using descriptive statistics for background information, and inferential statistics for hypothesis testing. This study conducted an independent sample t-test, Pearson correlation, one-sample t-test, and dependent sample t-test according to the hypotheses.
Results:
This study found positive feedback about CAAT acceptance from experts. The results demonstrated experts with less experience were significantly spent less time. They also scored higher in the intention to use the CAAT in the future. The median time for developing a new checklist by using the CAAT was 65.76 minutes while the traditional method was reported at 167.90 minutes. Experts expressed positive feelings towards crowdsourced authoring. Respondents expressed they were inclined to share checklists and would like contributors to be acknowledged for their contributions, and also agreed that the CAAT could potentially change practices in checklist development. 96% of participants indicated they were willing to recommend CAAT to others.
Conclusions:
In this study, the CAAT was evaluated and accepted by experts to be used in designing a medical competence assessment tool.The CAAT system represents a first step in the concept of online communities for assessment development, through a simple yet effective method that allows collaboration regardless of geographical location and local institutional development. The future of assessment lies in the collaboration of experts free from the barriers of geography, finances and time constraints, the CAAT takes steps towards promoting this future through the power of the crowdsource authoring.