Publication:
Comparison of Automatic Item Generation Methods in the Assessment of Clinical Reasoning Skills

Thumbnail Image
Files
Automated+Item+Generation+Methods+final.pdf(717.83 KB)
Date
2025
relationships.isAuthorOfPublication
relationships.isSecondaryAuthorOf
relationships.isDirectorOf
Authors
Emekli, Emre ; Karahan, Betül Nalan
item.page.secondaryauthor
item.page.director
Publisher
Universidad de Murcia. Servicio de publicaciones
publication.page.editor
publication.page.department
Departamentos
DOI
https://doi.org/10.6018/edumed.637221
item.page.type
info:eu-repo/semantics/article
Description
Abstract
The use of automatic item generation (AIG) methods offers potential for assessingclinical reasoning (CR) skills in medical education, a critical skill combining intuitive andanalytical thinking. In preclinical education, these skills are commonly evaluated through writtenexams and case-based multiple-choice questions (MCQs), which are widely used due to the highnumber of students, ease of standardization, and quick evaluation. This research generated CR-focused questions for medical exams using two primary AIG methods: template-based and non-template-based (using AI tools like ChatGPT for a flexible approach). A total of 18 questions wereproduced on ordering radiologic investigations for abdominal emergencies, alongside faculty-developed questions used in medical exams for comparison. Experienced radiologists evaluatedthe questions based on clarity, clinical relevance, and effectiveness in measuring CR skills. Resultsshowed that ChatGPT-generated questions measured CR skills with an 84.52% success rate,faculty-developed questions with 82.14%, and template-based questions with 78.57%, indicatingthat both AIG methods are effective in CR assessment, with ChatGPT performing slightly better.Both AIG methods received high ratings for clarity and clinical suitability, showing promise inproducing effective CR-assessing questions comparable to, and in some cases surpassing, faculty-developed questions. While template-based AIG is effective, it requires more time and effort,suggesting that both methods may offer time-saving potential in exam preparation for educators.
publication.page.subject
Automated item generation , Clinical reasoning , Template-based method , ChatGPT , Multiple-choice questions
Citation
Revista Española de Educación Médica, Vol. 6 Núm. 1 (2025)
URI
http://hdl.handle.net/10201/155792
item.page.embargo
Collections
Revista Española de Educación Médica Vol. 6 Nº 1 (2025)
Ir a Estadísticas