Browsing by Subject "LLM"

Now showing 1 - 2 of 2
  • Thumbnail Image
    Publication
    Open Access
    Machine vs Machine: Large Language Models (LLMs) in Applied Machine Learning High-Stakes Open-Book Exams
    (Universidad de Murcia, Servicio de Publicaciones, 2024-05-30) Quille, Keith; Alattyanyi, Csanad; Becker, Brett A.; Faherty, Róisín; Gordon, Damian; Harte, Miriam; Hensman, Svetlana; Hofmann, Markus; Jiménez García, Jorge; Kuznetsov, Anthony; Marais, Conrad; Nolan, Keith; Nicolai, Cianan; O’Leary, Ciarán; Zero, Andrzej
    There is a significant gap in Computing Education Research (CER) concerning the impact of Large Language Models (LLMs) in advanced stages of degree programmes. This study aims to address this gap by investigating the effectiveness of LLMs in answering exam questions within an applied machine learning final-year undergraduate course. The research examines the performance of LLMs in responding to a range of exam questions, including proctored closed-book and open-book questions spanning various levels of Bloom’s Taxonomy. Question formats encompassed open-ended, tabular data-based, and figure-based inquiries. To achieve this aim, the study has the following objectives: Comparative Analysis: To compare LLM-generated exam answers with actual student submissions to assess LLM performance. Detector Evaluation: To evaluate the efficacy of LLM detectors by directly inputting LLM-generated responses into these detectors. Additionally, assess detector performance on tampered LLM outputs designed to conceal their AI-generated origin. The research methodology used for this paper incorporates a staff-student partnership model involving eight academic staff and six students. Students play integral roles in shaping the project’s direction, particularly in areas unfamiliar to academic staff, such as specific tools to avoid LLM detection. This study contributes to the understanding of LLMs' role in advanced education settings, with implications for future curriculum design and assessment methodologies.
  • Thumbnail Image
    Publication
    Open Access
    Mapping Applications and Outcomes of Large-Language-Model-Generated Cases in Health Professions Education : A Scoping Review
    (Universidad de Murcia, Servicio de Publicaciones, 2026) Kaya, Abdullah Bedir; Emekli, Emre; Kiyak, Yavuz Selim; Sin departamento asociado
    Objective: Large language models (LLMs) have rapidly permeated health professions education and are increasingly used to generate clinical cases and vignettes, yet their characteristics, evaluation methods, and educational impact remain unclear. To map how LLMs are used to generate cases in health professions education and to summarize reported case characteristics, evaluation approaches, bias, and educational outcomes. Methods: We conducted a scoping review following Arksey and O’Malley’s framework and reported using PRISMA-ScR. PubMed, Web of Science, and Scopus were searched on 27 August 2025. Of 2023 records, 72 full texts were assessed and 23 studies met inclusion criteria. Data were charted with a structured extraction form. Results: Across the 23 studies, 33 distinct LLMs were used, most commonly GPT-based models (54.5%). Cases were mainly text-based (69%), with additional image- (20.7%) and audio-based (10.3%) formats across 23 clinical and educational domains. Prompts were reported in 65.2% of studies, and 60.9% included a formal quality evaluation, ranging from high quality to clearly problematic examples. Seven studies (30.4%) identified bias or discriminatory patterns. Student participation occurred in 39.1% of studies, but no higher-level educational outcomes such as behavior change or long-term performance were reported. Conclusions: LLM-generated cases appear feasible and versatile across health professions education but are supported by early, methodologically heterogeneous evidence. Future research should standardize quality evaluation, rigorously assess learning and behavioral outcomes, and systematically audit bias in generated content.