Browsing by Subject "LLMs"

Now showing 1 - 2 of 2
  • Thumbnail Image
    Publication
    Open Access
    Characterization of machine learning compilers for LLM inference on NVIDIA GPUs
    (Springer, 2015-05-15) Alejandro Carmona-Martínez,; Bernabé García, Gregorio; José Manuel García; Ingeniería y Tecnología de Computadores; Facultades de la UMU::Facultad de Informática
    AI inference is conflicted between Performance, developer Productivity, and device Portability–the P3 problem. Machine learning compilers (MLCs) aim to address this, but their ecosystem is fragmented, with tools that each prioritize a different issue. This paper evaluates the deployment trade-offs of PyTorch-based LLMs on NVIDIA GPUs using four intertwined prominent MLC tools: torch.compile, TensorRT, XLA, and ONNX Runtime. A dual methodology is used, leveraging synthetic PyTorch models to isolate optimizations and end-to-end benchmarks with State-of-the-Art (SOTA) models (TinyLlama-1.1B, Llama-2-7B) to measure realworld performance. Findings reveal that the peak performance of Ahead-Of-Time (AOT) compilation requires architecture-specific tools such as TensorRT-LLM, which are necessary for SOTA LLMs but are unusable for PyTorch models. As for Just-In-Time (JIT) solutions such as torch.compile and its backends, they are flexible and portable, compatible with all tested models, but they do not consistently accelerate LLMs; therefore, the choice of MLC depends on P3 considerations and model architecture.
  • Thumbnail Image
    Publication
    Open Access
    Machine vs Machine: Large Language Models (LLMs) in Applied Machine Learning High-Stakes Open-Book Exams
    (Universidad de Murcia, Servicio de Publicaciones, 2024-05-30) Quille, Keith; Alattyanyi, Csanad; Becker, Brett A.; Faherty, Róisín; Gordon, Damian; Harte, Miriam; Hensman, Svetlana; Hofmann, Markus; Jiménez García, Jorge; Kuznetsov, Anthony; Marais, Conrad; Nolan, Keith; Nicolai, Cianan; O’Leary, Ciarán; Zero, Andrzej
    There is a significant gap in Computing Education Research (CER) concerning the impact of Large Language Models (LLMs) in advanced stages of degree programmes. This study aims to address this gap by investigating the effectiveness of LLMs in answering exam questions within an applied machine learning final-year undergraduate course. The research examines the performance of LLMs in responding to a range of exam questions, including proctored closed-book and open-book questions spanning various levels of Bloom’s Taxonomy. Question formats encompassed open-ended, tabular data-based, and figure-based inquiries. To achieve this aim, the study has the following objectives: Comparative Analysis: To compare LLM-generated exam answers with actual student submissions to assess LLM performance. Detector Evaluation: To evaluate the efficacy of LLM detectors by directly inputting LLM-generated responses into these detectors. Additionally, assess detector performance on tampered LLM outputs designed to conceal their AI-generated origin. The research methodology used for this paper incorporates a staff-student partnership model involving eight academic staff and six students. Students play integral roles in shaping the project’s direction, particularly in areas unfamiliar to academic staff, such as specific tools to avoid LLM detection. This study contributes to the understanding of LLMs' role in advanced education settings, with implications for future curriculum design and assessment methodologies.