Application and Effectiveness of a Large Language Model-Based Educational Agent in Medical Education: A Study on the Machine Learning and Data Mining Course

What this Trial Is About

Researchers are evaluating the effectiveness of a Large Language Model (LLM)-based educational AI Agent for graduate students in medicine or nursing enrolled in the "Machine Learning and Data Mining" course. This study aims to determine if using the AI Agent improves academic performance, practical machine learning skills, learning confidence, satisfaction, and cognitive engagement compared to traditional teaching methods. The study uses a non-randomized design comparing students using the AI system to a historical control group from the previous academic year without AI support. The intervention includes a custom-developed AI Educational Agent system powered by LLMs and Knowledge Graph-based Retrieval-Augmented Generation (KGRAG) technology. The system has three modules: the Teaching Agent offers 24/7 tutoring with concept explanations and personalized study plans; the Research Agent aids literature review and academic writing; and the Practice Innovation Agent supports coding, debugging, and applying machine learning models using a Socratic tutoring method. Participants in the 2025-2026 academic year will use this system throughout the semester. Participants will engage with the AI Agent to enhance their learning experience while researchers assess outcomes such as a composite academic performance score measured at the end of the course (approximately week 3). Other evaluations include knowledge acquisition rates, technology acceptance, and patterns of AI Agent usage. Students' interaction with the AI system and their academic progress will be monitored to measure the intervention's impact during the study period ending in March 2026.

An AI Educational Agent for Medical Machine Learning Courses