Evaluating Artificial Intelligence to Assess Communication Skills of Anesthesiology Trainees Compared to Human Evaluators Using Voice-Based Simulations

What this Trial Is About

Researchers are evaluating a web-based artificial intelligence platform called Clinical Mind AI CMAI to assess communication skills in anesthesiology trainees such as residents and fellows. The study focuses on trainees ability to disclose medical errors, using an AI-generated voice simulation involving a clinical case of a pediatric patient who suffered a dental injury during intubation. The goal is to compare AI assessments with human evaluations to determine reliability, usability, and satisfaction with this innovative training method. Participants will engage remotely with the AI platform through a voice call simulating a conversation with the parent of a pediatric patient. After the simulation, their communication skills will be evaluated by both the AI system and two human raters using the same standardized rating scales. The study will measure how well the AI assessment matches human assessment, as well as gather feedback on the platforms usability and trainee satisfaction. During the study, participants will complete the AI voice simulation and then fill out questionnaires on their experience. Assessments occur immediately after the simulation to capture performance reliability and user impressions. The study includes graduate medical trainees able to communicate in English and able to use the necessary technology. Participation involves remote interaction and evaluation, with the entire process designed to assess communication skill training in complex clinical conversations.

Use of Artificial Intelligence to Assess Trainee Communication Compared to Human Assessment