Evaluating GPT-4o and Claude AI Models for HEART Score Accuracy and 30-Day Cardiac Event Prediction in Emergency Department Patients with Non-Traumatic Chest Pain

What this Trial Is About

This research aims to evaluate the diagnostic accuracy of large language models LLMs, specifically GPT-4o and Claude, in calculating HEART scores and predicting 30-day major adverse cardiac events MACE in adult patients presenting with non-traumatic chest pain in emergency departments. The study compares these AI-based calculations against a reference standard established by three emergency medicine physicians using expert consensus. It also investigates how well routine Turkish emergency anamnesis notes document HEART score parameters. The study involves submitting anonymized patient datacomprising free-text anamnesis, ECG report text, troponin values, and ageto each LLM independently using standardized API calls with deterministic settings. The LLMs generate HEART score components, total score, risk group, and indicate any indeterminate results. A three-expert consensus scoring of the same patient records serves as the reference standard. The study enrolls 600 consecutive adult patients and classifies cases with missing data into indeterminate tiers for analysis. Participants have their clinical data reviewed, and 30-day outcomes of major adverse cardiac events are tracked via national health databases and telephone follow-up. Researchers measure the agreement between LLMs and expert scores, diagnostic performance metrics such as sensitivity and specificity, and differences between the two LLMs predictions. The study also assesses documentation quality of clinical notes and evaluates potential algorithmic bias by age and sex subgroup analysis. The total participation involves data collection at the index emergency visit and follow-up over 30 days.

Diagnostic Accuracy of GPT-4o and Claude for HEART Score Calculation in Chest Pain