Benchmarking Advanced Large Language Models Against Multidisciplinary Tumour Boards for Cancer Treatment Planning in Breast, Lung, Urological, Digestive, and Gynaecological Cancers

What this Trial Is About

Researchers are evaluating how five advanced large language models LLMs compare to multidisciplinary tumour boards in recommending cancer treatments. This prospective, multicenter, blinded, non-interventional benchmark study uses synthetic, standardized oncology cases across breast, lung, urological, digestive, and gynaecological cancers to assess decision-making quality. The study aims to measure concordance between LLM and human tumour board recommendations using objective guidelines and consensus as references. Participants are not individuals but synthetic cases representing various cancer types. Each case is presented identically to two independent tumour boards and five cutting-edge LLMs. Recommendations in five decision domainsintent, surgery, radiotherapy, systemic therapy, and work-upbiomarkersare scored against a locked guideline matrix from ESMO and NCCN. Cases where tumour boards disagree are analyzed separately, and safety endpoints assess potentially harmful recommendations. The study involves automated scoring of recommendations once all data are collected, expected by October 2026. Outcomes include agreement levels between LLMs and tumour boards, potential serious harm in recommendations, and other metrics such as completeness and bias. All synthetic cases, scoring algorithms, and evaluation criteria are openly available for reproducibility, ensuring transparent benchmarking of AI in oncology decision support.

Benchmarking Large Language Models Against Tumour Boards for Oncology Treatment Recommendations