Artificial intelligence in cancer imaging: Clinical challenges and applications.
Wenya Linda Bi, Ahmed Hosny, Matthew B Schabath...
https://pubmed.ncbi.nlm.nih.gov/30720861Actively Recruiting
Age: 18Years - 75Years
FEMALE
Healthy Volunteers
ID07500428
Construction of a Standardized Benchmark Evaluation System for Intelligent Breast Ultrasound Image Interpretation and Systematic Performance Assessment of Multimodal Artificial Intelligence Models Based on ACR BI-RADS v2025 Criteria
Led by Peking Union Medical College Hospital · Updated on 2026-03-30
1380
Participants Needed
1
Research Sites
12 weeks
Total Duration
On this page
Sponsors
P
Peking Union Medical College Hospital
Lead Sponsor
C
Chinese Academy of Medical Sciences
Collaborating Sponsor
AI-Summary
What this Trial Is About
Researchers are conducting a single-center, retrospective observational study to develop a standardized benchmark system for evaluating intelligent breast ultrasound image interpretation. The study focuses on assessing the diagnostic accuracy of current mainstream multimodal artificial intelligence (AI) models in classifying breast ultrasound images according to the American College of Radiology (ACR) BI-RADS v2025 criteria. This research aims to address variability in ultrasound interpretation, especially for certain lesion categories, and to systematically evaluate AI performance using expert-annotated images. The study uses approximately 1,380 de-identified B-mode breast ultrasound images collected from an institutional archive and open-access datasets, covering normal breast tissue, benign lesions, and malignant lesions. Expert radiologists with varying experience levels will annotate all images independently. Baseline deep learning models (ResNet-50 and USFM) will establish performance baselines, and multiple multimodal large language models (MLLMs) will be evaluated using standardized chain-of-thought prompts through API calls. Safety assessments include out-of-distribution rejection testing and temperature-stability experiments. Participants are not directly involved as the study retrospectively analyzes existing images. Researchers will evaluate diagnostic accuracy, BI-RADS classification accuracy, agreement with expert consensus, and other performance metrics at study completion, approximately 12 months after starting. The study also monitors model robustness and safety through specific tests. The total study duration extends from March 2026 to March 2027.
CONDITIONS
Brief Title
Construction of a Benchmark for Breast Ultrasound AI Interpretation and Performance Evaluation of Multimodal AI Models
Who Can Participate
Age: 18Years - 75Years
FEMALE
Healthy Volunteers
Eligibility Criteria
You may qualify if you...
- Breast ultrasound B-mode grayscale images from institutional databases or approved open-access datasets
- Images must have adequate quality for clinical diagnosis with clear visualization
- Pathological diagnosis confirmed for benign and malignant lesions, or normal breast status confirmed by an experienced senior radiologist
- Images fully de-identified with no personal identification information
You will not qualify if you...
- Severely degraded image quality preventing meaningful BI-RADS assessment
- Duplicate images from the same patient (only the most representative retained)
- Images containing residual personal identification information after de-identification
- Cases with unclear, disputed, or missing pathological diagnosis
- Non-B-mode ultrasound images such as elastography, contrast-enhanced ultrasound, and Doppler imaging
AI-Screening
AI-Powered Screening
Complete this quick 3-step screening to check your eligibility
1
2
3
Your Study Journey
Screening
Duration - 2 to 4 weeks
Participants are screened for eligibility to participate in the trial.
No participant visits; eligibility is based on image dataset criteria.
Diagnostic Evaluation
Duration - Up to 12 months
Participants' breast ultrasound images are retrospectively evaluated by multiple AI systems and expert radiologists to assess diagnostic accuracy and BI-RADS classification.
No participant visits; evaluation is performed on de-identified images.
Long-term Monitoring
Duration - Up to 12 months
Ongoing analysis of AI model performance including out-of-distribution rejection tests and stability assessments.
No participant visits; monitoring conducted through data analysis.
Trial Site Locations
Total: 1 location
1
Peking Union Medical College Hospital
Beijing, China, 100730
Actively Recruiting
Loading map...
Research Team
Q
Qingli Zhu, MD
Y
Yinglan Wu, MD
How is the study designed?
Study Type
OBSERVATIONAL
Masking
N/A
Allocation
N/A
Model
N/A
Primary Purpose
N/A
Number of Arms
3
Similar Trials
Frequently Asked Questions
Have more questions? Get in touch with our team for quick support
Not the Right Trial for You?
Explore thousands of other clinical trials that might be a better match.
Sign up to get personalized trial recommendations delivered to your inbox.
Already have an account? Log in here
Published Research Related To This Trial
Performance of ChatGPT on a Radiology Board-style Examination: Insights into Current Strengths and Limitations.
Rajesh Bhayana, Satheesh Krishna, Robert R Bleakney
https://pubmed.ncbi.nlm.nih.gov/37191485The future landscape of large language models in medicine.
Jan Clusmann, Fiona R Kolbinger, Hannah Sophie Muti...
https://pubmed.ncbi.nlm.nih.gov/37816837Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations.
Laleh Seyyed-Kalantari, Haoran Zhang, Matthew B A McDermott...
https://pubmed.ncbi.nlm.nih.gov/34893776Foundation models for generalist medical artificial intelligence.
Michael Moor, Oishi Banerjee, Zahra Shakeri Hossein Abad...
https://pubmed.ncbi.nlm.nih.gov/37045921Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries.
Hyuna Sung, Jacques Ferlay, Rebecca L Siegel...
https://pubmed.ncbi.nlm.nih.gov/33538338Leveraging Large Language Models for Decision Support in Personalized Oncology.
Manuela Benary, Xing David Wang, Max Schmidt...
https://pubmed.ncbi.nlm.nih.gov/37976064Using a Large Language Model for Breast Imaging Reporting and Data System Classification and Malignancy Prediction to Enhance Breast Ultrasound Diagnosis: Retrospective Study.
Su Miaojiao, Liang Xia, Zeng Xian Tao...
https://pubmed.ncbi.nlm.nih.gov/40498674USFM: A universal ultrasound foundation model generalized to tasks and organs towards label efficient image analysis.
Jing Jiao, Jin Zhou, Xiaokang Li...
https://pubmed.ncbi.nlm.nih.gov/38788326Deep Learning-assisted Diagnosis of Breast Lesions on US Images: A Multivendor, Multicenter Study.
Huiling Xiang, Xi Wang, Min Xu...
https://pubmed.ncbi.nlm.nih.gov/37795135