KEY TAKEAWAYS
- The study aimed to improve the prediction accuracy of complications in patients with lung cancer following proton therapy using LLMs and meta-analysis.
- Researchers found that combining LLMs with meta-analysis improves efficiency in evaluating literature on complications.
Accurately predicting complications after proton therapy for lung cancer is crucial for treatment planning and patient care.
Pei-Ju Chao and the team aimed to enhance the prediction of post-proton therapy complications in patients with lung cancer using large language models (LLMs) and meta-analysis.
Researchers conducted a systematic review, sourcing studies from Web of Science, PubMed, and Scopus. Inclusion and exclusion criteria were applied, and data was managed using EndNote X20. Meta-analysis, heterogeneity assessment (Cochran’s Q test, I2 statistics), and subgroup analyses were performed.
Quality and bias assessment utilized the PROBAST tool and LLMs (ChatGPT-4, Llama2-13b, Llama3-8b). Evaluation metrics included AUC, accuracy, precision, recall, F1 score, and time efficiency (WPM).
Meta-analysis showed an overall effect size of 0.78 for model predictions with high heterogeneity (I2 = 72.88%, P< 0.001). Subgroup analysis (radiation-induced esophagitis and pneumonitis) indicated predictive effect sizes of 0.79 and 0.77, respectively, with no significant model differences (I2 = 0%).
ChatGPT-4 achieved the highest accuracy (90%) surpassing Llama3 and Llama2 (44% to 62%). LLM evaluations were 3229 times faster than manual assessments. Risk assessment by ChatGPT-4 was robust, identifying nine high-risk, three low-risk, and one unknown-risk study.
The study confirmed that combining LLMs and meta-analysis significantly improved literature evaluation efficiency, reduced assessment time, and showed no significant model differences in predicting post-proton therapy complications in patients with lung cancer.
The funding organization is not mentioned in the source link.
Source: https://pubmed.ncbi.nlm.nih.gov/39307562/
Chao PJ, Chang CH, Wu JJ, et al. (2024). “Improving Prediction of Complications Post-Proton Therapy in Lung Cancer Using Large Language Models and Meta-Analysis.” Cancer Control. 2024;31:10732748241286749. doi:10.1177/10732748241286749
