Artificial intelligence (AI) is increasingly integrated into healthcare, yet its role in thoracic surgery decisions remains uncertain. A recent study examined how well a large language model (LLM) agreed with human surgeons’ clinical recommendations. Researchers focused on outpatient consultations within a tertiary hospital during a single week.
The study compared Scholar GPT, based on GPT-4, with board-certified thoracic surgeons. Two independent specialists assessed diagnostic and treatment alignment using a six-point scale from zero to five. Statistical tools such as t-tests, ANOVA, and chi-square analyses supported the evaluation, though no prior sample-size estimation was made.
In total, 81 consultations were reviewed, producing a mean concordance score of 3.67 ± 1.17. Strong agreement, defined as scores between four and five, appeared in 56.8 percent of cases. This level of concordance was more frequent in oncological issues, particularly mediastinal and pleural tumors.
AI recommendations were less consistent in challenging or functional cases, such as metastatic lung disease and thoracic outlet syndrome. However, the study found no significant differences based on consultation type or format. These findings indicate that AI can align closely with surgeons in structured cases but struggles with complex conditions.
Researchers concluded that while AI tools like Scholar GPT can enhance efficiency in outpatient workflows, they should not replace expert judgment. The authors cautioned that their results are preliminary due to the study’s limited duration and single-center scope.