Responses From ChatGPT-4 Show Limited Correlation With Expert Consensus Statement on Anterior Shoulder Instability
Arthroscopy, Sports Medicine, and Rehabilitation(2024)
摘要
Purpose
The purpose of this study was to compare the similarity of answers provided by GPT-4 to those of a consensus statement on diagnosis, nonoperative management, and Bankart repair in anterior shoulder instability (ASI).
Methods
An expert consensus statement on ASI published by Hurley et al. in 2022 was reviewed and questions laid out to the expert panel were extracted. GPT-4, the subscription version of ChatGPT, was queried using the same set of questions. Answers provided by GPT-4 were compared to those of the expert panel and subjectively rated for similarity by two experienced shoulder surgeons. GPT-4 was then used to rate the similarity of its own responses to the consensus statement, classifying them as low, medium, or high. Rates of similarity as classified by the shoulder surgeons and GPT-4 were then compared and interobserver reliability calculated using weighted κ scores.
Results
The degree of similarity between responses of GPT-4 and the ASI consensus statement, as defined by shoulder surgeons, was high in 25.8%, medium in 45.2%, and low 29% of questions.GPT-4, assessed similarity as high in 48.3%, medium in 41.9%, and low 9.7% of questions.Surgeons and GPT-4 reached consensus on the classification of 18 questions (58.1%) and disagreement on 13 questions (41.9%).
Conclusion
The responses generated by artificial intelligence exhibit limited correlation with an expert statement on the diagnosis and treatment of ASI.
更多查看译文
关键词
Anterior Shoulder Instability,ChatGPT,artificial intelligence,Consensus Statement,Delphi consensus process
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要