Text dialogue analysis Based ChatGPT for Primary Screening of Mild Cognitive Impairment

Background AI models tailored to diagnose cognitive impairment have shown excellent results. However, it is unclear whether large linguistic models can rival specialized models by text alone. Objectives We would explore the effectiveness of ChatGPT for primary screening of mild cognitive impairment (MCI) and standardize the design steps and components of the prompt. Methods We obtained 174 participants from the DementiaBank screening and classified 70% of them into the training set and 30% into the test set. Three dimensions of variables were incorporated, including: vocabulary, syntax and grammar, and semantics. These variables were generated from published studies and statistical analyses. We used R 4.3.0. for the analysis of variables and diagnostic indicators. Results The final variables included by published studies included: word frequency and word ratio, phrase frequency and phrase ratio, lexical complexity, syntactic complexity, grammatical components, semantic density, and semantic coherence; variables included in the analysis included: tip-of-the-tongue phenomenon (P < 0.001), difficulty with complex ideas (P < 0.001), and memory issues (P < 0.001). The final GPT4 model achieved the sensitivity (SEN) of 0.8636, specificity (SPE) of 0.9487 and area under the curve (AUC) of 0.9062 on the training set; on the test set, the SEN, SPE and AUC reached 0.7727, 0.8333 and 0.8030, respectively. The prompt consisted of five main parts, including character setting, scoring system setting, indicator setting, output setting, and explanatory information setting. Conclusion ChatGPT was effective in primary screening of participants with possible MCI. Improved standardization of prompts by professional clinicians would also improve the performance of the model. It is important to note that ChatGPT is not a substitute for a clinician making a diagnosis. ### Competing Interest Statement The authors have declared no competing interest. ### Funding Statement This study did not receive any funding ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: We included a total of 174 participants from the DementiaBank English Protocol Delaware Corpus and the DementiaBank English Pitt Corpus. Lanzi, A. M., Saylor, A. K., Fromm, D., Liu, H., MacWhinney, B., & Cohen, M. (2023). DementiaBank: Theoretical rationale, protocol, and illustrative analyses. American Journal of Speech-Language Pathology. https://dementia.talkbank.org/access/English/Protocol/Delaware.html Becker, J. T., Boller, F., Lopez, O. L., Saxton, J., & McGonigle, K. L. (1994). The natural history of Alzheimer's disease: description of study cohort and accuracy of diagnosis. Archives of Neurology, 51(6), 585-594. https://dementia.talkbank.org/access/English/Pitt.html I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable. Yes All data produced in the present study are available upon reasonable request to the authors