Development of an eye-tracking system based on a deep learning model to assess executive function in patients with mental illnesses

Abstract Patients with mental illnesses, particularly psychosis and obsessive-compulsive disorder (OCD), frequently exhibit deficits in executive function and visuospatial memory. Traditional assessments, such as the Rey‒Osterrieth Complex Figure Test (RCFT), performed in clinical settings require time and effort. This study aimed to develop a deep learning model using the RCFT based on eye tracking to detect impaired executive function during visuospatial memory encoding in patients with mental illnesses. In 96 patients with first-episode psychosis, 49 with clinical high risk for psychosis, 104 with OCD, and 159 healthy controls, eye movements were recorded during a 3-minute RCFT figure memorization, and organization and immediate recall scores were obtained. These scores, along with the fixation points indicating eye-focused locations in the figure, were used to train a Long Short-Term Memory + Attention model for detecting impaired executive function and visuospatial memory. The model distinguished between normal and impaired executive function with an F1 score of 83.5% and identified visuospatial memory deficits with an F1 score of 80.7%, regardless of psychiatric diagnosis. These findings suggested that this eye-tracking-based deep learning model can directly and rapidly identify impaired executive function during visuospatial memory encoding, with potential applications in various psychiatric and neurological disorders.