Convolutional Neural Networks on Eye Tracking Trajectories Classify Patients with Spatial Neglect

Background and Objective Eye-movement trajectories are rich behavioral data, providing a window on how the brain processes information. We address the challenge of characterizing signs of visuo-spatial neglect from saccadic eye trajectories recorded in brain-damaged patients with spatial neglect as well as in healthy controls during a visual search task. Methods We establish a standardized preprocessing pipeline adaptable to other task-based eye-tracker measurements. We use a deep convolutional network, a very successful type of neural network architecture in many computer vision applications, including medical diagnosis systems, to automatically analyze eye trajectories. Results Our algorithm can classify brain-damaged patients vs. healthy individuals with an accuracy of 86±5%. Moreover, the algorithm scores correlate with the degree of severity of neglect signs estimated with standardized paper-and-pencil test and with white matter tracts impairment via Diffusion Tensor Imaging (DTI). Interestingly, the latter showed a clear correlation with the third branch of the superior longitudinal fasciculus (SLF), especially damaged in neglect. Conclusions The study introduces a new classification method to analyze eyes trajectories in patients with neglect syndrome. The method can likely be applied to other types of neurological diseases opening to the possibility of new computer-aided, precise, sensitive and non-invasive diagnosing tools. Highlights ### Competing Interest Statement The authors have declared no competing interest. ### Funding Statement B.F., A.M., received financial support for this work from the Fondation Asile des Aveugles (grant #232933 to M.M.M.), a grantor advised by Carigest SA (#232920 to M.M.M.), as well as the Swiss National Science Foundation (grants #169206 to M.M.M.). A.B. acknowledges the support of Swiss National Science Foundation to Armin Schnider under grant 32003B-175472 and to Radek Ptak under grant 32003B-184702. The work of P.B. and P.P. is supported by Agence Nationale de la Recherche through ANR-16-CE37-0005 and ANR-10-IAIHU-06. F.A. acknowledges the Center for Brains, Minds and Machines (CBMM), funded by NSF STC award CCF-1231216 and the financial support of the AFOSR projects FA9550-17-1-0390 and BAA-AFRL-AFOSR-2016-0007 (European Office of Aerospace Research and Development), and the EU H2020-MSCA-RISE project NoMADS - DLV-777826. ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: The dataset used in this study was reviewed by the INSERM ethical committee and received the approval of an Institutional Review Board (CPP Ile de France 1) I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable. Yes The dataset is detailed, analysed and discussed in In the current work we only used the dateset for supplementary automatized analyses.