Iterative decomposition of visuomotor, device and cognitive variance in large scale online cognitive test data

Abstract Online cognitive assessment technologies are gaining traction as scalable and cost-effective alternatives to traditional supervised testing. However, variability in peoples’ home devices, and their visual and motor abilities, confound the cognitive specificity of online task performance scores. To address these limitations, we develop IDoCT (Iterative Decomposition of Cognitive Tasks), a novel method for estimating cognitive abilities and trial-difficulty scales from task performance timecourses in a data-driven manner while accounting for device and visuomotor latencies, and speed-accuracy trade-offs. IDoCT can operate with any computerised task that manipulates cognitive difficulty across trials. Using data from 388,757 adults across 12 online cognitive tasks, we show that IDoCT successfully dissociates cognitive abilities from visuomotor response latencies. The resultant cognitive scores exhibit superior psychometric structure and associations with demographic factors while being insensitive to testing device. We propose that IDoCT can enhance the precision of online cognitive assessments for diverse clinical and research applications.