A Cross-Domain Approach to Designing an Unobtrusive System to Assess Human State and Predict Upcoming Performance Deficits

Many work environments are fraught with highly variable demands on cognitive workload, fluctuating between periods of high operational demand to the point of cognitive overload, to long periods of low workload bordering on boredom. When cognitive workload is not in an optimal range at either end of the spectrum, it can be detrimental to situational awareness and operational readiness, resulting in impaired cognitive functioning (Yerkes and Dodson, 1908). An unobtrusive system to assess the state of the human operator (e.g., stress, cognitive workload) and predict upcoming performance deficits could warn operators when steps should be taken to augment cognitive readiness. This system would also be useful during testing and evaluation (T&E) when new tools and systems are being evaluated for operational use. T&E researchers could accurately evaluate the cognitive and physical demands of these new tools and systems, and the effects they will have on task performance and accuracy. In this paper, we describe an approach to designing such a system that is applicable across environments. First, a suite of sensors is used to perform real-time synchronous data collection in a robust and unobtrusive fashion, and provide a holistic assessment of operators. Second, the best combination of indicators of operator state is extracted, fused, and interpreted. Third, performance deficits are comprehensively predicted, optimizing the likelihood of mission success. Finally, the data are displayed in such a way that supports the information requirements of any user. The approach described here is one we have successfully used in several projects, including modeling cognitive workload in the context of high-tempo, physically demanding environments, and modeling individual and team workload, stress, engagement, and performance while working together on a computerized task. We believe this approach is widely applicable and useful across domains to dramatically improve the mission readiness of human operators, and will improve the design and development of tools available to assist the operator in carrying out mission objectives. A system designed using this approach could enable crew to be aware of impending deficits to aid in augmenting mission performance, and will enable more effective T&E by measuring workload in response to new tools and systems while they are being designed and developed, rather than once they are deployed.