DR 3.3: Tailored Shared Situation Awareness Inter- faces

This thesis describes the design and evaluation of personalized situation awareness support (role and user) by showing a labeled grid with highlighted explored areas, assigned area per operator and safe/ unsafe areas in TDS. Relation to WP This thesis contributes to T3.3 Sentient dialogue for tailored SA, since the design of situation awareness support is tailored to specific roles and taking into account task load of the user. Availability Unrestricted. 2.2 S. Speckens (2017), “Situation Awareness enhanced tactical display system features for first and successive USAR sorties” Bibliography S. Speckens (2017), “Situation Awareness enhanced tactical display system features for first and successive USAR sorties”. MSc thesis, Department of Industrial Engineering & Innovation Sciences, Eindhoven University of Technology, The Netherlands. Abstract This thesis describes the influence of the presentation in the TDS of timeliness of robot and robot-camera direction and location trace on confidence, accuracy, timeliness and workload of a UGV operator.This thesis describes the influence of the presentation in the TDS of timeliness of robot and robot-camera direction and location trace on confidence, accuracy, timeliness and workload of a UGV operator. Relation to WP This thesis contributes to T3.3 Sentient dialogue for tailored SA, since the design of situation awareness support is tailored to specific roles and taking into account task load of the user. Availability Unrestricted. EU FP7 TRADR (ICT-60963) 22 Tailored Shared Situation Awareness Interfaces Pahlke et al. 2.3 N.J.J.M. Smets, M.A. Neerincx, C.M. Jonker, F. Baberg (2017), “Ontology-based situation awareness support for shared control” Bibliography N.J.J.M. Smets, M.A. Neerincx, C.M. Jonker, F. Baberg (2017), “Ontology-based situation awareness support for shared control”. Extended abstract HRI 2017 Abstract Situation Awareness (SA) during tele-operation in robot-assisted disaster management has a major impact on the effectiveness and efficiency. Data perceived by the human and robot agents should be processed and shared in such a way that these agents can understand and direct the other agent’s behaviors. E.g., for safe and effective tele-operation, the human (team leader and/or operator) and robot need to be aware of (1) the state, location, position and movement of the robot platform and its arms, and (2) the state of robot’s environment (such as obstacles, etc.). This paper presents an SA-ontology that formalizes the effects of SA-components on the shared control performance. It is based on literature research, interviews with subject matter experts and a field test during a disaster management exercise. The SA-ontology captured all information needs for the teleoperation, and provided further requirements for SA-support functions.Situation Awareness (SA) during tele-operation in robot-assisted disaster management has a major impact on the effectiveness and efficiency. Data perceived by the human and robot agents should be processed and shared in such a way that these agents can understand and direct the other agent’s behaviors. E.g., for safe and effective tele-operation, the human (team leader and/or operator) and robot need to be aware of (1) the state, location, position and movement of the robot platform and its arms, and (2) the state of robot’s environment (such as obstacles, etc.). This paper presents an SA-ontology that formalizes the effects of SA-components on the shared control performance. It is based on literature research, interviews with subject matter experts and a field test during a disaster management exercise. The SA-ontology captured all information needs for the teleoperation, and provided further requirements for SA-support functions. Relation to WP This abstract contributes to T3.3 Sentient dialogue for tailored SA, the SA-ontology formalizes the effects of SA components on the shared control performance.TBD Availability Restricted. Not included in the public version of this deliverable. 2.4 Sengupta, S., Nabizadeh, N., Racioppa, S., Kruijff-Korbayová, I. (2017), “Speech Interaction Processing in TRADR” Bibliography Sengupta, S., Nabizadeh, N., Racioppa, S., Kruijff-Korbayová, I. (2017), “Speech Interaction Processing in TRADR”. Unpublished technical report. Abstract This report describes the usage and analysis of speech as an input to (a) interact with the TDS system and (b) keep track of tasks performed by the team in the context of a rescue mission with the TRADR system. For (a) we describe the design of the multimodal interaction functionality developed so far. It enables the users to input information about points of interest by a combination of mouse clicks and speech and refine it in short dialogues. For (b) we examined the human-human conversations of the TRADR missions to understand the communicative acts that needThis report describes the usage and analysis of speech as an input to (a) interact with the TDS system and (b) keep track of tasks performed by the team in the context of a rescue mission with the TRADR system. For (a) we describe the design of the multimodal interaction functionality developed so far. It enables the users to input information about points of interest by a combination of mouse clicks and speech and refine it in short dialogues. For (b) we examined the human-human conversations of the TRADR missions to understand the communicative acts that need EU FP7 TRADR (ICT-60963) 23 Tailored Shared Situation Awareness Interfaces Pahlke et al. to be modeled in order to track the tasks assigned in the team. The report also describes the integration of a cloud-based speech engine (Nuance Mix NLU ASR) to improve the accuracy of speech recognition, and presents the results of an evaluation of this ASR with the audio data collected from the previous years. Relation to WP This technical report describes enhancements made to the speech processing framework which contribute to improving Joint Situation Awareness. Availability Restricted. Not included in the public version of this deliverable. EU FP7 TRADR (ICT-60963) 24