Can Robotic Experimenters help improve HRI Experiments? An Experimental Study.

To evaluate the design and skills of a robot or an algorithm for robotics, human-robot interaction user studies need to be performed. Classically, these studies are conducted by human experimenters, requiring considerable effort, and introducing variability and potential human error. In this paper, we investigate the use of robots in support of HRI experiments. Robots can perform repeated tasks accurately, thereby reducing human effort and improving validity through reduction of error and variability between participants. To assess the potential for robot led HRI experiments, we ran an HRI experiment with two participant groups, one led by a human experimenter and another led mostly by a robot experimenter.We show that the replacement of several repetitive experiment tasks through robots is not only possible but beneficial: Trials performed by the robot experimenter had fewer errors and were more fluent. There was no statistically significant difference in participants' perception w.r.t. cognitive load, comfortability, enjoyment, safety, trust and understandability between both groups. To the best of our knowledge, this is the first comparison between robot-led and human-led HRI experiments. It suggests that using robot experimenters can be beneficial and should be considered.