Eye and hand movements when playing a dynamic computer game (Pong)

Most eye movement studies investigate the behavior of a head-fixed observer looking at multiple repetitions of fixed trials on a computer monitor. Here we used a more naturalistic, complementary approach to investigate the interplay between eye and hand movements during natural sequential behavior in a dynamic scene. Participants played a modified version of the video game ‘Pong’ on an iPad by controlling a paddle via the touchscreen. Their task was to keep a linearly moving dot (ball) in the game by hitting it with the paddle to prevent it from passing. Gaze positions were recorded with a mobile Tobii Glasses 3 eye tracker and were mapped onto the tablet via computer vision algorithms. We manipulated the ball speed (slow vs. fast) and the shape of the paddles (bar vs. rhombus) in a blocked, counterbalanced order. Results of the overall playing performance (number of hits) revealed an impact of both, ball speed and shape of the paddle. For the faster speed and the rhombus shape participants’ performance decreased. Gaze data showed that participants tracked the ball with combinations of pursuit and saccades which adapted to the specific conditions and play time. Participants made more frequent saccades when interacting with faster moving targets. With increasing time playing, participants used more pursuit. Overall, the results indicate that participants adapt their eye movement strategies to the dynamic environment. There is a larger focus on tracking the ball under more difficult conditions. The increased proportions of pursuit over the course of the experiment presumably support a higher performance in the game.