Exploring Gestural Agency in Collaborative and Embodied Rates of Change Simulations

This chapter describes the theories, design, and analysis applied to a pair of collaborative mixed-reality simulations addressing different scientific content but linked by the crosscutting concept of “rates of change”. Body tracking technologies allowed users to interact with these simulations using one of two conventions: through gestures chosen by participants, or a prescribed gesture representing the slope of a line using one’s arms. The two simulations explore the relationship between inputs and outputs in the context of population ecology and climate change systems. Participant pairs were randomly assigned to one of 4 treatment groups for gestural convention (learner-defined or prescribed) and simulation order (population ecology or climate change first). Knowledge and engagement surveys found a significant overall increase in individual knowledge, but no significant difference across the four treatments. This motivated deeper analysis of the dialog and actions of the participant pairs as they engaged with the system, where it was found that the order of simulation exposure produced a shift in the ratio between collaborative directives and simulation behavior dialog. In addition, the gestures recorded in the learner-defined gesture group were also leveraged in un-expected ways, such as in facilitating synchronization between partners.