Augmented reality for human–swarm interaction in a swarm-robotic chemistry simulation

We present a novel augmented reality (AR) framework to show relevant information about swarm dynamics to a user in the absence of markers by using blinking frequency to distinguish between groups in the swarm. In order to distinguish between groups, clusters of the same group are identified by blinking at a specific time interval that is distinct from the time interval at which their neighbors blink. The problem is thus to find blinking sequences that are distinct for each group with respect to the group’s neighbors. Selecting an appropriate sequence is an instance of the distributed graph coloring problem, which can be solved in O(log (n)) time with n being the number of robots involved. We demonstrate our approach using a swarm chemistry simulation in which robots simulate individual atoms that form molecules following the rules of chemistry. An AR display is then used to display information about the internal state of individual swarm members as well as their topological relationship, corresponding to molecular bonds in a context that uses robot swarms to teach basic chemistry concepts.