A Multi-body Simulation Framework for Live Motion Tracking and Analysis within the Unity Environment

Synthesizing motion de novo for dynamic virtual organisms in a physics-based, virtual environment is presently intractable. Multi-body simulation development within current software systems is often cumbersome and interfacing third-party hardware (i.e. VR equipment, IMUs, haptic systems) is usually not straightforward for the average developer. Developing a simulation framework within a widely supported engine, such as Unity or Unreal Engine, would be much more practical because the resulting products can be designed to be scalable, highly customizable, easy-to-use, and open source. In this paper, we present a Unity-based multi-body simulation framework that can be used to simultaneously track and analyze human motion. The framework incorporates the musculoskeletal models derived from OpenSim and utilizes runtime-derived, closed-form definitions for muscle routing kinematics and kinematic Jacobians. The framework can be used to simulate multiple dynamic models simultaneously. Also, being integrated with the Unity environment, the framework naturally provides intuitive customization and third-party system integration. To demonstrate the performance and motion analysis capabilities of the framework, the paper also presents a live test simulation in which the muscular effort values of three different bicep curl forms were calculated while simultaneously tracking the tasks' motion at 100 Hz. One prospective use of the proposed framework is virtual reality and haptic-augmented motion training. The live tracking and motion analysis capabilities of the framework will allow immediate and effective dynamic feedback towards task performance optimization.