Hardware Synchronization of Multiple Kinects and Microphones for 3D Audiovisual Spatiotemporal Data Capture.

We present an approach that uses linear timecode to temporally synchronize visual data captured at 30 fps from multiple Kinect v2 sensors and audio data captured from external microphones. Existing approaches suffer from reduced frame rate due to latencies in relaying synchronization signals over a network, have inaccuracies due to ambient sound contamination when using audio for synchronization, or show multi-frame delays when using networking protocols to synchronize capture computer clocks. We align audio and color frame times to a timecode signal injected using a custom-designed hardware board to the audio processing board of each Kinect. Our approach provides synchronized capture unaffected by ambient noise or network latencies, with maximum synchronization offsets within a single frame for any number of devices. We show 3D point cloud reconstruction results with audio for a variety of single-and multi-person interactions captured using four Kinect v2 sensors and two external microphones synchronized by our approach.