Single-Pixel Imaging of Spatio-Temporal Flows using Differentiable Latent Dynamics

Imaging dynamic spatio-temporal flows typically requires high-speed, high-resolution sensors that may be physically or economically prohibitive. Single-pixel imaging (SPI) has emerged as a low-cost acquisition technique where light from a scene is projected through a spatial light modulator onto a single photodiode with a high temporal acquisition rate. The scene is then reconstructed from the temporal samples using computational techniques that leverage prior assumptions on the scene structure. In this paper, we propose to image spatio-temporal flows from incomplete measurements by leveraging scene priors in the form of a reduced-order model (ROM) of the dynamics learned from training data examples. By combining SPI acquisition with the ROM prior implemented as a neural ordinary differential equation, we achieve high-quality image sequence reconstruction with significantly reduced data requirements. Specifically, our approach achieves similar performance levels to leading methods despite using one to two orders of magnitude fewer samples. We demonstrate superior reconstruction at low sampling rates for simulated trajectories governed by Burgers' equation, Kolmogorov flow, and turbulent plumes emulating gas leaks.