Extended machine vision-control capabilities using digital holography and transformer neural networks

We develop a novel high‐profile application of machine learning techniques by elevating digital holography and sensing in robotics to a new level. The extraction of unknown metrics such as focusing distance and in plane positioning without full image restoration from digital holograms is performed by pre‐processing approach in space‐domain and/or in Fourier‐domain, including real‐time constraints. Measuring a single hologram, we successfully determine the axial distance of a complex object to the 10x microscope objective over a range of 100 µm with an accuracy of 1.25 µm. We apply a machine learning technique to the hologram to speed up tracking in the plane of the pseudo-periodic target position up to several tens of frames per second (fps). Such high frame rates enable real-time processing in many different application scenarios.