Deep Learning-Based 3 D Freehand Ultrasound Reconstruction with Inertial Measurement Units

This work aims at reconstructing 3D ultrasound volumes from sequences of freehand images acquired with standard 2D probes without any expensive or cumbersome external tracking hardware. We extend our previous method based on deep learning to integrate and also learn from measurements of an inertial measurement unit (IMU). Our system is evaluated on a dataset of 600 in vivo ultrasound sweeps, yielding accurate reconstructions with a median normalized drift of 5.2% even on long sweeps with complex trajectories, hence paving the way towards translation into clinical routine. This extended abstract is adapted from a journal paper to appear [4]. I1 I2 I3 2D Ultrasound Clip 3D Ultrasound Reconstruction I1 I2 Motion Estimation Neural Network IMU Our Method Frame-to-Frame Motion Estimation Figure 1: Our system turns 2D standard ultrasound clips into 3D volumes. The method is based on a frame-to-frame motion estimation performed by a neural network with the help of an IMU.