Real-time vessel segmentation and reconstruction for virtual fixtures for an active handheld microneurosurgical instrument

Complications related to vascular damage such as intra-operative bleeding may be avoided during neurosurgical procedures such as petroclival meningioma surgery. To address this and improve the patient’s safety, we designed a real-time blood vessel avoidance strategy that enables operation on deformable tissue during petroclival meningioma surgery using Micron, a handheld surgical robotic tool. We integrated real-time intra-operative blood vessel segmentation of brain vasculature using deep learning, with a 3D reconstruction algorithm to obtain the vessel point cloud in real time. We then implemented a virtual-fixture-based strategy that prevented Micron’s tooltip from entering a forbidden region around the vessel, thus avoiding damage to it. We achieved a median Dice similarity coefficient of 0.97, 0.86, 0.87 and 0.77 on datasets of phantom blood vessels, petrosal vein, internal carotid artery and superficial vessels, respectively. We conducted trials with deformable clay vessel phantoms, keeping the forbidden region 400 $$\mu $$ m outside and 400 $$\mu $$ m inside the vessel. Micron’s tip entered the forbidden region with a median penetration of just 8.84 $$\mu $$ m and 9.63 $$\mu $$ m, compared to 148.74 $$\mu $$ m and 117.17 $$\mu $$ m without our strategy, for the former and latter trials, respectively. Real-time control of Micron was achieved at 33.3 fps. We achieved improvements in real-time segmentation of brain vasculature from intra-operative images and showed that our approach works even on non-stationary vessel phantoms. The results suggest that by enabling precise, real-time control, we are one step closer to using Micron in real neurosurgical procedures.