Design and validation of a navigation system of multimodal medical images for neurosurgery based on mixed reality

Purpose: This paper aims to develop a navigation system based on mixed reality, which can display multimodal medical images in an immersive environment and help surgeons locate the target area and surrounding important tissues precisely.Methods: To be displayed properly in mixed reality, medical images are processed in this system. High-quality cerebral vessels and nerve fibers with proper colors are reconstructed and exported to mixed reality environment. Multimodal images and models are registered and fused, extracting their key information. The multiple processed images are fused with the real patient in the same coordinate system to guide the surgery.Results: The multimodal image system is designed and validated properly. In phantom experiments, the average error of preoperative registration is 1.003 mm and the standard deviation is 0.096 mm. The average proportion of well-registered areas is 94.9%. In patient experiments, the surgeons who participated in the experiments generally indicated that the system had excellent performance and great application prospect for neurosurgery.Conclusion: This article proposes a navigation system of multimodal images for neurosurgery based on mixed reality. Compared with other navigation methods, this system can help surgeons locate the target area and surrounding important tissues more precisely and rapidly.& COPY; 2023 The Authors. Published by Elsevier B.V. on behalf of Zhejiang University and Zhejiang University Press Co. Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).