DIFF$$\cdot $$3: A Latent Diffusion Model for the Generation of Synthetic 3D Echocardiographic Images and Corresponding Labels

Large amounts of labelled data are typically needed to develop robust deep learning methods for medical image analysis. However, issues related to the high costs of acquisition, time-consuming analysis, and patient privacy, have limited the number of publicly available datasets. Recently, latent diffusion models have been employed to generate synthetic data in several fields. Compared to other imaging modalities, the manipulation of 3D echocardiograms is particularly challenging due to the higher dimensionality and complex noise characteristics, and lack of objective ground truth. We present DIFF $$\cdot $$ 3, a latent diffusion model for synthesizing realistic 3D echocardiograms with high-quality labels from matching cardiovascular magnetic resonance imaging (CMR) scans. Using in vivo 3D echocardiograms from 134 participants and corresponding registered labels derived from CMR, source images and labels are initially compressed by a variational autoencoder, followed by diffusion in the latent space. Synthetic datasets were subsequently generated by randomly sampling from the latent distribution, and evaluated in terms of fidelity and diversity. DIFF $$\cdot $$ 3 may provide an effective and more efficient means of generating labelled 3D echocardiograms to supplement real patient data.