Neural Modeling of Eukaryotic Cell Ultrastructure Obtained from 3D Electron Microscopy

Electron microscopy (EM) allows biologists to create 3D images of eukaryotic cells and their constituent nanoscale ultrastructure. The ultrastructural environment is a complex network of organelles each of which performs specialized biochemical functions. EM images of cellular or tissue samples can be segmented to produce quantitative 3D models of the morphology and distribution of organelles, creating a semantically rich visualization that improves our understanding of cellular and systems biology. However, this is a tedious process when done by hand, and manual segmentation does not scale to the large sample volumes that modern EM platforms are capable of imaging. Recent studies have explored deep learning-based semantic segmentation for cellular EM datasets, and here we extend those methods to solve the broader problem of 3D structural model generation. Structural model generation addresses the needs of the end-user structural biologist more directly than semantic segmentation alone, and it provides a method for refining semantic segmentation predictions. We have created and validated an automated pipeline by applying this approach to blood platelets imaged with a serial block-face scanning electron microscope (SBF-SEM). Our approach combines neural instance and semantic segmentation modules to localize and classify objects within image regions, and to assign a geometric mesh to each region suitable for manipulation using common 3D rendering tools such as ImageJ, Blender, or Unity.