StarDist Image Segmentation Improves Circulating Tumor Cell Detection

Simple Summary Automated enumeration of circulating tumor cells (CTC) from immunofluorescence images starts with a selection of areas containing potential CTC. The CellSearch system has a built-in selection algorithm that has been observed to fail in samples with high cell density, thereby underestimating the true CTC load. We evaluated the deep learning method StarDist for the selection of possible CTC. In whole blood sample images, StarDist recovered 99.95% of CTC detected by CellSearch and segmented 10% additional CTC. In diagnostic leukapheresis (DLA) samples, StarDist segmented 20% additional CTC and performed well, whereas CellSearch had serious failures in 9% of samples. After a CellSearch-processed circulating tumor cell (CTC) sample is imaged, a segmentation algorithm selects nucleic acid positive (DAPI+), cytokeratin-phycoerythrin expressing (CK-PE+) events for further review by an operator. Failures in this segmentation can result in missed CTCs. The CellSearch segmentation algorithm was not designed to handle samples with high cell density, such as diagnostic leukapheresis (DLA) samples. Here, we evaluate deep-learning-based segmentation method StarDist as an alternative to the CellSearch segmentation. CellSearch image archives from 533 whole blood samples and 601 DLA samples were segmented using CellSearch and StarDist and inspected visually. In 442 blood samples from cancer patients, StarDist segmented 99.95% of CTC segmented by CellSearch, produced good outlines for 98.3% of these CTC, and segmented 10% more CTC than CellSearch. Visual inspection of the segmentations of DLA images showed that StarDist continues to perform well when the cell density is very high, whereas CellSearch failed and generated extremely large segmentations (up to 52% of the sample surface). Moreover, in a detailed examination of seven DLA samples, StarDist segmented 20% more CTC than CellSearch. Segmentation is a critical first step for CTC enumeration in dense samples and StarDist segmentation convincingly outperformed CellSearch segmentation.