A Two-Stage U-Net Algorithm for Segmentation of Nuclei in H&E-Stained Tissues

Flowchart of the proposed algorithm

Mahbod A., Schaefer G., Ellinger I., Ecker R., Smedby Ö., Wang C. (2019) In: Reyes-Aldasoro C., Janowczyk A., Veta M., Bankhead P., Sirinukunwattana K. (eds) Digital Pathology. ECDP 2019. Lecture Notes in Computer Science, vol 11435. Springer, Cham

About the book „Digital Pathology“

This book constitutes the refereed proceedings of the 15^th European Congress on Digital Pathology,  ECDP 2019, held in Warwick, UK in April 2019. The 21 full papers presented in this volume were carefully reviewed and selected from 30 submissions. The congress theme was to accelerate clinical deployment, with a focus on computational pathology and leveraging the power of big data and artificial intelligence to bridge the gaps between research, development, and clinical uptake.

About the paper

Nuclei segmentation is an important but challenging task in the analysis of hematoxylin and eosin (H&E)-stained tissue sections. While various segmentation methods have been proposed, machine learning-based algorithms and in particular deep learning-based models have been shown to deliver better segmentation performance. In this work, we propose a novel approach to segment touching nuclei in H&E-stained microscopic images using U-Net-based models in two sequential stages (see Fig.1). In the first stage, we perform semantic segmentation using a classification U-Net that separates nuclei from the background. In the second stage, the distance map of each nucleus is created using a regression U-Net. The final instance segmentation masks are then created using a watershed algorithm based on the distance maps. Evaluated on a publicly available dataset containing images from various human organs, the proposed algorithm achieves an average aggregate Jaccard index of 56.87%, outperforming several state-of-the-art algorithms applied on the same dataset.

With his proposed Two-Stage U-Net Algorithm for Segmentation of Nuclei in H&E-Stained Tissues, Amirreza Mahbod participated in the MoNuSeg 2018 Challenge. Due to his very good ranking (he was ranked place 10 out of 32), his work became part of the following publication that summarized the results of MoNuSeg 2018 Challenge
Kumar N, et al. A Multi-organ Nucleus Segmentation Challenge. IEEE Trans Med Imaging. 2019 Oct 23. doi: 10.1109/TMI.2019.2947628. [Epub ahead of print] PubMed PMID: 31647422.

Generalized nucleus segmentation techniques can contribute greatly to reducing the time to develop and validate visual biomarkers for new digital pathology datasets. This paper summarizes the results of MoNuSeg 2018 Challenge whose objective was to develop generalizable nuclei segmentation techniques in digital pathology. The challenge was an official satellite event of the MICCAI 2018 conference in which 32 teams with more than 80 participants from geographically diverse institutes participated. Contestants were given a training set with 30 images from seven organs with annotations of 21,623 individual nuclei. A test dataset with 14 images taken from seven organs, including two organs that did not appear in the training set was released without annotations. Entries were evaluated based on average aggregated Jaccard index (AJI) on the test set to prioritize accurate instance segmentation as opposed to mere semantic segmentation. More than half the teams that completed the challenge outperformed a previous baseline 1. Among the trends observed that contributed to increased accuracy were the use of color normalization as well as heavy data augmentation. Additionally, fully convolutional networks inspired by variants of U-Net 2, FCN 3, and Mask-RCNN 4 were popularly used, typically based on ResNet 5 or VGG 6 base architectures. Watershed segmentation on predicted semantic segmentation maps was a popular post-processing strategy. Several of the top techniques compared favorably to an individual human annotator and can be used with confidence for nuclear morphometrics.