Nuclei Detection Using Mixture Density Networks

  • Navid Alemi KoohababniEmail author
  • Mostafa Jahanifar
  • Ali Gooya
  • Nasir Rajpoot
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11046)


Nuclei detection is an important task in the histology domain as it is a main step toward further analysis such as cell counting, cell segmentation, study of cell connections, etc. This is a challenging task due to complex texture of histology image, variation in shape, and touching cells. To tackle these hurdles, many approaches have been proposed in the literature where deep learning methods stand on top in terms of performance. Hence, in this paper, we propose a novel framework for nuclei detection based on Mixture Density Networks (MDNs). These networks are suitable to map a single input to several possible outputs and we utilize this property to detect multiple seeds in a single image patch. A new modified form of a cost function is proposed for training and handling patches with missing nuclei. The probability maps of the nuclei in the individual patches are next combined to generate the final image-wide result. The experimental results show the state-of-the-art performance on complex colorectal adenocarcinoma dataset.


Mixture density network Histology Nuclei detection 


  1. 1.
    Grau, V., Mewes, A., Alcaniz, M., Kikinis, R., Warfield, S.K.: Improved watershed transform for medical image segmentation using prior information. IEEE Trans. Med. Imaging 23(4), 447–458 (2004)CrossRefGoogle Scholar
  2. 2.
    Quelhas, P., Marcuzzo, M., Mendonça, A.M., Campilho, A.: Cell nuclei and cytoplasm joint segmentation using the sliding band filter. IEEE Trans. Med. Imaging 29(8), 1463–1473 (2010)CrossRefGoogle Scholar
  3. 3.
    Schmitt, O., Hasse, M.: Radial symmetries based decomposition of cell clusters in binary and gray level images. Pattern Recognit. 41(6), 1905–1923 (2008)CrossRefGoogle Scholar
  4. 4.
    Parvin, B., Yang, Q., Han, J., Chang, H., Rydberg, B., Barcellos-Hoff, M.H.: Iterative voting for inference of structural saliency and characterization of subcellular events. IEEE Trans. Image Process. 16(3), 615–623 (2007)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Qi, X., Xing, F., Foran, D.J., Yang, L.: Robust segmentation of overlapping cells in histopathology specimens using parallel seed detection and repulsive level set. IEEE Trans. Biomed. Eng. 59(3), 754–765 (2012)CrossRefGoogle Scholar
  6. 6.
    Hafiane, A., Bunyak, F., Palaniappan, K.: Fuzzy clustering and active contours for histopathology image segmentation and nuclei detection. In: International Conference on Advanced Concepts for Intelligent Vision Systems, pp. 903–914. Springer, Berlin (2008)CrossRefGoogle Scholar
  7. 7.
    Akakin, H.C., et al.: Automated detection of cells from immunohistochemically-stained tissues: application to Ki-67 nuclei staining. In: Medical Imaging 2012: Computer-Aided Diagnosis. Volume 8315, International Society for Optics and Photonics (2012) 831503Google Scholar
  8. 8.
    Yang, L., Tuzel, O., Meer, P., Foran, D.J.: Automatic image analysis of histopathology specimens using concave vertex graph. In: International Conference on Medical Image Computing and Computer-Assisted Intervention,pp. 833–841. Springer, Berlin (2008)CrossRefGoogle Scholar
  9. 9.
    Jung, C., Kim, C.: Segmenting clustered nuclei using H-minima transform-based marker extraction and contour parameterization. IEEE Trans. Biomed. Eng. 57(10), 2600–2604 (2010)CrossRefGoogle Scholar
  10. 10.
    Thomas, R.M., John, J.: A review on cell detection and segmentation in microscopic images. In: 2017 International Conference on Circuit, Power and Computing Technologies (ICCPCT), pp. 1–5. IEEE (2017)Google Scholar
  11. 11.
    Cireşan, D.C., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Mitosis detection in breast cancer histology images with deep neural networks. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 411–418. Springer, Berlin (2013)CrossRefGoogle Scholar
  12. 12.
    Xie, Y., Xing, F., Shi, X., Kong, X., Su, H., Yang, L.: Efficient and robust cell detection: a structured regression approach. Med. Image Anal. 44, 245–254 (2018)CrossRefGoogle Scholar
  13. 13.
    Xu, J., Xiang, L., Liu, Q., Gilmore, H., Wu, J., Tang, J., Madabhushi, A.: Stacked sparse autoencoder (SSAE) for nuclei detection on breast cancer histopathology images. IEEE Trans. Med. Imaging 35(1), 119–130 (2016)CrossRefGoogle Scholar
  14. 14.
    Sirinukunwattana, K., Raza, S.E.A., Tsang, Y.W., Snead, D.R., Cree, I.A., Rajpoot, N.M.: Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images. IEEE Trans. Med. Imaging 35(5), 1196–1206 (2016)CrossRefGoogle Scholar
  15. 15.
    Bishop, C.M.: Mixture density networks. Technical report. Citeseer (1994)Google Scholar
  16. 16.
    Xie, W., Noble, J.A., Zisserman, A.: Microscopy cell counting and detection with fully convolutional regression networks. Comput. Methods Biomech. Biomed. Eng.: Imaging Vis. 6(3), 283–292 (2018)Google Scholar
  17. 17.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Navid Alemi Koohababni
    • 1
    • 4
    Email author
  • Mostafa Jahanifar
    • 2
  • Ali Gooya
    • 3
  • Nasir Rajpoot
    • 1
    • 4
  1. 1.Department of Computer ScienceUniversity of WarwickCoventryUK
  2. 2.Department of Biomedical EngineeringTarbiat Modares UniversityTehranIran
  3. 3.Department of Electronic and Electrical EngineeringUniversity of SheffieldSheffieldUK
  4. 4.Alan Turing InstituteLondonUK

Personalised recommendations