Robust Segmentation of Overlapping Cells in Cervical Cytology Using Light Convolution Neural Network

  • Shusong Xu
  • Chen Sang
  • Yulan Jin
  • Tao Wan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11307)


Automated segmentation of cells in cervical cytology images poses a great challenge due to the presence of fuzzy and overlapping cells, noisy background, and poor cytoplasmic contrast. We present an improved method for segmenting nuclei and cytoplasm from a cluster of cervical cells using convolutional network and fast multi-cell labeling. A light convolutional neural network (CNN) model is employed to generate nuclei candidates, which can serve as accurate initializations for the subsequent level set segmentation and provide a priori knowledge for the cytoplasm segmentation. A fast multi-cell labeling method based on the superpixel map is devised to roughly segment clumped and inhomogeneous cytoplasm before applying a cell boundary refinement approach. A shape constraint in conjunction with boundary and region information drive a level set formulation to perform a robust cell segmentation. The qualitative and quantitative evaluations demonstrated that the presented cellular segmentation method is effective and efficient.


Nuclei detection Cytoplasm segmentation Cervical cytology Convolutional Neural Network Multi-cell labeling 


  1. 1.
    Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Susstrunk, S.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34, 2274–2282 (2012)CrossRefGoogle Scholar
  2. 2.
    Hinton, G.E.: A practical guide to training restricted Boltzmann machines. In: Montavon, G., Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade. LNCS, vol. 7700, pp. 599–619. Springer, Heidelberg (2012). Scholar
  3. 3.
    Jing, J., Wan, T., Cao, J., Qin, Z.: An improved hybrid active contour model for nuclear segmentation on breast cancer histopathology. In: IEEE International Symposium on Biomedical Imaging, pp. 1155–1158 (2016)Google Scholar
  4. 4.
    Kong, H., Akakin, H., Sarma, S.: A generalized Laplacian of Gaussian filter for blob detection and its applications. IEEE Trans. Cybern. 43(6), 1719–1733 (2013)CrossRefGoogle Scholar
  5. 5.
    Lee, H., Kim, J.: Segmentation of overlapping cervical cells in microscopic images with superpixel partitioning and cell-wise contour refinement. In: the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1367–1373 (2016)Google Scholar
  6. 6.
    Li, C., Xu, C., Gui, C., Fox, M.: Distance regularized level set evolution and its application to image segmentation. IEEE Trans. Image Process. 19(12), 3243–3254 (2010)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Lu, Z., Carneiro, G., Bradley, A.: An improved joint optimization of multiple level set functions for the segmentation of overlapping cervical cells. IEEE Trans. Image Process. 24(4), 1261–1272 (2015)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Lu, Z., et al.: Evaluation of three algorithms for the segmentation of overlapping cervical cells. IEEE J. Biomed. Health Inform. 21, 441–450 (2017)Google Scholar
  9. 9.
    McGuire, S.: World cancer report 2014 Geneva, Switzerland: world health organizaiton, international agency for research on cancer. Adv. Nutr. 7(2), 418–419 (2016)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Phoulady, H., Goldgof, D., Hall, L., Mouton, P.: A framework for nucleus and overlapping cytoplasm segmentation in cervical cytology extended depth of field and volume images. Comput. Med. Imaging Graph. 59, 38–49 (2017)CrossRefGoogle Scholar
  11. 11.
    Rousson, M., Paragios, N.: Shape priors for level set representations. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2351, pp. 78–92. Springer, Heidelberg (2002). Scholar
  12. 12.
    Song, Y., et al.: Accurate cervical cell segmentation from overlapping clumps in pap smear images. IEEE Trans. Med. Imaging 36(1), 288–300 (2017)CrossRefGoogle Scholar
  13. 13.
    Song, Y., Zhang, L., Chen, S., Ni, D., Lei, B., Wang, T.: Accurate segmentation of cervical cytoplasm and nuclei based on multiscale convolutional network and graph partitioning. IEEE Trans. Biomed. Eng. 62(10), 2421–2433 (2015)CrossRefGoogle Scholar
  14. 14.
    Tareef, A., et al.: Automatic segmentation of overlapping cervical smear cells based on local distinctive features and guided shape deformation. Neurocomputing 221, 94–107 (2017)CrossRefGoogle Scholar
  15. 15.
    Wan, T., Cao, J., Chen, J., Qin, Z.: Automated grading of breast cancer histopathology using cascaded ensemble with combination of multi-level image features. Neurocomputing 229, 34–44 (2017)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.School of Biomedical Science and Medical Engineering, Beijing Advanced Innovation Centre for Biomedical EngineeringBeihang UniversityBeijingChina
  2. 2.Department of Pathology, Beijing Obstetrics and Gynecology HospitalCapital Medical UniversityBeijingChina

Personalised recommendations