Instance Segmentation of Biomedical Images with an Object-Aware Embedding Learned with Local Constraints

  • Long Chen
  • Martin Strauch
  • Dorit MerhofEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11764)


Automatic instance segmentation is a problem that occurs in many biomedical applications. State-of-the-art approaches either perform semantic segmentation or refine object bounding boxes obtained from detection methods. Both suffer from crowded objects to varying degrees, merging adjacent objects or suppressing a valid object. In this work, we assign an embedding vector to each pixel through a deep neural network. The network is trained to output embedding vectors of similar directions for pixels from the same object, while adjacent objects are orthogonal in the embedding space, which effectively avoids the fusion of objects in a crowd. Our method yields state-of-the-art results even with a light-weighted backbone network on a cell segmentation (BBBC006 + DSB2018) and a leaf segmentation data set (CVPPP2017). The code and model weights are public available (


Instance segmentation CNN Object embedding 


  1. 1.
    Scharr, H., et al.: Leaf segmentation in plant phenotyping: a collation study. Mach. Vis. Appl. 27(4), 585–606 (2016)CrossRefGoogle Scholar
  2. 2.
    Ulman, V., et al.: An objective comparison of cell-tracking algorithms. Nat. Methods 14, 1141–1152 (2017)CrossRefGoogle Scholar
  3. 3.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). Scholar
  4. 4.
    Chen, H., Qi, X., Yu L., Dou, Q., Qin, J., Heng, P.A.: DCAN: deep contour-aware networks for accurate gland segmentation. In: 2016 CVPR, pp. 2487–2496 (2016)Google Scholar
  5. 5.
    Ren, S., He, K, Girshick, R., Sun, J,: Faster R-CNN: towards real-time object detection with region proposal networks. In: 28th NIPS, pp. 91–99 (2015)Google Scholar
  6. 6.
    Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). Scholar
  7. 7.
    He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: 2017 ICCV, pp. 2980-2988 (2017)Google Scholar
  8. 8.
    Schmidt, U., Weigert, M., Broaddus, C., Myers, G.: Cell detection with star-convex polygons. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 265–273. Springer, Cham (2018). Scholar
  9. 9.
    Jetley, S., Sapienza, M., Golodetz, S., Torr, P.H.: Straight to shapes: real-time detection of encoded shapes. In: 2017 CVPR, pp. 4207–4216 (2017)Google Scholar
  10. 10.
    De Brabandere, B., Neven, D., Van Gool, L.: Semantic instance segmentation with a discriminative loss function. CoRR (2017)Google Scholar
  11. 11.
    Fathi, A., et al.: Semantic instance segmentation via deep metric learning. CoRR (2017)Google Scholar
  12. 12.
    De Brabandere, B., Neven, D., Van Gool, L.: Semantic instance segmentation for autonomous driving. In: 2017 CVPR Workshop, pp. 478–480 (2017)Google Scholar
  13. 13.
    Kong, S., Fowlkes, C.C.: Recurrent pixel embedding for instance grouping. In: CVPR, pp. 9018–9028 (2018)Google Scholar
  14. 14.
    Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)CrossRefGoogle Scholar
  15. 15.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 CVPR, pp. 770–778 (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Institute of Imaging & Computer VisionRWTH Aachen UniversityAachenGermany

Personalised recommendations