SSHMT: Semi-supervised Hierarchical Merge Tree for Electron Microscopy Image Segmentation

  • Ting Liu
  • Miaomiao Zhang
  • Mehran Javanmardi
  • Nisha Ramesh
  • Tolga Tasdizen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9905)


Region-based methods have proven necessary for improving segmentation accuracy of neuronal structures in electron microscopy (EM) images. Most region-based segmentation methods use a scoring function to determine region merging. Such functions are usually learned with supervised algorithms that demand considerable ground truth data, which are costly to collect. We propose a semi-supervised approach that reduces this demand. Based on a merge tree structure, we develop a differentiable unsupervised loss term that enforces consistent predictions from the learned function. We then propose a Bayesian model that combines the supervised and the unsupervised information for probabilistic learning. The experimental results on three EM data sets demonstrate that by using a subset of only \(3\,\%\) to \(7\,\%\) of the entire ground truth data, our approach consistently performs close to the state-of-the-art supervised method with the full labeled data set, and significantly outperforms the supervised method with the same labeled subset.


Image segmentation Electron microscopy Semi-supervised learning Hierarchical segmentation Connectomics 



This work was supported by NSF IIS-1149299 and NIH 1R01NS075314-01. We thank the National Center for Microscopy and Imaging Research at the University of California, San Diego, for providing the mouse neuropil data set. We also thank Mehdi Sajjadi at the University of Utah for the constructive discussions.

Supplementary material

419956_1_En_9_MOESM1_ESM.pdf (107 kb)
Supplementary material 1 (pdf 107 KB)


  1. 1.
    Sporns, O., Tononi, G., Kötter, R.: The human connectome: a structural description of the human brain. PLoS Comput. Biol. 1(4), e42 (2005)CrossRefGoogle Scholar
  2. 2.
    Famiglietti, E.V.: Synaptic organization of starburst amacrine cells in rabbit retina: analysis of serial thin sections by electron microscopy and graphic reconstruction. J. Comp. Neurol. 309(1), 40–70 (1991)CrossRefGoogle Scholar
  3. 3.
    Briggman, K.L., Helmstaedter, M., Denk, W.: Wiring specificity in the direction-selectivity circuit of the retina. Nature 471(7337), 183–188 (2011)CrossRefGoogle Scholar
  4. 4.
    Helmstaedter, M.: Cellular-resolution connectomics: challenges of dense neural circuit reconstruction. Nat. Methods 10(6), 501–507 (2013)CrossRefGoogle Scholar
  5. 5.
    Briggman, K.L., Denk, W.: Towards neural circuit reconstruction with volume electron microscopy techniques. Current Opin. Neurobiol. 16(5), 562–570 (2006)CrossRefGoogle Scholar
  6. 6.
    Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Patt. Anal. Mach. Intell. 33(5), 898–916 (2011)CrossRefGoogle Scholar
  7. 7.
    Ren, Z., Shakhnarovich, G.: Image segmentation by cascaded region agglomeration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2011–2018 (2013)Google Scholar
  8. 8.
    Arbeláez, P., Pont-Tuset, J., Barron, J., Marques, F., Malik, J.: Multiscale combinatorial grouping. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 328–335 (2014)Google Scholar
  9. 9.
    Liu, T., Seyedhosseini, M., Tasdizen, T.: Image segmentation using hierarchical merge tree. IEEE Trans. Image Process. 25(10), 4596–4607 (2016). doi: 10.1109/TIP.2016.2592704 CrossRefGoogle Scholar
  10. 10.
    Sommer, C., Straehle, C., Koethe, U., Hamprecht, F.A.: ilastik: Interactive learning and segmentation toolkit. In: 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 230–233. IEEE (2011)Google Scholar
  11. 11.
    Ciresan, D., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Deep neural networks segment neuronal membranes in electron microscopy images. Adv. Neural Inf. Process. Syst. 25, 2852–2860 (2012)Google Scholar
  12. 12.
    Seyedhosseini, M., Sajjadi, M., Tasdizen, T.: Image segmentation with cascaded hierarchical models and logistic disjunctive normal networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2168–2175 (2013)Google Scholar
  13. 13.
    Arganda-Carreras, I., Turaga, S.C., Berger, D.R., Cireşan, D.: Crowdsourcing the creation of image segmentation algorithms for connectomics. Front. Neuroanat. 9, 142 (2015)CrossRefGoogle Scholar
  14. 14.
    Nunez-Iglesias, J., Kennedy, R., Parag, T., Shi, J., Chklovskii, D.B.: Machine learning of hierarchical clustering to segment 2D and 3D images. PLoS ONE 8(8), e71715 (2013)CrossRefGoogle Scholar
  15. 15.
    Kaynig, V., Vazquez-Reina, A., Knowles-Barley, S., Roberts, M., Jones, T.R., Kasthuri, N., Miller, E., Lichtman, J., Pfister, H.: Large-scale automatic reconstruction of neuronal processes from electron microscopy images. Med. Image Anal. 22(1), 77–88 (2015)CrossRefGoogle Scholar
  16. 16.
    Krasowski, N., Beier, T., Knott, G., Koethe, U., Hamprecht, F., Kreshuk, A.: Improving 3D EM data segmentation by joint optimization over boundary evidence and biological priors. In: 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI), pp. 536–539. IEEE (2015)Google Scholar
  17. 17.
    Liu, T., Jones, C., Seyedhosseini, M., Tasdizen, T.: A modular hierarchical approach to 3D electron microscopy image segmentation. J. Neurosci. Methods 226, 88–102 (2014)CrossRefGoogle Scholar
  18. 18.
    Funke, J., Hamprecht, F.A., Zhang, C.: Learning to segment: training hierarchical segmentation under a topological loss. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 268–275. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-24574-4_32 CrossRefGoogle Scholar
  19. 19.
    Uzunbas, M.G., Chen, C., Metaxas, D.: An efficient conditional random field approach for automatic and interactive neuron segmentation. Med. Image Anal. 27, 31–44 (2016)CrossRefGoogle Scholar
  20. 20.
    Parag, T., Plaza, S., Scheffer, L.: Small sample learning of superpixel classifiers for EM segmentation. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014. LNCS, vol. 8673, pp. 389–397. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-10404-1_49 Google Scholar
  21. 21.
    Parag, T., Ciresan, D.C., Giusti, A.: Efficient classifier training to minimize false merges in electron microscopy segmentation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 657–665 (2015)Google Scholar
  22. 22.
    Arganda-Carreras, I., Seung, H.S., Vishwanathan, A., Berger, D.: 3D segmentation of neurites in EM images challenge - ISBI 2013 (2013). Accessed 16 Feb 2016
  23. 23.
    Deerinck, T.J., Bushong, E.A., Lev-Ram, V., Shu, X., Tsien, R.Y., Ellisman, M.H.: Enhancing serial block-face scanning electron microscopy to enable high resolution 3-D nanohistology of cells and tissues. Microsc. Microanal. 16(S2), 1138–1139 (2010)CrossRefGoogle Scholar
  24. 24.
    Knott, G., Marchman, H., Wall, D., Lich, B.: Serial section scanning electron microscopy of adult brain tissue using focused ion beam milling. J. Neurosci. 28(12), 2959–2964 (2008)CrossRefGoogle Scholar
  25. 25.
    Beucher, S., Meyer, F.: The morphological approach to segmentation: the watershed transformation. Math. Morphol. Image Process. 34, 433–481 (1993). Marcel Dekker AGGoogle Scholar
  26. 26.
    Schindelin, J., Arganda-Carreras, I., Frise, E., Kaynig, V., Longair, M., Pietzsch, T., Preibisch, S., Rueden, C., Saalfeld, S., Schmid, B., et al.: Fiji: an open-source platform for biological-image analysis. Nat. Methods 9(7), 676–682 (2012)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Ting Liu
    • 1
  • Miaomiao Zhang
    • 2
  • Mehran Javanmardi
    • 1
  • Nisha Ramesh
    • 1
  • Tolga Tasdizen
    • 1
  1. 1.Scientific Computing and Imaging InstituteUniversity of UtahSalt Lake CityUSA
  2. 2.CSAILMassachusetts Institute of TechnologyCambridgeUSA

Personalised recommendations