Competition vs. Concatenation in Skip Connections of Fully Convolutional Networks

  • Santiago EstradaEmail author
  • Sailesh Conjeti
  • Muneer Ahmad
  • Nassir Navab
  • Martin Reuter
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11046)


Increased information sharing through short and long-range skip connections between layers in fully convolutional networks have demonstrated significant improvement in performance for semantic segmentation. In this paper, we propose Competitive Dense Fully Convolutional Networks (CDFNet) by introducing competitive maxout activations in place of naïve feature concatenation for inducing competition amongst layers. Within CDFNet, we propose two architectural contributions, namely competitive dense block (CDB) and competitive unpooling block (CUB) to induce competition at local and global scales for short and long-range skip connections respectively. This extension is demonstrated to boost learning of specialized sub-networks targeted at segmenting specific anatomies, which in turn eases the training of complex tasks. We present the proof-of-concept on the challenging task of whole body segmentation in the publicly available VISCERAL benchmark and demonstrate improved performance over multiple learning and registration based state-of-the-art methods.


  1. 1.
    Badrinarayanan, V., Kendall, A., Cipolla, R.: Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)CrossRefGoogle Scholar
  2. 2.
    Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). Scholar
  3. 3.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778 (2016)Google Scholar
  4. 4.
    Huang, G., Liu, Z., Weinberger, K.Q., van der Maaten, L.: Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, vol. 1, p. 3 (2017)Google Scholar
  5. 5.
    Goodfellow, I.J., Warde-Farley, D., Mirza, M., Courville, A., Bengio, Y.: Maxout networks. In: Proceedings of the 30th International Conference on International Conference on Machine Learning-Volume 28, pp. III-1319. JMLR. org (2013)Google Scholar
  6. 6.
    Liao, Z., Carneiro, G.: A deep convolutional neural network module that promotes competition of multiple-size filters. Pattern Recognit. 71, 94–105 (2017)CrossRefGoogle Scholar
  7. 7.
    Roy, A.G., Conjeti, S., Navab, N., Wachinger, C.: Quicknat: Segmenting MRI neuroanatomy in 20 s. arXiv preprint arXiv:1801.04161 (2018)
  8. 8.
    Jimenez-del Toro, O., Müller, H., Krenn, M., Gruenberg, K., Taha, A.A., Winterstein, M., Eggel, I., Foncubierta-Rodríguez, A., Goksel, O., Jakab, A.: Cloud-based evaluation of anatomical structure segmentation and landmark detection algorithms: visceral anatomy benchmarks. IEEE Trans. Med. Imaging 35(11), 2459–2475 (2016)CrossRefGoogle Scholar
  9. 9.
    Srivastava, R.K., Masci, J., Gomez, F., Schmidhuber, J.: Understanding locally competitive networks. arXiv preprint arXiv:1410.1165 (2014)
  10. 10.
    Liao, Z., Carneiro, G.: On the importance of normalisation layers in deep learning with piecewise linear activation units. In: 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1–8. IEEE (2016)Google Scholar
  11. 11.
    Jégou, S., Drozdzal, M., Vazquez, D., Romero, A., Bengio, Y.: The one hundred layers tiramisu: fully convolutional densenets for semantic segmentation. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1175–1183. IEEE (2017)Google Scholar
  12. 12.
    Roy, A.G., Conjeti, S., Sheet, D., Katouzian, A., Navab, N., Wachinger, C.: Error Corrective Boosting for Learning Fully Convolutional Networks with Limited Data. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 231–239. Springer, Cham (2017). Scholar
  13. 13.
    Chollet, F., et al.: Keras. (2015)

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Santiago Estrada
    • 1
    • 2
    Email author
  • Sailesh Conjeti
    • 1
    • 2
  • Muneer Ahmad
    • 1
    • 2
  • Nassir Navab
    • 2
  • Martin Reuter
    • 1
    • 3
  1. 1.German Center for Neurodegenerative Diseases (DZNE)BonnGermany
  2. 2.Computer Aided Medical ProceduresTechnische Universität MünchenMünchenGermany
  3. 3.Department of RadiologyHarvard Medical SchoolBostonUSA

Personalised recommendations