An Adaptive Sampling Scheme to Efficiently Train Fully Convolutional Networks for Semantic Segmentation

  • Lorenz BergerEmail author
  • Hyde Eoin
  • M. Jorge Cardoso
  • Sébastien Ourselin
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 894)


Deep convolutional neural networks (CNNs) have shown excellent performance in object recognition tasks and dense classification problems such as semantic segmentation. However, training deep neural networks on large and sparse datasets is still challenging and can require large amounts of computation and memory. In this work, we address the task of performing semantic segmentation on large data sets, such as three-dimensional medical images. We propose an adaptive sampling scheme that uses a-posterior error maps, generated throughout training, to focus sampling on difficult regions, resulting in improved learning. Our contribution is threefold: (1) We give a detailed description of the proposed sampling algorithm to speed up and improve learning performance on large images. (2) We propose a deep dual path CNN that captures information at fine and coarse scales, resulting in a network with a large field of view and high resolution outputs. (3) We show that our method is able to attain new state-of-the-art results on the VISCERAL Anatomy benchmark.


Adaptive Sampling Strategy Proposed Sampling Algorithm Convolutional Neural Network (CNN) Density Classification Problem Dice Score 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This research was part funded by a NIHR i4i-connect grant.


  1. 1.
    Kamnitsas, K., et al.: Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med. Image Anal. 36, 61–78 (2017)CrossRefGoogle Scholar
  2. 2.
    Christ, P.F., et al.: Automatic liver and lesion segmentation in CT using cascaded fully convolutional neural networks and 3D conditional random fields. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 415–423. Springer, Cham (2016). Scholar
  3. 3.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). Scholar
  4. 4.
    Dou, Q., Chen, H., Jin, Y., Yu, L., Qin, J., Heng, P.-A.: 3D deeply supervised network for automatic liver segmentation from CT volumes. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 149–157. Springer, Cham (2016). Scholar
  5. 5.
    Bengio, Y., Louradour, J., Collobert, R., Weston, J.: Curriculum learning. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 41–48. ACM (2009)Google Scholar
  6. 6.
    Kumar, M.P., Packer, B., Koller, D.: Self-paced learning for latent variable models. In: Advances in Neural Information Processing Systems, pp. 1189–1197 (2010)Google Scholar
  7. 7.
    Avramova, V.: Curriculum learning with deep convolutional neural networks. KTH, School of Computer Science and Communication (CSC) (2015).
  8. 8.
    Qi, X., Liu, Z., Shi, J., Zhao, H., Jia, J.: Augmented feedback in semantic segmentation under image level supervision. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 90–105. Springer, Cham (2016). Scholar
  9. 9.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  10. 10.
    Fidon, L., et al.: Generalised Wasserstein dice score for imbalanced multi-class segmentation using holistic convolutional networks. arXiv preprint arXiv:1707.00478 (2017)
  11. 11.
    Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S., Cardoso, M.J.: Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. arXiv preprint arXiv:1707.03237 (2017)
  12. 12.
    Jimenez-del Toro, O., et al.: Cloud-based evaluation of anatomical structure segmentation and landmark detection algorithms: VISCERAL anatomy benchmarks. IEEE Trans. Med. Imaging 35(11), 2459–2475 (2016)CrossRefGoogle Scholar
  13. 13.
    Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 249–256 (2010)Google Scholar
  14. 14.
    Goyal, P., et al.: Accurate, large minibatch SGD: training ImageNet in 1 hour. arXiv preprint arXiv:1706.02677 (2017)
  15. 15.
    Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course, vol. 87. Springer, New York (2013)zbMATHGoogle Scholar
  16. 16.
    Wang, C., Smedby, Ö.: Multi-organ segmentation using shape model guided local phase analysis. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 149–156. Springer, Cham (2015). Scholar
  17. 17.
    Vincent, G., Guillard, G., Bowes, M.: Fully automatic segmentation of the prostate using active appearance models. In: MICCAI Grand Challenge: Prostate MR Image Segmentation, vol. 2012 (2012)Google Scholar
  18. 18.
    Gass, T., Szekely, G., Goksel, O.: Multi-atlas segmentation and landmark localization in images with large field of view. In: Menze, B., et al. (eds.) MCV 2014. LNCS, vol. 8848, pp. 171–180. Springer, Cham (2014). Scholar
  19. 19.
    Krähenbühl, P., Koltun, V.: Efficient inference in fully connected CRFs with Gaussian edge potentials. In: Advances in Neural Information Processing Systems, pp. 109–117 (2011)Google Scholar
  20. 20.
    Jiménez del Toro, O.A., Müller, H.: Hierarchic multi–atlas based segmentation for anatomical structures: evaluation in the VISCERAL anatomy benchmarks. In: Menze, B., et al. (eds.) MCV 2014. LNCS, vol. 8848, pp. 189–200. Springer, Cham (2014). Scholar
  21. 21.
    Kéchichian, R., Valette, S., Sdika, M., Desvignes, M.: Automatic 3D multiorgan segmentation via clustering and graph cut using spatial relations and hierarchically-registered atlases. In: Menze, B., et al. (eds.) MCV 2014. LNCS, vol. 8848, pp. 201–209. Springer, Cham (2014). Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Lorenz Berger
    • 1
    Email author
  • Hyde Eoin
    • 1
  • M. Jorge Cardoso
    • 2
  • Sébastien Ourselin
    • 2
  1. 1.Innersight LabsLondonUK
  2. 2.Kings College LondonLondonUK

Personalised recommendations