Skip to main content

Benchmarking Deep Learning Infrastructures by Means of TensorFlow and Containers

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 11887)


Ever growing interest and usage of deep learning rises a question on the performance of various infrastructures suitable for training of neural networks. We present here our approach and first results of tests performed with TensorFlow Benchmarks which use best practices for multi-GPU and distributed training. We pack the Benchmarks in Docker containers and execute them by means of uDocker and Singularity container tools on a single machine and in the HPC environment. The Benchmarks comprise a number of convolutional neural network models run across synthetic data and e.g. the ImageNet dataset. For the same Nvidia K80 GPU card we achieve the same performance in terms of processed images per second and similar scalability between 1-2-4 GPUs as presented by the TensorFlow developers. We therefore do not obtain statistically significant overhead due to the usage of containers in the multi-GPU case, and the approach of using TF Benchmarks in a Docker container can be applied across various systems.


  • Benchmarks
  • TensorFlow
  • ConvNet
  • Containers

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD   79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions


  1. Computational resource ForHLR-II available at Karlsruhe Institute of Technology. Accessed 12 Apr 2019

  2. DEEP as a Service (DEEPaaS) API. Accessed 12 Apr 2019

  3. Docker Hub website. Accessed 12 Apr 2019

  4. ImageNet Object Localization Challenge at Kaggle. Accessed 12 Apr 2019

  5. Nvidia Geforce GTX980 Ti specifications. Accessed 12 Apr 2019

  6. Nvidia Tesla K80 GPU Accelerator, Board Specification (2015). Accessed 12 Apr 2019

  7. Results of TF Benchmarks runs, GitHub repository. Accessed 12 Apr 2019

  8. Scripts for downloading and converting ImageNet data to TFRecord format. Accessed 12 Apr 2019

  9. TensorFlow Benchmarks, GitHub repository. Accessed 8 Apr 2019

  10. TensorFlow Benchmarks webpage. Accessed 8 Apr 2019

  11. TF Benchmarks fork for TF 1.10.0, KIT Gitlab. Accessed 12 Apr 2019

  12. udocker: A basic user tool to execute simple docker containers in batch or interactive systems without root privileges. GiHub repository. Accessed 8 Apr 2019

  13. Baker, M.: Is there a reproducibility crisis? Nature 533, 452–454 (2016)

    CrossRef  Google Scholar 

  14. Gomes, J., et al.: Enabling rootless linux containers in multi-user environments: the udocker tool. Comput. Phys. Commun. 232, 84–97 (2018).

    CrossRef  Google Scholar 

  15. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2015)

    Google Scholar 

  16. Jacobsen, D.M., Canon, R.S.: Contain this, unleashing Docker for HPC. Cray User Group 2015 (2015)

    Google Scholar 

  17. Krizhevsky, A.: Learning multiple layers of features from tiny images. Technical report (2009).

  18. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105. Curran Associates, Inc. (2012).

  19. Kurtzer, G.M., Sochat, V., Bauer, M.W.: Singularity: scientific containers for mobility of compute. PLoS One (2017).

    CrossRef  Google Scholar 

  20. Lin, T., et al.: Microsoft COCO: common objects in context. CoRR abs/1405.0312 (2014).

  21. Merkel, D.: Docker: lightweight Linux containers for consistent development and deployment (2014).

  22. Panayotov, V., Chen, G., Povey, D., Khudanpur, S.: Librispeech: an ASR corpus based on public domain audio books, pp. 5206–5210, April 2015.

  23. Priedhorsky, R., Randles, T.C., Randles, T.: Charliecloud: unprivileged containers for user-defined software stacks in HPC. In: SC17: International Conference for High Performance Computing, Networking, Storage and Analysis (2017).

  24. Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. (IJCV) 115(3), 211–252 (2015).

    CrossRef  MathSciNet  Google Scholar 

  25. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014)

    Google Scholar 

  26. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: InceptionV3. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2818–2826 (2016)

    Google Scholar 

Download references


uDocker is being developed within the DEEP HybridDataCloud project, which receives funding from the European Union’s Horizon 2020 research and innovation program under agreement RIA 777435.

A part of this work was performed on the computational resource ForHLR-II funded by the Ministry of Science, Research and the Arts Baden-Wuerttemberg and DFG (“Deutsche Forschungsgemeinschaft”).

Author information

Authors and Affiliations


Corresponding author

Correspondence to Valentin Kozlov .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Grupp, A., Kozlov, V., Campos, I., David, M., Gomes, J., López García, Á. (2019). Benchmarking Deep Learning Infrastructures by Means of TensorFlow and Containers. In: Weiland, M., Juckeland, G., Alam, S., Jagode, H. (eds) High Performance Computing. ISC High Performance 2019. Lecture Notes in Computer Science(), vol 11887. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-34355-2

  • Online ISBN: 978-3-030-34356-9

  • eBook Packages: Computer ScienceComputer Science (R0)