Skip to main content
SpringerLink
Log in

Performance Evaluation of Deep Learning Frameworks over Different Architectures

  • Conference paper
  • First Online:
High Performance Computing for Computational Science – VECPAR 2018 (VECPAR 2018)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 11333))

Included in the following conference series:

Abstract

We evaluate the performance of two well-known Deep Learning frameworks – Caffe and TensorFlow – on two different types of computing devices – GPU and NUMA CPU architecture – using two popular network models as benchmark – AlexNet and GoogLeNet. We variate batch sizes between trainings and estimate the average training time per iteration and per image on each configuration. Both frameworks presented similar times for the AlexNet model, and TensorFlow outperforms Caffe by presenting times up to 2 times lower than Caffe for the GoogLeNet Model. The work also presents the impact of lack of support by the frameworks for NUMA Architectures, and relates a problem stated on loss computation by the Caffe Framework.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://www.image-net.org/challenges/LSVRC/.

  2. 2.

    http://caffe.berkeleyvision.org.

  3. 3.

    https://www.tensorflow.org.

  4. 4.

    https://pypi.org/project/pip/.

  5. 5.

    https://github.com/intel/caffe.

  6. 6.

    https://www.openblas.net.

  7. 7.

    http://math-atlas.sourceforge.net.

  8. 8.

    https://software.intel.com/en-us/mkl.

  9. 9.

    https://developer.nvidia.com/nccl.

  10. 10.

    https://github.com/BVLC/caffe/blob/master/src/caffe/solver.cpp.

References

  1. AAbadi, M., et al.: Tensorflow: large-scalemachine learning on heterogeneous distributed systems (2016). CoRR abs/1603.04467. http://arxiv.org/abs/1603.04467

  2. Abdelfattah, A., Haidar, A., Tomov, S., Dongarra, J.: Performance, design, and autotuning of batched GEMM for GPUs. In: Kunkel, J.M., Balaji, P., Dongarra, J. (eds.) ISC High Performance 2016. LNCS, vol. 9697, pp. 21–38. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-41321-1_2

    Chapter  Google Scholar 

  3. Bahrampour, S., Ramakrishnan, N., Schott, L., Shah, M.: Comparative study of caffe, neon, theano, and torch for deep learning (2015). CoRR abs/1511.06435. http://arxiv.org/abs/1511.06435

  4. Cecka, C.: Pro Tip: cuBLAS Strided Batched Matrix Multiply, July 2018. https://devblogs.nvidia.com/cublas-strided-batched-matrix-multiply/

  5. Google: Deep Learning - Google Trends, May 2018. https://trends.google.com.br/trends/explore?date=all&q=%2Fm%2F0h1fn8h

  6. Google Inc.: TensorFlow Architecture, July 2018. https://www.tensorflow.org/extend/architecture

  7. Intel Corporation: Introducing Batch GEMM Operations, July 2018. https://software.intel.com/en-us/articles/introducing-batch-gemm-operations

  8. Jia, Y., et al.: Caffe: convolutional architecture for fast feature embedding (2014). CoRR abs/1408.5093. http://arxiv.org/abs/1408.5093

  9. Keskar, N.S., Mudigere, D., Nocedal, J., Smelyanskiy, M., Tang, P.T.P.: On large-batch training for deep learning: generalization gap and sharp minima (2016). CoRR abs/1609.04836. http://arxiv.org/abs/1609.04836

  10. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105. Curran Associates, Inc. (2012). http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf

  11. Pena, D., Forembski, A., Xu, X., Moloney, D.: Benchmarking of CNNs for low-cost, low-power robotics applications. In: Robotics: Science and Systems (RSS 2017) Workshop - New Frontier for Deep Learning in Robotics, July 2017

    Google Scholar 

  12. Roy, P., Song, S.L., Krishnamoorthy, S., Vishnu, A., Sengupta, D., Liu, X.: NUMA-Caffe: NUMA-aware deep learning neural networks. ACM Trans. Archit. Code Optim. 15(2), 24:1–24:26 (2018). https://doi.org/10.1145/3199605

    Article  Google Scholar 

  13. Shams, S., Platania, R., Lee, K., Park, S.J.: Evaluation of deep learning frameworks over different HPC architectures. In: 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), pp. 1389–1396, June 2017. https://doi.org/10.1109/ICDCS.2017.259

  14. Shi, S., Chu, X.: Performance modeling and evaluation of distributed deep learning frameworks on GPUs (2017). CoRR abs/1711.05979. http://arxiv.org/abs/1711.05979

  15. Szegedy, C., et al.: Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9, June 2015. https://doi.org/10.1109/CVPR.2015.7298594

  16. Vargas, R., Mosavi, A., Ruiz, L.: Deep learning: a review. In: Advances in Intelligent Systems and Computing (2017). https://www.researchgate.net/publication/318447392_DEEP_LEARNING_A_REVIEW

Download references

Acknowledgments

We thank CNPq for supporting the development of this work, and NVIDIA support with the donation of the NVIDIA GTX Titan X GPU used for our experiments.

Author information

Authors and Affiliations

  1. Computer Science Graduate Program (PPGCC), Universidade Federal de Santa Maria, Santa Maria, Brazil

    Rafael Gauna Trindade

  2. Department of Languages and Computer Systems, Universidade Federal de Santa Maria, Santa Maria, Brazil

    João Vicente Ferreira Lima & Andrea Schwerner Charão

Authors
  1. Rafael Gauna Trindade
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. João Vicente Ferreira Lima
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andrea Schwerner Charão
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Federal University of São Carlos, São Carlos, São Paulo, Brazil

    Hermes Senger

  2. Lawrence Berkeley National Laboratory, Berkeley, CA, USA

    Osni Marques

  3. Universidade Estadual Paulista Júlio de Mesquita Filho, Presidente Prudente, São Paulo, Brazil

    Rogerio Garcia

  4. Universidade Estadual Paulista Júlio de Mesquita Filho, São Paulo, São Paulo, Brazil

    Tatiana Pinheiro de Brito

  5. Universidade Estadual Paulista Júlio de Mesquita Filho, São Paulo, São Paulo, Brazil

    Rogério Iope

  6. Universidade Estadual Paulista Júlio de Mesquita Filho, São Paulo, São Paulo, Brazil

    Silvio Stanzani

  7. Universidad Nacional de San Luis, San Luis, Argentina

    Veronica Gil-Costa

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Trindade, R.G., Lima, J.V.F., Charão, A.S. (2019). Performance Evaluation of Deep Learning Frameworks over Different Architectures. In: Senger, H., et al. High Performance Computing for Computational Science – VECPAR 2018. VECPAR 2018. Lecture Notes in Computer Science(), vol 11333. Springer, Cham. https://doi.org/10.1007/978-3-030-15996-2_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-15996-2_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-15995-5

  • Online ISBN: 978-3-030-15996-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation