Abstract
Training convolutional neural networks (CNNs) on large sets of high-resolution images is too computationally intense to be performed on commodity CPUs. Such architectures, however, achieve state-of-the-art results on low-resolution machine vision tasks such as recognition of handwritten characters. We have adapted the inherent multi-level parallelism of CNNs for Nvidia’s CUDA GPU architecture to accelerate the training by two orders of magnitude. This dramatic speedup permits to apply CNN architectures to pattern recognition tasks on datasets with high-resolution natural images.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Behnke, S.: Hierarchical Neural Networks for Image Interpretation. LNCS, vol. 2766. Springer, Heidelberg (2003)
Chellapilla, K., Puri, S., Simard, P.: High Performance Convolutional Neural Networks for Document Processing. In: Tenth International Workshop on Frontiers in Handwriting Recognition, La Baule, France, Université de Rennes (2006)
Lahabar, S., Agrawal, P., Narayanan, P.J.: High Performance Pattern Recognition on GPU. In: Proc. of National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics (January 2008)
Larochelle, H., Bengio, Y., Louradour, J., Lamblin, P.: Exploring Strategies for Training Deep Neural Networks. Journal of Machine Learning Research (2009)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-Based Learning Applied to Document Recognition. Proceedings of the IEEE 86(11), 2278–2324 (1998)
LeCun, Y., Huang, F., Bottou, L.: Learning Methods for Generic Object Recognition with Invariance to Pose and Lighting. In: Proc. of CVPR 2004 (2004)
Nvidia Corporation. CUDA Programming Guide 3.0 (February 2010)
Osadchy, M., LeCun, Y., Miller, M.: Synergistic Face Detection and Pose Estimation with Energy-Based Models. Journal of ML Research (2007)
Raina, R., Madhavan, A., Ng, A.Y.: Large-scale deep unsupervised learning using graphics processors. In: ACM Intl. Conference Proceeding Series (2009)
Riedmiller, M., Braun, H.: RPROP – A fast adaptive learning algorithm. In: Proc. of the Int. Symposium on Computer and Information Science VII (1992)
Simard, P.Y., Steinkraus, D., Platt, J.C.: Best Practice for Convolutional Neural Networks Applied to Visual Document Analysis. In: ICDAR (2003)
Uetz, R., Behnke, S.: Large-scale Object Recognition with CUDA-accelerated Hierarchical Neural Networks. In: Proc. of ICIS (2009)
Šerbedžija, N.: Simulating artificial neural networks on parallel architectures (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Scherer, D., Schulz, H., Behnke, S. (2010). Accelerating Large-Scale Convolutional Neural Networks with Parallel Graphics Multiprocessors. In: Diamantaras, K., Duch, W., Iliadis, L.S. (eds) Artificial Neural Networks – ICANN 2010. ICANN 2010. Lecture Notes in Computer Science, vol 6354. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15825-4_9
Download citation
DOI: https://doi.org/10.1007/978-3-642-15825-4_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15824-7
Online ISBN: 978-3-642-15825-4
eBook Packages: Computer ScienceComputer Science (R0)