Skip to main content
SpringerLink
Log in

Adaptation of RBM Learning for Intel MIC Architecture

  • Conference paper
Artificial Intelligence and Soft Computing (ICAISC 2015)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9119))

Included in the following conference series:

Abstract

In the paper, the parallel realization of the Boltzmann Restricted Machine (RBM) is proposed. The implementation intends to use multicore architectures of modern CPUs and Intel Xeon Phi coprocessor. The learning procedure is based on the matrix description of RBM, where the learning samples are grouped into packages, and represented as matrices. The influence of the package size on convergence of learning, as well as on performance of computation, are studied for various number of threads, using conventional CPU and Intel Phi architecures. Our research confirms a potential usefulness of MIC parallel architecture for implementation of RBM and similar algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bilski, J., Nowicki, R., Scherer, R., Litwiński, S.: Application of signal processor TMS320c30 to neural networks realisation. In: Proceedings of the Second Conference Neural Networks and Their Applications, Czêstochowa, pp. 53–59 (1996)

    Google Scholar 

  2. Bilski, J., Smolag, J.: Parallel architectures for learning the RTRN and Elman dynamic neural networks. IEEE Transactions on Parallel and Distributed Systems PP(99) (2014)

    Google Scholar 

  3. Bilski, J., Litwiński, S., Smoląg, J.: Parallel realisation of QR algorithm for neural networks learning. In: Rutkowski, L., Siekmann, J.H., Tadeusiewicz, R., Zadeh, L.A. (eds.) ICAISC 2004. LNCS (LNAI), vol. 3070, pp. 158–165. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  4. Bilski, J., Smoląg, J., Galushkin, A.I.: The parallel approach to the conjugate gradient learning algorithm for the feedforward neural networks. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2014, Part I. LNCS (LNAI), vol. 8467, pp. 12–21. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  5. Bilski, J., Smoląg, J.: Parallel realisation of the recurrent RTRN neural network learning. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2008. LNCS (LNAI), vol. 5097, pp. 11–16. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  6. Bilski, J., Smoląg, J.: Parallel realisation of the recurrent elman neural network learning. In: Rutkowski, L., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2010, Part II. LNCS (LNAI), vol. 6114, pp. 19–25. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  7. Bilski, J., Smoląg, J.: Parallel realisation of the recurrent multi layer perceptron learning. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2012, Part I. LNCS, vol. 7267, pp. 12–20. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  8. Bilski, J., Smoląg, J.: Parallel approach to learning of the recurrent Jordan neural network. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013, Part I. LNCS, vol. 7894, pp. 32–40. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  9. Chu, J.L., Krzyzak, A.: The recognition of partially occluded objects with support vector machines and convolutional neural networks and deep belief networks. Journal of Artificial Intelligence and Soft Computing Research 4(1), 5–19 (2014)

    Article  Google Scholar 

  10. Intel Corporation: Intel Xeon Phi Coprocessor System Software Developer’s Guide. Technical report, The Intel Corporation (June 2013)

    Google Scholar 

  11. Cpałka, K., Rutkowski, L.: Flexible Takagi-Sugeno fuzzy systems. In: Proc. IEEE International Joint Conference on Neural Networks (IJCNN), vol. 3, pp. 1764–1769 (2005)

    Google Scholar 

  12. Cpałka, K., Łapa, K., Przybył, A., Zalasiński, M.: A new method for designing neuro-fuzzy systems for nonlinear modelling with interpretability aspects. Neurocomputing 135, 203–217 (2014)

    Article  Google Scholar 

  13. Cpałka, K., Rebrova, O., Nowicki, R., Rutkowski, L.: On design of flexible neuro-fuzzy systems for nonlinear modelling. International Journal of General Systems 42(6), 706–720 (2013)

    Article  MATH  Google Scholar 

  14. Dourlens, S., Ramdane-Cherif, A.: Modeling & understanding environment using semantic agents. Journal of Artificial Intelligence and Soft Computing Research 1(4), 301–314 (2011)

    Google Scholar 

  15. Fang, J., Varbanescu, A.L., Sips, H.: Benchmarking Intel Xeon Phi to Guide Kernel Design. Delft University of Technology Parallel and Distributed Systems Report Series, No. PDS-2013-005, pp. 1–22 (2013)

    Google Scholar 

  16. Gabryel, M., Korytkowski, M., Scherer, R., Rutkowski, L.: Object detection by simple fuzzy classifiers generated by boosting. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013, Part I. LNCS, vol. 7894, pp. 540–547. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  17. Galkowski, T., Rutkowski, L.: Nonparametric fitting of multivariate functions. IEEE Transactions on Automatic Control 31(8), 785–787 (1986)

    Article  Google Scholar 

  18. Gałkowski, T.: Kernel estimation of regression functions in the boundary regions. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013, Part II. LNCS, vol. 7895, pp. 158–166. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  19. Galkowski, T., Pawlak, M.: Nonparametric function fitting in the presence of nonstationary noise. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2014, Part I. LNCS (LNAI), vol. 8467, pp. 531–538. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  20. Gaweda, A.E., Scherer, R.: Fuzzy number-based hierarchical fuzzy system. In: Rutkowski, L., Siekmann, J.H., Tadeusiewicz, R., Zadeh, L.A. (eds.) ICAISC 2004. LNCS (LNAI), vol. 3070, pp. 302–307. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  21. Hinton, G.: Training products of experts by minimizing contrastive divergence. Neural Computation 14(8), 1771–1800 (2002)

    Article  MathSciNet  Google Scholar 

  22. Hinton, G.: A practical guide to training restricted Boltzmann machines. Momentum 9(1), 926 (2010)

    Google Scholar 

  23. Hinton, G., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Computation 18(7), 1527–1554 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  24. http://miclab.pl : MICLAB Pilot laboratory of massively parallel systems. Web Page (2015)

  25. http://yann.lecun.com/exdb/mnist/: The mnist database of handwritten digits

  26. Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., Fei-Fei, L.: Large-scale video classification with convolutional neural networks. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1725–1732 (June 2014)

    Google Scholar 

  27. Korytkowski, M., Rutkowski, L., Scherer, R.: From ensemble of fuzzy classifiers to single fuzzy rule base classifier. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2008. LNCS (LNAI), vol. 5097, pp. 265–272. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  28. Krizhevsky, A.: One weird trick for parallelizing convolutional neural networks. arXiv preprint arXiv:1404.5997 (2014)

    Google Scholar 

  29. Laskowski, L., Laskowska, M.: Functionalization of SBA-15 mesoporous silica by cu-phosphonate units: Probing of synthesis route. Journal of Solid State Chemistry 220, 221–226 (2014)

    Article  Google Scholar 

  30. Laskowski, L., Laskowska, M., Balanda, M., Fitta, M., Kwiatkowska, J., Dzilinski, K., Karczmarska, A.: Mesoporous silica SBA-15 functionalized by nickel-phosphonic units: Raman and magnetic analysis. Microporous and Mesoporous Materials 200, 253–259 (2014)

    Article  Google Scholar 

  31. Laskowski, Ł., Laskowska, M., Jelonkiewicz, J., Boullanger, A.: Spin-glass implementation of a Hopfield neural structure. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2014, Part I. LNCS (LNAI), vol. 8467, pp. 89–96. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  32. Le Roux, N., Bengio, Y.: Representational power of restricted boltzmann machines and deep belief networks. Neural Computation 20(6), 1631–1649 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  33. Pabiasz, S., Starczewski, J.: Face reconstruction for 3D systems. In: Rutkowska, D., Cader, A., Przybyszewski, K. (eds.) Selected Topics in Computer Science Applications, pp. 54–63. Academic Publishing House EXIT (2011)

    Google Scholar 

  34. Pabiasz, S., Starczewski, J.T.: Meshes vs. depth maps in face recognition systems. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2012, Part I. LNCS, vol. 7267, pp. 567–573. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  35. Pabiasz, S., Starczewski, J.T., Marvuglia, A.: A new three-dimensional facial landmarks in recognition. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2014, Part II. LNCS (LNAI), vol. 8468, pp. 179–186. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  36. Pabiasz, S., Starczewski, J.T.: A new approach to determine three-dimensional facial landmarks. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013, Part II. LNCS, vol. 7895, pp. 286–296. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  37. Patan, K., Patan, M.: Optimal training strategies for locally recurrent neural networks. Journal of Artificial Intelligence and Soft Computing Research 1(2), 103–114 (2011)

    Google Scholar 

  38. Reinders, J.: An Overview of Programming for Intel Xeon Processors and Intel Xeon Phi Coprocessors. Technical report, The Intel Corporation (2012)

    Google Scholar 

  39. Rosenblatt, F.: The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review 65(65), 386–408 (1958)

    Article  MathSciNet  Google Scholar 

  40. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al.: Imagenet large scale visual recognition challenge. arXiv preprint arXiv:1409.0575 (2014)

    Google Scholar 

  41. Rutkowski, L., Przybył, A., Cpałka, K., Er, M.J.: Online speed profile generation for industrial machine tool based on neuro-fuzzy approach. In: Rutkowski, L., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2010, Part II. LNCS (LNAI), vol. 6114, pp. 645–650. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  42. Saule, E., Kaya, K., Çatalyürek, Ü.V.: Performance Evaluation of Sparse Matrix Multiplication Kernels on Intel Xeon Phi. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Waśniewski, J. (eds.) PPAM 2013, Part I. LNCS, vol. 8384, pp. 559–570. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  43. Scherer, R., Rutkowski, L.: A fuzzy relational system with linguistic antecedent certainty factors. In: Rutkowski, L., Kacprzyk, J. (eds.) Proceedings of the Sixth International Conference on Neural Network and Soft Computing. Advances in Soft Computing, pp. 563–569. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  44. Scherer, R.: Neuro-fuzzy relational systems for nonlinear approximation and prediction. Nonlinear Analysis 71, e1420–e1425 (2009)

    Google Scholar 

  45. Smolensky, P.: Information processing in dynamical systems: Foundations of harmony theory. In: Rumelhart, D.E., McLelland, J.L. (eds.) Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Foundations, vol. 1, pp. 194–281. MIT (1986)

    Google Scholar 

  46. Staff, C.I., Reinders, J.: Parallel Programming and Optimization with Intel® Xeon PhiTM Coprocessors: Handbook on the Development and Optimization of Parallel Applications for Intel® Xeon Coprocessors and Intel® Xeon PhiTM Coprocessors. Colfax International (2013)

    Google Scholar 

  47. Szustak, L., Rojek, K., Gepner, P.: Using Intel Xeon Phi coprocessor to accelerate computations in MPDATA algorithm. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Waśniewski, J. (eds.) PPAM 2013, Part I. LNCS, vol. 8384, pp. 582–592. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  48. Szustak, L., Rojek, K., Olas, T., Kuczynski, L., Halbiniak, K., Gepner, P.: Adaptation of MPDATA heterogeneous stencil computation to Intel Xeon Phi coprocessor. Scientific Programming (in press, 2015)

    Google Scholar 

  49. Tambouratzis, T., Chernikova, D., Pázsit, I.: Pulse shape discrimination of neutrons and gamma rays using Kohonen artificial neural networks. Journal of Artificial Intelligence and Soft Computing Research 3(2), 77–88 (2013)

    Article  Google Scholar 

  50. Wyrzykowski, R., Szustak, L., Rojek, K.: Parallelization of 2d MPDATA EULAG algorithm on hybrid architectures with GPU accelerators. Parallel Computing 40(8), 425–447 (2014)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Computer and Information Sciences, Czestochowa University of Technology, ul. Dabrowskiego 73, 42-200, Czestochowa, Poland

    Tomasz Olas & Roman Wyrzykowski

  2. Institute of Computational Intelligence, Czestochowa University of Technology, Al. Armii Krajowej 36, 42-200, Czestochowa, Poland

    Wojciech K. Mleczko & Robert K. Nowicki

  3. Department of Computer Science and Software Engineering, Concordia University, Montreal, Canada

    Adam Krzyzak

Authors
  1. Tomasz Olas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wojciech K. Mleczko
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Robert K. Nowicki
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Roman Wyrzykowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Adam Krzyzak
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tomasz Olas .

Editor information

Editors and Affiliations

  1. Częstochowa University of Technology, Częstochowa, Poland

    Leszek Rutkowski

  2. Częstochowa University of Technology, Częstochowa, Poland

    Marcin Korytkowski

  3. Częstochowa University of Technology, Częstochowa, Poland

    Rafal Scherer

  4. AGH University of Science and Technology, Krakow, Poland

    Ryszard Tadeusiewicz

  5. University of California, Berkeley, California, USA

    Lotfi A. Zadeh

  6. University of Louisville, Louisville, Kentucky, USA

    Jacek M. Zurada

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Olas, T., Mleczko, W.K., Nowicki, R.K., Wyrzykowski, R., Krzyzak, A. (2015). Adaptation of RBM Learning for Intel MIC Architecture. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L., Zurada, J. (eds) Artificial Intelligence and Soft Computing. ICAISC 2015. Lecture Notes in Computer Science(), vol 9119. Springer, Cham. https://doi.org/10.1007/978-3-319-19324-3_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-19324-3_9

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-19323-6

  • Online ISBN: 978-3-319-19324-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation