Skip to main content
SpringerLink
Log in

Parallel Bayesian ARTMAP and Its OpenCL Implementation

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

The Bayesian ARTMAP neural network, introduced by Vigdor and Lerner, is an incremental learning algorithm which can efficiently process massive datasets for classification, regression, and probabilistic inference tasks. We introduce the parallelized version of the BA neural network and implement it in OpenCL. Our implementation runs on both multi-core CPUs and GPUs architectures. We test the Parallel Bayesian ARTMAP on several classification and regression benchmarks focusing on speedup and scalability. In some cases, the parallel BA runs by an order of magnitude faster than the sequential implementation. Our implementation has the potential to scale for OpenCL devices with increasing number of compute units.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. The ratio between sequential and parallel execution times.

References

  1. Andonie R, Fabry-Asztalos L, Sasu L (2014) Bayesian ARTMAP prediction of biological activities for potential HIV-1 protease inhibitors using a small molecular dataset. In: Proceedings of the IEEE symposium on computational intelligence and computational biology (CIBCB 2014) Honolulu, HI, pp 1–8 (2014)

  2. Andonie R, Sasu L (2006) Fuzzy ARTMAP with input relevances. IEEE Trans Neural Netw 17:929–941

    Article  Google Scholar 

  3. Bottou L, Cun YL (2004) Large scale online learning. In: Thrun S, Saul L, Schölkopf B (eds) Advances in neural information processing systems, vol 16. MIT Press, Cambridge, pp 217–224

    Google Scholar 

  4. Brandstetter A, Artusi A (2008) Radial basis function networks GPU-based implementation. IEEE Trans Neural Netw 19(12):2150–2154

    Article  Google Scholar 

  5. Carpenter GA, Grossberg S, Markuzon N, Reynolds JH, Rosen DB (1992) Fuzzy ARTMAP: a neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Trans Neural Netw 3:698–713

    Article  Google Scholar 

  6. Chan TF, Golub GH, Leveque RJ (1983) Algorithms for computing the sample variance: analysis and recommendations. Am Stat 37(3):242–247

    MathSciNet  MATH  Google Scholar 

  7. Constantinopoulos C, Likas A (2006) An incremental training method for the probabilistic RBF network. IEEE Trans Neural Netw 17(4):966–974

    Article  MATH  Google Scholar 

  8. Demmel JW (1987) The geometry of ill-conditioning. J Complex 3:201–229

    Article  MathSciNet  MATH  Google Scholar 

  9. Dietterich T (2000) Ensemble methods in machine learning. In: Multiple classifier systems. Lecture notes in computer science, vol 1857. Springer, Berlin, pp 1–15

  10. Dinkelbach HÜ, Vitay J, Beuth F, Hamker FH (2012) Comparison of GPU- and CPU-implementations of mean-firing rate neural networks on parallel hardware. Netw Comput Neural Syst 23(4):212–236

    Google Scholar 

  11. Duda RO, Hart PE, Stork DG (2000) Pattern classification, 2nd edn. Wiley, New York

    MATH  Google Scholar 

  12. Esposito A, Marinaro M, Oricchio D, Scarpetta S (1999) A new incremental strategy for function approximation by radial basis function neural networks. In: Marinaro M, Tagliaferri R (eds) Neural nets WIRN VIETRI-98 (Perspectives in neural computing). Springer, London, pp 113–119

    Chapter  Google Scholar 

  13. Fukushima K (2004) Neocognitron capable of incremental learning. Neural Netw 17:37–46

    Article  MATH  Google Scholar 

  14. Hand DJ (2013) Data, not dogma: big data, open data, and the opportunities ahead. In: Tucker A, Hppner F, Siebes A, Swift S (eds) Advances in intelligent data analysis XII. Lecture notes in computer science, vol 8207. Springer, Berlin, pp 1–12

    Chapter  Google Scholar 

  15. Harris M (2008) Optimizing parallel reduction in CUDA. Technical report, nVidia. http://developer.download.nvidia.com/assets/cuda/files/reduction.pdf

  16. Jones E, Oliphant T, Peterson P et al (2001) SciPy: open source scientific tools for Python. http://www.scipy.org/

  17. Khronos OpenCL Working Group: The OpenCL Specification, version 1.12 (2012). https://www.khronos.org/registry/cl/specs/opencl-1.2.pdf

  18. Klöckner A, Pinto N, Lee Y, Catanzaro B, Ivanov P, Fasih A (2012) PyCUDA and PyOpenCL: a scripting-based approach to GPU run-time code generation. Parallel Comput 38(3):157–174

    Article  Google Scholar 

  19. Kuncheva LI (2004) Combining pattern classifiers: methods and algorithms. Wiley-Interscience, Hoboken

    Book  MATH  Google Scholar 

  20. Lee VW, Kim C, Chhugani J, Deisher M, Kim D, Nguyen AD, Satish N, Smelyanskiy M, Chennupaty S, Hammarlund P, Singhal R, Dubey P (2010) Debunking the 100X GPU vs. CPU Myth: an evaluation of throughput computing on CPU and GPU. SIGARCH Comput Archit News 38(3):451–460

    Article  Google Scholar 

  21. Lerner B, Guterman H (2008) Advanced developments and applications of the fuzzy ARTMAP neural network in pattern classification. In: Jain LC, Sato-Ilic M, Virvou M, Tsihrintzis GA, Balas VE, Abeynayake C (eds) Computational intelligence paradigms studies in computational intelligence, vol 137. Springer, New York, pp 77–107

    Chapter  Google Scholar 

  22. Lichman M (2013) UCI machine learning repository. http://archive.ics.uci.edu/ml

  23. Ling RF (1974) Comparison of several algorithms for computing sample means and variances. J Am Stat Assoc 69(348):859–866

    Article  MATH  Google Scholar 

  24. Lopes L, Ribeiro B (2011) GPUMLib: an efficient open-source GPU machine learning library. Int J Comput Inf Syst Ind Manag Appl 3:2150–7988

    Google Scholar 

  25. Martínez-Zarzuela M, Díaz Pernas FJ, Díez Higuera JF, Rodríguez MA (2007) Fuzzy ART neural network parallel computing on the GPU. In: Sandoval F, Prieto A, Cabestany J, Graa M (eds) Computational and ambient intelligence. Lecture notes in computer science, vol 4507. Springer, Berlin, pp 463–470

    Chapter  Google Scholar 

  26. Martínez-Zarzuela M, Pernas F, Pablos A, Rodríguez M, Higuera J, Giralda D, Ortega D (2009) Adaptative resonance theory fuzzy networks parallel computation using CUDA. In: Cabestany J, Sandoval F, Prieto A, Corchado J (eds) Bio-inspired systems: computational and ambient intelligence. Lecture notes in computer science, vol 5517. Springer, Berlin, pp 149–156

    Chapter  Google Scholar 

  27. Murphy KP (2012) Machine learning: a probabilistic perspective (Adaptive computation and machine learning series). The MIT Press, Cambridge

    MATH  Google Scholar 

  28. Nickolls J, Buck I, Garland M, Skadron K (2008) Scalable parallel programming with cuda. Queue 6(2):40–53. doi:10.1145/1365490.1365500

    Article  Google Scholar 

  29. Pébay P (2008) Formulas for robust, one-pass parallel computation of covariances and arbitrary-order statistical moments. Sandia Report SAND2008-6212, Sandia National Laboratories

  30. Sasu L, Andonie R (2013) Bayesian ARTMAP for regression. Neural Netw 46:23–31

    Article  MATH  Google Scholar 

  31. Tax DM, van Breukelen M, Duin RP, Kittler J (2000) Combining multiple classifiers by averaging or by multiplying? Pattern Recognit 33(9):1475–1485

    Article  Google Scholar 

  32. Vigdor B, Lerner B (2007) The Bayesian ARTMAP. IEEE Trans Neural Netw 18:1628–1644

    Article  Google Scholar 

  33. Welford B (1962) Note on a method for calculating corrected sums of squares and products. Technometrics 4(3):419–420

    Article  MathSciNet  Google Scholar 

  34. Williamson J (1996) Gaussian ARTMAP: a neural network for fast incremental learning of noisy multidimensional maps. Neural Netw 9:881–897

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Electronics and Computers Department, Transilvania University of Braşov, Braşov, Romania

    István Lőrentz & Răzvan Andonie

  2. Computer Science Department, Central Washington University, Ellensburg, WA, USA

    Răzvan Andonie

  3. Mathematics and Informatics Department, Transilvania University of Braşov, Braşov, Romania

    Lucian M. Sasu

  4. Siemens Corporate Technology, Braşov, Romania

    Lucian M. Sasu

Authors
  1. István Lőrentz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Răzvan Andonie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lucian M. Sasu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to István Lőrentz.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lőrentz, I., Andonie, R. & Sasu, L.M. Parallel Bayesian ARTMAP and Its OpenCL Implementation. Neural Process Lett 47, 491–507 (2018). https://doi.org/10.1007/s11063-017-9663-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-017-9663-x

Keywords

Search

Navigation