Adaptive Resonance Theory Design in Mixed Memristive-Fuzzy Hardware

Chapter

Abstract

Fuzzification of neural networks show great promise in improving system reliability and computational efficiency. In the present work we explore the possibility of combining fuzzy inference with Adaptive Resonance Theory (ART) neural networks implemented on massively parallel hardware architectures including memristive devices. Memristive hardware holds promise to greatly reduce power requirements of such neuromorphic applications by increasing synaptic memory storage capacity and decreasing wiring length between memory storage and computational modules. Storing and updating synaptic weight values based on synaptic plasticity rules is one of the most computationally demanding operations in biologically-inspired neural networks such as Adaptive Resonance Theory (ART). Our work indicates that Fuzzy Inference Systems (FIS) can significantly improve computational efficiency. In this chapter, we introduce a novel method, based on fuzzy inference, to reduce the computational burden of a class of recurrent networks named recurrent competitive fields (RCFs). A novel algorithmic scheme is presented to more efficiently perform the synaptic learning component of ART networks in memristive hardware. RCF networks using FIS are able to learn synaptic weightswith small absolute error rates, and classify correctly. Using the FIS methodology it is possible to significantly reduce the computational complexity of the proposed memristive hardware using computationally cheaper and more robust fuzzy operators.

References

  1. 1.
    Grossberg S (1976) Adaptive pattern classification and universal recording: I parallel development and coding of neural feature detectors. Biol Cybernet 23:121–134CrossRefGoogle Scholar
  2. 2.
    Grossberg S (1976) On the development of feature detectors in the visual cortex with applications to learning and reaction-diffusion systems. Biol Cybernet 21:145–159CrossRefGoogle Scholar
  3. 3.
    Carpenter G, Grossberg S (1987) A massively parallel architecture for a self-organizing neural pattern recognition machine. Comput Vision Graph Image Process 37:54–115CrossRefGoogle Scholar
  4. 4.
    Dunyak J, Wunsch II DC (1999) Fuzzy number neural networks. Fuzzy Sets Syst 108(1):49–58CrossRefGoogle Scholar
  5. 5.
    Dunyak J, Wunsch II DC (2000) Fuzzy regression by fuzzy number neural networks. Fuzzy Sets Syst 112(3):371–380CrossRefGoogle Scholar
  6. 6.
    Xia Q et al (2009) Memristor—CMOS hybrid integrated circuits for reconfigurable logic. Nano Lett 9(10):3640–3645PubMedCrossRefGoogle Scholar
  7. 7.
    Snider GS, Amerson R, Carter D, Abdalla H, Qureshi S, Leville J, Versace M, Ames H, Patrick S, Chandler B, Gorchetchnikov A, Mingolla E (2011) Adaptive computation with memristive memory. IEEE Comput 44(2):2944–2951CrossRefGoogle Scholar
  8. 8.
    Snider GS (2011) Instar and outstar learning with memristive nanodevices. Nanotechnology 22:015201PubMedCrossRefGoogle Scholar
  9. 9.
    Snider GS (2007) Self-organized computation with unreliable, memristive nanodevices. Nanotechnology 18(36):365202CrossRefGoogle Scholar
  10. 10.
    Merrikh-Bayat F, Shouraki SB (2011) Efficient neuro-fuzzy system and its memristor crossbar-based hardware implementation. CoRR, abs/1103.1156, 2011Google Scholar
  11. 11.
    Merrikh-Bayat F, Shouraki SB (2011) Memristor crossbar-based hardware implementation of fuzzy membership functions. CoRR, abs/1009.0896, 2011Google Scholar
  12. 12.
    Klimo M, Such O (2011) Memristors can implement fuzzy logic. CoRR, abs/1110.2074, 2011Google Scholar
  13. 13.
    Merrikh-Bayat F, Shouraki SB, Rohani A (2011) Memristor crossbar-based hardware implementation of the IDS method. IEEE Trans Fuzzy Syst 19(6):1083–1096 (art. no. 5893932)CrossRefGoogle Scholar
  14. 14.
    Zhong QS, Yu Y-B, Yu J-B (2010) Fuzzy modeling and impulsive control of a memristor-based chaotic system. Chin Phys Lett 27(2) (art. no. 020501)Google Scholar
  15. 15.
    Wunsch II DC, Caudell TP, Capps D, Marks II RJ, Falk RA (1993, July) An optoelectronic implementation of the adaptive resonance neural network. IEEE Trans Neural Networks 4(4):673–684CrossRefGoogle Scholar
  16. 16.
    Kogge P (2011) The tops in FLOPS. Spectrum, IEEE, 2011Google Scholar
  17. 17.
    Dunyak J, Wunsch D, Saad IW (1999, June) A theory of independent fuzzy probabilities for system reliability. IEEE Trans Fuzzy Syst 7(3):286–294CrossRefGoogle Scholar
  18. 18.
    Grossberg S (1982) Contour enhancement, short term memory, and constancies in reverberating neural networks. Studies of mind and brain (Chapter 8). Kluwer/Reidel Press, BostonGoogle Scholar
  19. 19.
    Sugeno M (1985) Industrial applications of fuzzy control. Elsevier, OxfordGoogle Scholar
  20. 20.
    Chua LO (1971) Memristor—the missing circuit element. IEEE Trans Circuit Theor 18(5):507–519CrossRefGoogle Scholar
  21. 21.
    Strukov DB, Snider GS, Stewart DR, Williams RS (2008) The missing memristor found. Nature 453:80–83PubMedCrossRefGoogle Scholar
  22. 22.
    Chua LO, Kang SM (1976) Memristive devices and systems. Proc IEEE 64(2):209–223CrossRefGoogle Scholar
  23. 23.
    Chua LO (2003) Nonlinear circuit foundations for nanodevices, part I: the four-element torus. Proc IEEE 91(11):1830–1859CrossRefGoogle Scholar
  24. 24.
    Versace M, Chandler B (2011) MoNETA: a mind made from memristors. IEEE Spectrum, December 2011Google Scholar
  25. 25.
    Jo SH, Chang T, Ebong I, Bhadviya BB, Mazumder P, Lu W (2010) Nanoscale memristor device as synapse in neuromorphic systems. Nano Lett 10:1297–1301PubMedCrossRefGoogle Scholar
  26. 26.
    Pazienza G, Kozma R (2011) Memristor as an archetype of dynamic data-driven systems and applications to sensor networks. Dynamic data driven application systems, DDDAS 2011, Tsukuba, Japan, June 2–3 2011Google Scholar
  27. 27.
    Bezdek JC, Keller J, Krisnapuram R, Pal N (2005) Fuzzy models and algorithms for pattern recognition and image processing. Springer, New YorkGoogle Scholar
  28. 28.
    Kosko B (1999) The fuzzy future: from society and science to heaven in a chip. Harmony Books, New YorkGoogle Scholar
  29. 29.
    Castillo O, Melin P, Ross OM, Cruz RS, Pedrycz W, Kacprzyk J (2007) Theoretical advances and applications of fuzzy logic and soft computing. Springer, HeidelbergCrossRefGoogle Scholar
  30. 30.
    Kreinovich V, Nguyen H-T, Yam Y (2000) Fuzzy systems are universal approximators for a smooth function and its derivatives. Int J Intelligent Syst 15(6):565–574CrossRefGoogle Scholar
  31. 31.
    Kosko B (1994) Fuzzy systems are universal approximators. IEEE Trans Comput 44(11):1329–1333CrossRefGoogle Scholar
  32. 32.
    Carpenter G, Grossberg S, Markuzon N, Reynolds J, Rosen D (1992) Fuzzy ARTMAP: a neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Trans Neural Networks 3:698–713CrossRefGoogle Scholar
  33. 33.
    Anagnostopoulos G, Georgiopoulos M (2001) Ellipsoid ART and ARTMAP for incremental unsupervised and supervised learning. In: Proceedings of the international joint conference on neural networks, vol 2, pp 1221–1226, 2001Google Scholar
  34. 34.
    Carpenter G (2003) Default ARTMAP. In: Proceedings of the international conference on neural networks, pp 1396–1401, 2003Google Scholar
  35. 35.
    Xu R, Wunsch D (2011) BARTMAP: a viable structure for biclustering. Neural Networks 24(7):709–716PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2012

Authors and Affiliations

  • Max Versace
    • 1
  • Robert T. Kozma
    • 2
  • Donald C. Wunsch
    • 3
  1. 1.Neuromorphics LabBoston UniversityBostonUSA
  2. 2.Department of MathematicsSUNY Stony BrookStony BrookUSA
  3. 3.Applied Computational Intelligence LabMissouri University of Science & TechnologyRollaUSA

Personalised recommendations