Advertisement

Applied Intelligence

, Volume 48, Issue 11, pp 4174–4191 | Cite as

A novel adaptive learning algorithm for low-dimensional feature space using memristor-crossbar implementation and on-chip training

  • Sajad Haghzad Klidbary
  • Saeed Bagheri Shouraki
Article

Abstract

Proposing an efficient algorithm with an appropriate hardware implementation has always been an interesting and a rather challenging field of research in Artificial Intelligence (AI). Fuzzy logic is one of the techniques that can be used for accurate and high-speed modeling as well as controlling complex and nonlinear systems. The “defuzzification” process during the test phase as well as the repetitive processes in order to find the optimal parameters during the training phase, lead to some serious limitations in real-time applications and hardware implementation of these algorithms. The proposed algorithm employs Ink Drop Spread (IDS) concept to mimic the functionality of human brain. In this algorithm, learning is based on the distance between training data and the “learning plane”. Unlike previous algorithms, the new one does not need to partition nor the input space neither the calculation of IDS plane features. Besides, the output is obtained without using the optimization methods. The proposed algorithm is a numerical foundation that does not encounter a processing problem and lack of memory in dealing with different datasets consisting of a large number of samples. This algorithm can be efficiently implemented on memristor crossbar/CMOS hardware platform in terms of area and speed. This hardware has the ability to learn and adapt to the environment regardless of a host system (on-chip learning capability). Finally, to verify the performance of the proposed algorithm, it has been compared to ALM, RBF and PNN algorithms which have a similar functionality.

Keywords

Ink Drop Spread (IDS) operator Radial basis function (RBF) Probabilistic neural network (PNN) Memristor-crossbar 

Notes

Acknowledgments

All the experiments and ideas of this research work have been developed in Artificial Creatures Lab (ACL), Electrical Engineering Department, Sharif University of Technology, Tehran, IRAN. The authors would like to thank Nasim Bagheri Shouraki for her useful and insightful comments. This research is partially supported by Iran National Science Foundation (INSF) grant number 96000943.

Compliance with Ethical Standards

Conflict of interests

The authors declare that they have no conflict of interest.

References

  1. 1.
    Bosque G, del Campo I, Echanobe J (2014) Fuzzy systems, neural networks and neuro-fuzzy systems: a vision on their hardware implementation and platforms over two decades. Eng Appl Artif Intell 32:283–331CrossRefGoogle Scholar
  2. 2.
    Soudry D et al (2015) Memristor-based multilayer neural networks with online gradient descent training. IEEE Trans Neural Netw Learn Syst 26(10):2408–2421MathSciNetCrossRefGoogle Scholar
  3. 3.
    Paliwal M, Kumar UA (2009) Neural networks and statistical techniques: a review of applications. Exp Syst Appl 36(1):2–17CrossRefGoogle Scholar
  4. 4.
    Mamdani EH (1974) Application of fuzzy algorithms for control of simple dynamic plant. In: Proceedings of the institution of electrical engineers. IETGoogle Scholar
  5. 5.
    Kulkarni AD, Cavanaugh CD (2000) Fuzzy neural network models for classification. Appl Intell 12 (3):207–215CrossRefGoogle Scholar
  6. 6.
    Gomperts A, Ukil A, Zurfluh F (2011) Development and implementation of parameterized FPGA-based general purpose neural networks for online applications. IEEE Trans Indust Inf 7(1):78–89CrossRefGoogle Scholar
  7. 7.
    Zhang GP (2000) Neural networks for classification: a survey. IEEE Trans Syst Man Cybern Part C (Appl Rev) 30(4):451– 462CrossRefGoogle Scholar
  8. 8.
    Misra J, Saha I (2010) Artificial neural networks in hardware: a survey of two decades of progress. Neurocomputing 74(1–3):239–255CrossRefGoogle Scholar
  9. 9.
    Savich AW, Moussa M, Areibi S (2007) The impact of arithmetic representation on implementing MLP-BP on FPGAs: a study. IEEE Trans Neural Netw 18(1):240–252CrossRefGoogle Scholar
  10. 10.
    Zadeh LA (1996) Fuzzy sets, in fuzzy sets, fuzzy logic, and fuzzy systems: selected papers by Lotfi A Zadeh. World Scientific 6:394–432Google Scholar
  11. 11.
    Zadeh LA (1973) Outline of a new approach to the analysis of complex systems and decision processes. IEEE Trans Syst Man Cybern 1973(1):28–44MathSciNetCrossRefGoogle Scholar
  12. 12.
    Zadeh LA (1994) Soft computing and fuzzy logic. IEEE Softw 11(6):48–56CrossRefGoogle Scholar
  13. 13.
    Shouraki SB (2000) A novel fuzzy approach to modeling and control and its hardware implementation based on brain functionality and specificationsGoogle Scholar
  14. 14.
    Shouraki SB, Honda N (1999) Simulation of brain learning process through a novel fuzzy hardware approach. In: Proceedings of 1999 IEEE international conference on systems, man, and cybernetics, IEEE SMC’99. IEEEGoogle Scholar
  15. 15.
    Shouraki SB, Honda N, Yuasa G (1999) Fuzzy interpretation of human intelligence. Int J Uncertain Fuzziness Knowl-Based Syst 7(4):407–414CrossRefGoogle Scholar
  16. 16.
    Shouraki SB, Honda N (1999) Recursive fuzzy modeling based on fuzzy interpolation. JACIII 3(2):114–125CrossRefGoogle Scholar
  17. 17.
    Bahrpeyma F, Zakerolhoseini A, Haghighi H (2015) Using IDS fitted Q to develop a real-time adaptive controller for dynamic resource provisioning in cloud’s virtualized environment. Appl Soft Comput 26:285–298CrossRefGoogle Scholar
  18. 18.
    Sakurai Y (2005) A study of the learning control method using PBALM-a nonlinear modeling method. PhD, The University of Electro-Communications, TokyoGoogle Scholar
  19. 19.
    Murakami M (2008) Practicality of modeling systems using the IDS method: Performance investigation and hardware implementationGoogle Scholar
  20. 20.
    Firouzi M, Shouraki SB, Conradt J (2014) Sensorimotor control learning using a new adaptive spiking neuro-fuzzy machine spike-IDS and STDP. In: International conference on artificial neural networks. SpringerGoogle Scholar
  21. 21.
    Merrikh-Bayat F, Merrikh-Bayat F, Shouraki SB (2014) The neuro-fuzzy computing system with the capacity of implementation on a memristor crossbar and optimization-free hardware training. IEEE Trans Fuzzy Syst 22(5):1272–1287CrossRefGoogle Scholar
  22. 22.
    Ghorbani MJ, Choudhry MA, Feliachi A (2014) Distributed multi-agent based load shedding in power distribution systems. In: 2014 IEEE 27th Canadian conference on electrical and computer engineering (CCECE). IEEEGoogle Scholar
  23. 23.
    Shahdi SA, Shouraki SB (2002) Supervised active learning method as an intelligent linguistic controller and its hardware implementation. In: IASTEDGoogle Scholar
  24. 24.
    Shouraki SB, Honda N (1998) Fuzzy controller design by an active learning method. In: 31st symposium of intelligent controlGoogle Scholar
  25. 25.
    Javadian M, Shouraki SB, Kourabbaslou SS (2017) A novel density-based fuzzy clustering algorithm for low dimensional feature space. Fuzzy Sets Syst 318:34–55MathSciNetCrossRefGoogle Scholar
  26. 26.
    Firouzi M, Shouraki SB, Afrakoti IEP (2014) Pattern analysis by active learning method classifier. J Intell Fuzzy Syst 26(1):49–62MathSciNetzbMATHGoogle Scholar
  27. 27.
    Klidbary SH et al (2017) Outlier robust fuzzy active learning method (ALM). In: IEEE, 2017, 7th international conference on computer and knowledge engineering (ICCKE), pp 347–352Google Scholar
  28. 28.
    Klidbary SH, Shouraki SB, Afrakoti IEP (2016) Fast IDS computing system method and its memristor crossbar-based hardware implementation. arXiv:1602.06787
  29. 29.
    Chen T, Chen H (1995) Approximation capability to functions of several variables, nonlinear functionals, and operators by radial basis function neural networks. IEEE Trans Neural Netw 6(4):904–910CrossRefGoogle Scholar
  30. 30.
    Wedding DK, Cios KJ (1996) Time series forecasting by combining RBF networks, certainty factors, and the Box-Jenkins model. Neurocomputing 10(2):149–168CrossRefGoogle Scholar
  31. 31.
    Nekoukar V, Beheshti MTH (2010) A local linear radial basis function neural network for financial time-series forecasting. Appl Intell 33(3):352–356CrossRefGoogle Scholar
  32. 32.
    Rouhani M, Javan DS (2016) Two fast and accurate heuristic RBF learning rules for data classification. Neural Netw 75:150–161CrossRefGoogle Scholar
  33. 33.
    Park B-J, Pedrycz W, Oh S-K (2010) Polynomial-based radial basis function neural networks (p-RBF NNs) and their application to pattern classification. Appl Intell 32(1):27–46CrossRefGoogle Scholar
  34. 34.
    Er MJ et al (2002) Face recognition with radial basis function (RBF) neural networks. IEEE Trans Neural Netw 13(3):697–710CrossRefGoogle Scholar
  35. 35.
    Maglogiannis I, Zafiropoulos E, Anagnostopoulos I (2009) An intelligent system for automated breast cancer diagnosis and prognosis using SVM based classifiers. Appl Intell 30(1):24–36CrossRefGoogle Scholar
  36. 36.
    Mantzaris D, Anastassopoulos G, Adamopoulos A (2011) Genetic algorithm pruning of probabilistic neural networks in medical disease estimation. Neural Netw 24(8):831–835CrossRefGoogle Scholar
  37. 37.
    Rafiei MH, Adeli H (2017) A new neural dynamic classification algorithm. IEEE Trans Neural Netw Learn Syst 28(12):3074– 3083MathSciNetCrossRefGoogle Scholar
  38. 38.
    Shang L et al (2006) Palmprint recognition using fastICA algorithm and radial basis probabilistic neural network. Neurocomputing 69(13–15):1782–1786CrossRefGoogle Scholar
  39. 39.
    Sun X-Y et al (2016) Improved probabilistic neural network PNN and its application to defect recognition in rock bolts. Int J Mach Learn Cybern 7(5):909–919CrossRefGoogle Scholar
  40. 40.
    Rutkowski L (2004) Adaptive probabilistic neural networks for pattern classification in time-varying environment. IEEE Trans Neural Netw 15(4):811–827CrossRefGoogle Scholar
  41. 41.
    Murakami M, Honda N (2007) A study on the modeling ability of the IDS method: a soft computing technique using pattern-based information processing. Int J Approx Reason 45(3):470– 487CrossRefGoogle Scholar
  42. 42.
    Murakami M, Honda N (2006) A basic constructive algorithm for the IDS method. In: SCIS & ISIS SCIS & ISIS. 2006. Japan Society for Fuzzy Theory and Intelligent InformaticsGoogle Scholar
  43. 43.
    Chua L (1971) Memristor-the missing circuit element. IEEE Trans Circ Theory 18(5):507–519CrossRefGoogle Scholar
  44. 44.
    Chua LO, Kang SM (1976) Memristive devices and systems. Proc IEEE 64(2):209–223MathSciNetCrossRefGoogle Scholar
  45. 45.
    Strukov DB et al (2008) The missing memristor found. Nature 453(7191):80–83CrossRefGoogle Scholar
  46. 46.
    Snider G et al (2011) From synapses to circuitry: Using memristive memory to explore the electronic brain. Computer 44(2):21–28CrossRefGoogle Scholar
  47. 47.
    Li T et al (2016) A spintronic memristor-based neural network with radial basis function for robotic manipulator control implementation. IEEE Trans Syst Man Cybern Syst 46(4):582–588CrossRefGoogle Scholar
  48. 48.
    Waser R, Aono M (2007) Nanoionics-based resistive switching memories. Nat Mater 6(11):833–840CrossRefGoogle Scholar
  49. 49.
    Kuekes P (2008) Material implication: digital logic with memristors. In: Memristor and memristive systems symposiumGoogle Scholar
  50. 50.
    Raja T, Mourad S (2010) Digital logic implementation in memristor-based crossbars-a tutorial. In: Fifth IEEE international symposium on electronic design, test and application, DELTA’10. 2010. IEEEGoogle Scholar
  51. 51.
    Pérez-Carrasco JA et al (2010) On neuromorphic spiking architectures for asynchronous STDP memristive systems. In: Proceedings of 2010 IEEE international symposium on circuits and systems (ISCAS). IEEEGoogle Scholar
  52. 52.
    Pershin YV, La Fontaine S, Di Ventra M (2009) Memristive model of amoeba learning. Phys Rev E 80 (2):021926CrossRefGoogle Scholar
  53. 53.
    Li T et al (2016) An improved design of RBF neural network control algorithm based on spintronic memristor crossbar array. Neural Comput Appl 28:1–8CrossRefGoogle Scholar
  54. 54.
    Kolka Z, Biolek D, Biolkova V (2015) Improved model of TiO2 memristor. Radioengineering 24 (2):378–383CrossRefGoogle Scholar
  55. 55.
    Naous R, Al-Shedivat M, Salama KN (2016) Stochasticity modeling in memristors. IEEE Trans Nanotechnol 15(1):15–28CrossRefGoogle Scholar
  56. 56.
    Biolek D et al (2015) Reliable modeling of ideal generic memristors via state-space transformation. Radioengineering 24(2):393–407CrossRefGoogle Scholar
  57. 57.
    Hu M et al (2014) Memristor crossbar-based neuromorphic computing system: a case study. IEEE Trans Neural Netw Learn Syst 25(10):1864–1878CrossRefGoogle Scholar
  58. 58.
    Starzyk JA (2014) Memristor crossbar architecture for synchronous neural networks. IEEE Trans Circ Syst I: Reg Papers 61(8):2390–2401Google Scholar
  59. 59.
    Afrakoti IEP et al (2017) Using a memristor crossbar structure to implement a novel adaptive real-time fuzzy modeling algorithm. Fuzzy Sets Syst 307:115–128MathSciNetCrossRefGoogle Scholar
  60. 60.
    Merrikh-Bayat F, Shouraki SB (2013) Memristive neuro-fuzzy system. IEEE Trans Cybern 43(1):269–285CrossRefGoogle Scholar
  61. 61.
    Versace M, Kozma RT, Wunsch DC (2012) Adaptive resonance theory design in mixed memristive-fuzzy hardware. In: Advances in neuromorphic memristor science and applications. Springer, pp 133–153Google Scholar
  62. 62.
    Park Y, Lee J-S (2017) Artificial synapses with short-and long-term memory for spiking neural networks based on renewable materials. ACS Nano 11(9):8962–8969CrossRefGoogle Scholar
  63. 63.
    Prezioso M et al (2016) Self-adaptive spike-time-dependent plasticity of metal-oxide memristors. Scientific Rep 6:21331CrossRefGoogle Scholar
  64. 64.
    Merrikh-Bayat F, Shouraki SB, Rohani A (2011) Memristor crossbar-based hardware implementation of the IDS method. IEEE Trans Fuzzy Syst 19(6):1083–1096CrossRefGoogle Scholar
  65. 65.
    Afrakoti IEP, Shouraki SB, Haghighat B (2014) An optimal hardware implementation for active learning method based on memristor crossbar structures. IEEE Syst J 8(4):1190–1199CrossRefGoogle Scholar
  66. 66.
    Hwang J-N et al (1994) Regression modeling in back-propagation and projection pursuit learning. IEEE Trans Neural Netw 5(3):342–353CrossRefGoogle Scholar
  67. 67.
    Kwok T-Y, Yeung D-Y (1997) Objective functions for training new hidden units in constructive neural networks. IEEE Trans Neural Netw 8(5):1131–1148CrossRefGoogle Scholar
  68. 68.
    Ma L, Khorasani K (2005) Constructive feedforward neural networks using Hermite polynomial activation functions. IEEE Trans Neural Netw 16(4):821–833CrossRefGoogle Scholar
  69. 69.
    Jang J-S (1993) ANFIS: adaptive-network-based fuzzy inference system. IEEE Trans Syst Man Cybern 23 (3):665–685CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Research Group for Brain Simulation and Cognitive Science, Artificial Creatures Laboratory (ACL), Department of Electrical EngineeringSharif University of TechnologyTehranIran

Personalised recommendations