Skip to main content

Advertisement

SpringerLink
Log in

A novel adaptive learning algorithm for low-dimensional feature space using memristor-crossbar implementation and on-chip training

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Proposing an efficient algorithm with an appropriate hardware implementation has always been an interesting and a rather challenging field of research in Artificial Intelligence (AI). Fuzzy logic is one of the techniques that can be used for accurate and high-speed modeling as well as controlling complex and nonlinear systems. The “defuzzification” process during the test phase as well as the repetitive processes in order to find the optimal parameters during the training phase, lead to some serious limitations in real-time applications and hardware implementation of these algorithms. The proposed algorithm employs Ink Drop Spread (IDS) concept to mimic the functionality of human brain. In this algorithm, learning is based on the distance between training data and the “learning plane”. Unlike previous algorithms, the new one does not need to partition nor the input space neither the calculation of IDS plane features. Besides, the output is obtained without using the optimization methods. The proposed algorithm is a numerical foundation that does not encounter a processing problem and lack of memory in dealing with different datasets consisting of a large number of samples. This algorithm can be efficiently implemented on memristor crossbar/CMOS hardware platform in terms of area and speed. This hardware has the ability to learn and adapt to the environment regardless of a host system (on-chip learning capability). Finally, to verify the performance of the proposed algorithm, it has been compared to ALM, RBF and PNN algorithms which have a similar functionality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Bosque G, del Campo I, Echanobe J (2014) Fuzzy systems, neural networks and neuro-fuzzy systems: a vision on their hardware implementation and platforms over two decades. Eng Appl Artif Intell 32:283–331

    Article  Google Scholar 

  2. Soudry D et al (2015) Memristor-based multilayer neural networks with online gradient descent training. IEEE Trans Neural Netw Learn Syst 26(10):2408–2421

    Article  MathSciNet  Google Scholar 

  3. Paliwal M, Kumar UA (2009) Neural networks and statistical techniques: a review of applications. Exp Syst Appl 36(1):2–17

    Article  Google Scholar 

  4. Mamdani EH (1974) Application of fuzzy algorithms for control of simple dynamic plant. In: Proceedings of the institution of electrical engineers. IET

  5. Kulkarni AD, Cavanaugh CD (2000) Fuzzy neural network models for classification. Appl Intell 12 (3):207–215

    Article  Google Scholar 

  6. Gomperts A, Ukil A, Zurfluh F (2011) Development and implementation of parameterized FPGA-based general purpose neural networks for online applications. IEEE Trans Indust Inf 7(1):78–89

    Article  Google Scholar 

  7. Zhang GP (2000) Neural networks for classification: a survey. IEEE Trans Syst Man Cybern Part C (Appl Rev) 30(4):451– 462

    Article  Google Scholar 

  8. Misra J, Saha I (2010) Artificial neural networks in hardware: a survey of two decades of progress. Neurocomputing 74(1–3):239–255

    Article  Google Scholar 

  9. Savich AW, Moussa M, Areibi S (2007) The impact of arithmetic representation on implementing MLP-BP on FPGAs: a study. IEEE Trans Neural Netw 18(1):240–252

    Article  Google Scholar 

  10. Zadeh LA (1996) Fuzzy sets, in fuzzy sets, fuzzy logic, and fuzzy systems: selected papers by Lotfi A Zadeh. World Scientific 6:394–432

    Google Scholar 

  11. Zadeh LA (1973) Outline of a new approach to the analysis of complex systems and decision processes. IEEE Trans Syst Man Cybern 1973(1):28–44

    Article  MathSciNet  Google Scholar 

  12. Zadeh LA (1994) Soft computing and fuzzy logic. IEEE Softw 11(6):48–56

    Article  Google Scholar 

  13. Shouraki SB (2000) A novel fuzzy approach to modeling and control and its hardware implementation based on brain functionality and specifications

  14. Shouraki SB, Honda N (1999) Simulation of brain learning process through a novel fuzzy hardware approach. In: Proceedings of 1999 IEEE international conference on systems, man, and cybernetics, IEEE SMC’99. IEEE

  15. Shouraki SB, Honda N, Yuasa G (1999) Fuzzy interpretation of human intelligence. Int J Uncertain Fuzziness Knowl-Based Syst 7(4):407–414

    Article  Google Scholar 

  16. Shouraki SB, Honda N (1999) Recursive fuzzy modeling based on fuzzy interpolation. JACIII 3(2):114–125

    Article  Google Scholar 

  17. Bahrpeyma F, Zakerolhoseini A, Haghighi H (2015) Using IDS fitted Q to develop a real-time adaptive controller for dynamic resource provisioning in cloud’s virtualized environment. Appl Soft Comput 26:285–298

    Article  Google Scholar 

  18. Sakurai Y (2005) A study of the learning control method using PBALM-a nonlinear modeling method. PhD, The University of Electro-Communications, Tokyo

  19. Murakami M (2008) Practicality of modeling systems using the IDS method: Performance investigation and hardware implementation

  20. Firouzi M, Shouraki SB, Conradt J (2014) Sensorimotor control learning using a new adaptive spiking neuro-fuzzy machine spike-IDS and STDP. In: International conference on artificial neural networks. Springer

  21. Merrikh-Bayat F, Merrikh-Bayat F, Shouraki SB (2014) The neuro-fuzzy computing system with the capacity of implementation on a memristor crossbar and optimization-free hardware training. IEEE Trans Fuzzy Syst 22(5):1272–1287

    Article  Google Scholar 

  22. Ghorbani MJ, Choudhry MA, Feliachi A (2014) Distributed multi-agent based load shedding in power distribution systems. In: 2014 IEEE 27th Canadian conference on electrical and computer engineering (CCECE). IEEE

  23. Shahdi SA, Shouraki SB (2002) Supervised active learning method as an intelligent linguistic controller and its hardware implementation. In: IASTED

  24. Shouraki SB, Honda N (1998) Fuzzy controller design by an active learning method. In: 31st symposium of intelligent control

  25. Javadian M, Shouraki SB, Kourabbaslou SS (2017) A novel density-based fuzzy clustering algorithm for low dimensional feature space. Fuzzy Sets Syst 318:34–55

    Article  MathSciNet  Google Scholar 

  26. Firouzi M, Shouraki SB, Afrakoti IEP (2014) Pattern analysis by active learning method classifier. J Intell Fuzzy Syst 26(1):49–62

    MathSciNet  MATH  Google Scholar 

  27. Klidbary SH et al (2017) Outlier robust fuzzy active learning method (ALM). In: IEEE, 2017, 7th international conference on computer and knowledge engineering (ICCKE), pp 347–352

  28. Klidbary SH, Shouraki SB, Afrakoti IEP (2016) Fast IDS computing system method and its memristor crossbar-based hardware implementation. arXiv:1602.06787

  29. Chen T, Chen H (1995) Approximation capability to functions of several variables, nonlinear functionals, and operators by radial basis function neural networks. IEEE Trans Neural Netw 6(4):904–910

    Article  Google Scholar 

  30. Wedding DK, Cios KJ (1996) Time series forecasting by combining RBF networks, certainty factors, and the Box-Jenkins model. Neurocomputing 10(2):149–168

    Article  Google Scholar 

  31. Nekoukar V, Beheshti MTH (2010) A local linear radial basis function neural network for financial time-series forecasting. Appl Intell 33(3):352–356

    Article  Google Scholar 

  32. Rouhani M, Javan DS (2016) Two fast and accurate heuristic RBF learning rules for data classification. Neural Netw 75:150–161

    Article  Google Scholar 

  33. Park B-J, Pedrycz W, Oh S-K (2010) Polynomial-based radial basis function neural networks (p-RBF NNs) and their application to pattern classification. Appl Intell 32(1):27–46

    Article  Google Scholar 

  34. Er MJ et al (2002) Face recognition with radial basis function (RBF) neural networks. IEEE Trans Neural Netw 13(3):697–710

    Article  Google Scholar 

  35. Maglogiannis I, Zafiropoulos E, Anagnostopoulos I (2009) An intelligent system for automated breast cancer diagnosis and prognosis using SVM based classifiers. Appl Intell 30(1):24–36

    Article  Google Scholar 

  36. Mantzaris D, Anastassopoulos G, Adamopoulos A (2011) Genetic algorithm pruning of probabilistic neural networks in medical disease estimation. Neural Netw 24(8):831–835

    Article  Google Scholar 

  37. Rafiei MH, Adeli H (2017) A new neural dynamic classification algorithm. IEEE Trans Neural Netw Learn Syst 28(12):3074– 3083

    Article  MathSciNet  Google Scholar 

  38. Shang L et al (2006) Palmprint recognition using fastICA algorithm and radial basis probabilistic neural network. Neurocomputing 69(13–15):1782–1786

    Article  Google Scholar 

  39. Sun X-Y et al (2016) Improved probabilistic neural network PNN and its application to defect recognition in rock bolts. Int J Mach Learn Cybern 7(5):909–919

    Article  Google Scholar 

  40. Rutkowski L (2004) Adaptive probabilistic neural networks for pattern classification in time-varying environment. IEEE Trans Neural Netw 15(4):811–827

    Article  Google Scholar 

  41. Murakami M, Honda N (2007) A study on the modeling ability of the IDS method: a soft computing technique using pattern-based information processing. Int J Approx Reason 45(3):470– 487

    Article  Google Scholar 

  42. Murakami M, Honda N (2006) A basic constructive algorithm for the IDS method. In: SCIS & ISIS SCIS & ISIS. 2006. Japan Society for Fuzzy Theory and Intelligent Informatics

  43. Chua L (1971) Memristor-the missing circuit element. IEEE Trans Circ Theory 18(5):507–519

    Article  Google Scholar 

  44. Chua LO, Kang SM (1976) Memristive devices and systems. Proc IEEE 64(2):209–223

    Article  MathSciNet  Google Scholar 

  45. Strukov DB et al (2008) The missing memristor found. Nature 453(7191):80–83

    Article  Google Scholar 

  46. Snider G et al (2011) From synapses to circuitry: Using memristive memory to explore the electronic brain. Computer 44(2):21–28

    Article  Google Scholar 

  47. Li T et al (2016) A spintronic memristor-based neural network with radial basis function for robotic manipulator control implementation. IEEE Trans Syst Man Cybern Syst 46(4):582–588

    Article  Google Scholar 

  48. Waser R, Aono M (2007) Nanoionics-based resistive switching memories. Nat Mater 6(11):833–840

    Article  Google Scholar 

  49. Kuekes P (2008) Material implication: digital logic with memristors. In: Memristor and memristive systems symposium

  50. Raja T, Mourad S (2010) Digital logic implementation in memristor-based crossbars-a tutorial. In: Fifth IEEE international symposium on electronic design, test and application, DELTA’10. 2010. IEEE

  51. Pérez-Carrasco JA et al (2010) On neuromorphic spiking architectures for asynchronous STDP memristive systems. In: Proceedings of 2010 IEEE international symposium on circuits and systems (ISCAS). IEEE

  52. Pershin YV, La Fontaine S, Di Ventra M (2009) Memristive model of amoeba learning. Phys Rev E 80 (2):021926

    Article  Google Scholar 

  53. Li T et al (2016) An improved design of RBF neural network control algorithm based on spintronic memristor crossbar array. Neural Comput Appl 28:1–8

    Article  Google Scholar 

  54. Kolka Z, Biolek D, Biolkova V (2015) Improved model of TiO2 memristor. Radioengineering 24 (2):378–383

    Article  Google Scholar 

  55. Naous R, Al-Shedivat M, Salama KN (2016) Stochasticity modeling in memristors. IEEE Trans Nanotechnol 15(1):15–28

    Article  Google Scholar 

  56. Biolek D et al (2015) Reliable modeling of ideal generic memristors via state-space transformation. Radioengineering 24(2):393–407

    Article  Google Scholar 

  57. Hu M et al (2014) Memristor crossbar-based neuromorphic computing system: a case study. IEEE Trans Neural Netw Learn Syst 25(10):1864–1878

    Article  Google Scholar 

  58. Starzyk JA (2014) Memristor crossbar architecture for synchronous neural networks. IEEE Trans Circ Syst I: Reg Papers 61(8):2390–2401

    Google Scholar 

  59. Afrakoti IEP et al (2017) Using a memristor crossbar structure to implement a novel adaptive real-time fuzzy modeling algorithm. Fuzzy Sets Syst 307:115–128

    Article  MathSciNet  Google Scholar 

  60. Merrikh-Bayat F, Shouraki SB (2013) Memristive neuro-fuzzy system. IEEE Trans Cybern 43(1):269–285

    Article  Google Scholar 

  61. Versace M, Kozma RT, Wunsch DC (2012) Adaptive resonance theory design in mixed memristive-fuzzy hardware. In: Advances in neuromorphic memristor science and applications. Springer, pp 133–153

  62. Park Y, Lee J-S (2017) Artificial synapses with short-and long-term memory for spiking neural networks based on renewable materials. ACS Nano 11(9):8962–8969

    Article  Google Scholar 

  63. Prezioso M et al (2016) Self-adaptive spike-time-dependent plasticity of metal-oxide memristors. Scientific Rep 6:21331

    Article  Google Scholar 

  64. Merrikh-Bayat F, Shouraki SB, Rohani A (2011) Memristor crossbar-based hardware implementation of the IDS method. IEEE Trans Fuzzy Syst 19(6):1083–1096

    Article  Google Scholar 

  65. Afrakoti IEP, Shouraki SB, Haghighat B (2014) An optimal hardware implementation for active learning method based on memristor crossbar structures. IEEE Syst J 8(4):1190–1199

    Article  Google Scholar 

  66. Hwang J-N et al (1994) Regression modeling in back-propagation and projection pursuit learning. IEEE Trans Neural Netw 5(3):342–353

    Article  Google Scholar 

  67. Kwok T-Y, Yeung D-Y (1997) Objective functions for training new hidden units in constructive neural networks. IEEE Trans Neural Netw 8(5):1131–1148

    Article  Google Scholar 

  68. Ma L, Khorasani K (2005) Constructive feedforward neural networks using Hermite polynomial activation functions. IEEE Trans Neural Netw 16(4):821–833

    Article  Google Scholar 

  69. Jang J-S (1993) ANFIS: adaptive-network-based fuzzy inference system. IEEE Trans Syst Man Cybern 23 (3):665–685

    Article  Google Scholar 

Download references

Acknowledgments

All the experiments and ideas of this research work have been developed in Artificial Creatures Lab (ACL), Electrical Engineering Department, Sharif University of Technology, Tehran, IRAN. The authors would like to thank Nasim Bagheri Shouraki for her useful and insightful comments. This research is partially supported by Iran National Science Foundation (INSF) grant number 96000943.

Author information

Authors and Affiliations

  1. Research Group for Brain Simulation and Cognitive Science, Artificial Creatures Laboratory (ACL), Department of Electrical Engineering, Sharif University of Technology, Azadi Avenue, Tehran, 11365-11155, Iran

    Sajad Haghzad Klidbary & Saeed Bagheri Shouraki

Authors
  1. Sajad Haghzad Klidbary
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Saeed Bagheri Shouraki
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sajad Haghzad Klidbary.

Ethics declarations

Conflict of interests

The authors declare that they have no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Klidbary, S.H., Shouraki, S.B. A novel adaptive learning algorithm for low-dimensional feature space using memristor-crossbar implementation and on-chip training. Appl Intell 48, 4174–4191 (2018). https://doi.org/10.1007/s10489-018-1202-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-018-1202-6

Keywords

Search

Navigation