Advertisement

Digital hardware realization of a novel adaptive ink drop spread operator and its application in modeling and classification and on-chip training

  • Sajad Haghzad KlidbaryEmail author
  • Saeed Bagheri Shouraki
  • Bernabe Linares-Barranco
Original Article
  • 33 Downloads

Abstract

In artificial intelligence (AI), proposing an efficient algorithm with an appropriate hardware implementation has always been a challenge because of the well-accepted fact that AI hardware implementations should ideally be comparable to biological systems in terms of hardware area. Active learning method (ALM) is a fuzzy learning algorithm inspired by human brain computations. Unlike traditional algorithms, which employ complicated computations, ALM tries to model human brain computations using qualitative and behavioral descriptions of the problem. The main computational engine in ALM is the ink drop spread (IDS) operator, but this operator imposes high memory requirements and computational costs, making the ALM algorithm and its hardware implementation unsuitable for some of the applications. This paper proposes an adaptive alternative method for implementing the IDS operator; a method which results in a marked reduction in the algorithm’s computational complexity and in the amount of memory required and hardware. To check its validity and performance, the method was used to carry out modeling and pattern classification tasks. This paper used challenging and real-world datasets and compared with well-known algorithms (adaptive neuro-fuzzy inference system and multi-layer perceptron) in software simulation and hardware implementation. Compared to traditional implementations of the ALM algorithm and other learning algorithms, the proposed FPGA implementation offers higher speed, less hardware, and improved performance, thus facilitating real-time application. Our ultimate goal in this paper was to present a hardware implementation with an on-chip training that allows it to adapt to its environment without dependency on the host system (on-chip learning).

Keywords

Soft computing Ink drop spread (IDS) operator Fuzzy modeling Pattern classification Field-programmable gate array (FPGA) implementation 

Notes

Acknowledgements

The authors would like to thank Mohsen Firouzi and Menoua Keshishian for their generous contribution to our analysis. This work was partially supported by the INSF (Iran National Science Foundation) Grant number 96000943.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Supplementary material

13042_2018_890_MOESM1_ESM.rar (361 kb)
Supplementary material 1 (RAR 360 KB)

References

  1. 1.
    Tsoukalas LH, Uhrig RE (1996) Fuzzy and neural approaches in engineering. Wiley, New YorkGoogle Scholar
  2. 2.
    Samarasinghe S (2006) Neural networks for applied sciences and engineering: from fundamentals to complex pattern recognition. CRC Press, Boca RatonCrossRefGoogle Scholar
  3. 3.
    Zadeh LA (1965) Fuzzy sets. Inf Control 8(3):338–353CrossRefGoogle Scholar
  4. 4.
    Zadeh LA (1973) Outline of a new approach to the analysis of complex systems and decision processes. IEEE Trans Syst Man Cybern SMC-3(1):28–44MathSciNetCrossRefGoogle Scholar
  5. 5.
    Widrow B, Rumelhart DE, Lehr MA (1994) Neural networks: applications in industry, business and science. Commun ACM 37(3):93–106CrossRefGoogle Scholar
  6. 6.
    Basheer I, Hajmeer M (2000) Artificial neural networks: fundamentals, computing, design, and application. J Microbiol Methods 43(1):3–31CrossRefGoogle Scholar
  7. 7.
    Ukil A (2007) Intelligent systems and signal processing in power engineering. Springer Science & Business Media, BerlinGoogle Scholar
  8. 8.
    Paliwal M, Kumar UA (2009) Neural networks and statistical techniques: a review of applications. Expert Syst Appl 36(1):2–17CrossRefGoogle Scholar
  9. 9.
    Haykin S (1994) Neural networks: a comprehensive foundation. Macmillan College Publishing Company, New YorkzbMATHGoogle Scholar
  10. 10.
    Park B-J, Pedrycz W, Oh S-K (2010) Polynomial-based radial basis function neural networks (P-RBF NNs) and their application to pattern classification. Appl Intell 32(1):27–46CrossRefGoogle Scholar
  11. 11.
    Sun X-Y et al (2016) Improved probabilistic neural network PNN and its application to defect recognition in rock bolts. Int J Mach Learn Cybern 7(5):909–919CrossRefGoogle Scholar
  12. 12.
    Bezdek JC (1973) Fuzzy mathematics in pattern classification. Ph. D. Dissertation, Applied Mathematics, Cornell UniversityGoogle Scholar
  13. 13.
    Mamdani EH (1974) Application of fuzzy algorithms for control of simple dynamic plant. In: Electrical engineers, Proceedings of the Institution of 1974, vol 121, no 12, pp 1585–1588CrossRefGoogle Scholar
  14. 14.
    Takagi T, Sugeno M (1985) Fuzzy identification of systems and its applications to modeling and control. IEEE Trans Syst Man Cybern SMC-15(1):116–132CrossRefGoogle Scholar
  15. 15.
    Tanaka K, Wang HO (2004) Fuzzy control systems design and analysis: a linear matrix inequality approach. Wiley, New YorkGoogle Scholar
  16. 16.
    Terano T, Asai K, Sugeno M (2014) Applied fuzzy systems. Academic Press, New YorkzbMATHGoogle Scholar
  17. 17.
    Bosque G, Campo I, Echanobe J (2014) Fuzzy systems, neural networks and neuro-fuzzy systems: a vision on their hardware implementation and platforms over two decades. Eng Appl Artif Intell 32:283–331CrossRefGoogle Scholar
  18. 18.
    Singh P (2017) A brief review of modeling approaches based on fuzzy time series. Int J Mach Learn Cybern 8(2):397–420CrossRefGoogle Scholar
  19. 19.
    Misra J, Saha I (2010) Artificial neural networks in hardware: A survey of two decades of progress. Neurocomputing 74(1):239–255CrossRefGoogle Scholar
  20. 20.
    Monmasson E et al (2011) FPGAs in industrial control applications. IEEE Trans Ind Inform 7(2):224–243CrossRefGoogle Scholar
  21. 21.
    Saldaña HJB, Cárdenas CS (2010) Design and implementation of an adaptive neuro-fuzzy inference system on an FPGA used for nonlinear function generation. In: ANDESCON, 2010 IEEE. IEEEGoogle Scholar
  22. 22.
    Gomperts A, Ukil A, Zurfluh F (2011) Development and implementation of parameterized FPGA-based general purpose neural networks for online applications. IEEE Trans Ind Inform 7(1):78–89CrossRefGoogle Scholar
  23. 23.
    Eldredge JG, Hutchings BL (1994) RRANN: a hardware implementation of the backpropagation algorithm using reconfigurable FPGAs. In: Neural networks, 1994. IEEE world congress on computational intelligence, IEEE international conference on. 1994. IEEEGoogle Scholar
  24. 24.
    Yun SB et al (2002) Hardware implementation of neural network with expansible and reconfigurable architecture. In: Neural information processing, ICONIP’02. Proceedings of the 9th international conference on. 2002. IEEEGoogle Scholar
  25. 25.
    Del Campo, I et al (2008) Efficient hardware/software implementation of an adaptive neuro-fuzzy system. IEEE Trans Fuzzy Syst 16(3):761–778CrossRefGoogle Scholar
  26. 26.
    Baptista FD, Morgado-Dias F (2017) Automatic general-purpose neural hardware generator. Neural Comput Appl 28(1):25–36CrossRefGoogle Scholar
  27. 27.
    Lacey GJ (2016) Deep learning on FPGAs. DissertationGoogle Scholar
  28. 28.
    Ortega-Zamorano F et al (2017) Layer multiplexing FPGA implementation for deep back-propagation learning. Integr Comput Aided Eng 24(2):171–185CrossRefGoogle Scholar
  29. 29.
    Himavathi S, Anitha D, Muthuramalingam A (2007) Feedforward neural network implementation in FPGA using layer multiplexing for effective resource utilization. IEEE Trans Neural Netw 18(3):880–888CrossRefGoogle Scholar
  30. 30.
    Ortega-Zamorano F et al (2016) Efficient implementation of the backpropagation algorithm in fpgas and microcontrollers. IEEE Trans Neural Netw Learn Syst 27(9):1840–1850MathSciNetCrossRefGoogle Scholar
  31. 31.
    Soudry D et al (2015) Memristor-based multilayer neural networks with online gradient descent training. IEEE Trans Neural Netw Learn Syst 26(10):2408–2421MathSciNetCrossRefGoogle Scholar
  32. 32.
    Ortigosa EM et al (2006) Hardware description of multi-layer perceptrons with different abstraction levels. Microprocess Microsyst 30(7):435–444CrossRefGoogle Scholar
  33. 33.
    Shouraki SB (2000) A novel fuzzy approach to modeling and control and its hardware implementation based on brain functionality and specifications. DissertationGoogle Scholar
  34. 34.
    Sugeno M, Yasukawa T (1993) A fuzzy-logic-based approach to qualitative modeling. IEEE Trans Fuzzy Syst 1(1):7–31CrossRefGoogle Scholar
  35. 35.
    Murakami M, Honda N (2007) A study on the modeling ability of the IDS method: a soft computing technique using pattern-based information processing. Int J Approx Reason 45(3):470–487CrossRefGoogle Scholar
  36. 36.
    Bahrpeyma F, Zakerolhoseini A, Haghighi H (2015) Using IDS fitted Q to develop a real-time adaptive controller for dynamic resource provisioning in Cloud’s virtualized environment. Appl Soft Comput 26:285–298CrossRefGoogle Scholar
  37. 37.
    Sakurai Y (2005) A study of the learning control method using PBALM—a nonlinear modeling method. Ph.D., The University of Electro-Communications, TokyoGoogle Scholar
  38. 38.
    Shahdi SA, Shouraki SB (2002) Supervised active learning method as an intelligent linguistic controller and its hardware implementation. In: 2nd IASTEAD international conference on artificial intelligence and applications (AIA’02), Malaga, SpainGoogle Scholar
  39. 39.
    Shouraki SB, Honda N (1998) Fuzzy controller design by an active learning method. In: 31th symposium of intelligent control. Tokyo, JapanGoogle Scholar
  40. 40.
    Murakami M (2008) Practicality of modeling systems using the IDS method: performance investigation and hardware implementation. The University of Electro-CommunicationsGoogle Scholar
  41. 41.
    Firouzi M, Shouraki SB, Conradt J (2014) Sensorimotor control learning using a new adaptive spiking neuro-fuzzy machine, spike-IDS and STDP. In: International conference on artificial neural networks. SpringerGoogle Scholar
  42. 42.
    Cranganu C, Bahrpeyma F (2015) Use of active learning method to determine the presence and estimate the magnitude of abnormally pressured fluid zones: a case study from the Anadarko Basin, Oklahoma. In: Artificial intelligent approaches in petroleum geosciences. Springer, pp 191–208Google Scholar
  43. 43.
    Merrikh-Bayat F, Merrikh-Bayat F, Shouraki SB (2014) The neuro-fuzzy computing system with the capacity of implementation on a memristor crossbar and optimization-free hardware training. IEEE Trans Fuzzy Syst 22(5):1272–1287CrossRefGoogle Scholar
  44. 44.
    Ghorbani MJ, Choudhry MA, Feliachi A (2014) Distributed multi-agent based load shedding in power distribution systems. In: Electrical and computer engineering (CCECE), IEEE 27th Canadian conference on. 2014. IEEEGoogle Scholar
  45. 45.
    Shouraki SB, Honda N, Yuasa G (1999) Fuzzy interpretation of human intelligence. Int J Uncertain Fuzziness Knowl Based Syst 7(04):407–414CrossRefGoogle Scholar
  46. 46.
    Firouzi M, Shouraki SB, Afrakoti IEP (2014) Pattern analysis by active learning method classifier. J Intell Fuzzy Syst 26(1):49–62MathSciNetzbMATHGoogle Scholar
  47. 47.
    Javadian M, Shouraki SB, Kourabbaslou SS (2017) A novel density-based fuzzy clustering algorithm for low dimensional feature space. Fuzzy Sets Syst 318:34–55MathSciNetCrossRefGoogle Scholar
  48. 48.
    Klidbary SH et al (2017) Outlier robust fuzzy active learning method (ALM). In: 2017 7th international conference on computer and knowledge engineering (ICCKE)Google Scholar
  49. 49.
    Shahraiyni TH et al (2007) Application of the Active Learning Method for the estimation of geophysical variables in the Caspian Sea from satellite ocean colour observations. Int J Remote Sens 28(20):4677–4683CrossRefGoogle Scholar
  50. 50.
    Sagha H et al (2008) Real-Time IDS using reinforcement learning. In: Intelligent information technology application, 2008. IITA’08. Second international symposium on. IEEEGoogle Scholar
  51. 51.
    Merrikh-Bayat F, Shouraki SB, Rohani A (2011) Memristor crossbar-based hardware implementation of the IDS method. IEEE Trans Fuzzy Syst 19(6):1083–1096CrossRefGoogle Scholar
  52. 52.
    Shouraki SB, Honda N (1999) Recursive fuzzy modeling based on fuzzy interpolation. J Adv Comput Intell 3(2):114–125CrossRefGoogle Scholar
  53. 53.
    Shouraki SB, Honda N (1998) Outlines of a soft computer for brain simulation. In: International conference on soft computing information/intelligence systemsGoogle Scholar
  54. 54.
    Bagheri S, Honda N (1999) Hardware simulation of brain learning process. In: 15 fuzzy symposiumGoogle Scholar
  55. 55.
    Murakami M, Honda N (2004) Hardware for a new fuzzy-based modeling system and its redundancy. In: Fuzzy information, processing NAFIPS’04. IEEE annual meeting of the. 2004. IEEEGoogle Scholar
  56. 56.
    Tarkhan M, Shouraki SB, Khasteh SH (2009) A novel hardware implementation of IDS method. IEICE Electron Express 6(23):1626–1630CrossRefGoogle Scholar
  57. 57.
    Rabaey JM, Chandrakasan AP, Nikolic B (2002) Digital integrated circuits, vol 2. Prentice Hall, Englewood CliffsGoogle Scholar
  58. 58.
    Firouzi M et al (2010) A novel pipeline architecture of replacing ink drop spread. In: Nature and biologically inspired computing (NaBIC), 2010 second world congress on. IEEEGoogle Scholar
  59. 59.
    Mazumder P, Kang S-M, Waser R (2012) Memristors: devices, models, and applications. In: Proceedings of the IEEE, 2012, vol 100, no 6, pp 1911–1919CrossRefGoogle Scholar
  60. 60.
    Afrakoti IEP, Shouraki SB, Haghighat B (2014) An optimal hardware implementation for active learning method based on memristor crossbar structures. IEEE Syst J 8(4):1190–1199CrossRefGoogle Scholar
  61. 61.
    Klidbary SH, Shouraki SB, Afrakoti IEP (2016) Fast IDS computing system method and its memristor crossbar-based hardware implementation. arXiv:1602.06787CrossRefGoogle Scholar
  62. 62.
    Klidbary SH, Shouraki SB (2018) A novel adaptive learning algorithm for low-dimensional feature space using memristor-crossbar implementation and on-chip training. Appl Intell 48:4174–4191CrossRefGoogle Scholar
  63. 63.
    Klidbary SH, Shouraki SB, Afrakoti IEP (2018) An adaptive efficient memristive ink drop spread (IDS) computing system. Neural Comput Appl.  https://doi.org/10.1007/s00521-018-3604-0 CrossRefGoogle Scholar
  64. 64.
    Iakymchuk T et al (2014) An AER handshake-less modular infrastructure PCB with ×8 2.5 Gbps LVDS serial links. In: Circuits and systems (ISCAS), 2014 IEEE international symposium on. IEEEGoogle Scholar
  65. 65.
    Saldaña HJB, Silva-Cárdenas C (2012) A digital hardware architecture for a three-input one-output zero-order ANFIS. In: Circuits and systems (LASCAS), 2012 IEEE third Latin American symposium on. IEEEGoogle Scholar
  66. 66.
    Gómez-Castañeda F et al (2014) Photovoltaic panel emulator in FPGA technology using ANFIS approach. In: Electrical engineering, computing science and automatic control (CCE), 11th international conference on. 2014. IEEEGoogle Scholar
  67. 67.
    Bahoura M, Park C-W (2011) FPGA-implementation of high-speed MLP neural network. In: Electronics, circuits and systems (ICECS), 2011 18th IEEE international conference on. IEEEGoogle Scholar
  68. 68.
    Gironés RG et al (2005) FPGA implementation of a pipelined on-line backpropagation. J VLSI Signal Process Syst Signal Image Video Technol 40(2):189–213CrossRefGoogle Scholar
  69. 69.
    Echanobe J, Finker R, del Campo I (2015) A divide-and-conquer strategie for FPGA implementations of large MLP-based classifiers. In: Neural networks (IJCNN), 2015 international joint conference on. 2015. IEEEGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  • Sajad Haghzad Klidbary
    • 1
    Email author
  • Saeed Bagheri Shouraki
    • 1
  • Bernabe Linares-Barranco
    • 2
  1. 1.Research Group for Brain Simulation and Cognitive Science, Artificial Creatures Laboratory (ACL), Department of Electrical EngineeringSharif University of TechnologyTehranIran
  2. 2.Instituto de Microelectronica de Sevilla (IMSE-CNM), Consejo Superior de Investigaciones Cientificas (CSIC)Universidad de SevillaSevilleSpain

Personalised recommendations