Abstract
Since the 1960s, artificial neural networks (ANNs) have been implemented and applied in various areas of knowledge. Most of these implementations had their development guided by imperative programming (IP), usually resulting in highly coupled programs. Thus, even though intrinsically parallel in theory, ANNs do not easily take an effective distribution on multiple processors when developed under IP. As an alternative, the notification-oriented paradigm (NOP) emerges as a new programming technique. NOP facilitates the development of decoupled and distributed systems, using abstraction of knowledge through logical–causal rules, as well as the generation of an optimized code. Both features are possible by means of a notification-oriented inference process, which avoids structural and temporal redundancies in the logic–causal evaluations. These advantages are relevant to systems that have parts decoupled in order to run in parallel, such as ANN. In this sense, this work presents the development of a multilayer perceptron ANN using backpropagation training algorithm based on the concepts of a NOP implementation. Such implementation allows, transparently from high-level programming, parallel code generation that runs on multicore platforms. Furthermore, the solution based on NOP, when compared against the equivalent on IP, presents a high level of decoupling and explicit use of logic–causal elements, which are, respectively, useful to distribution, understanding and improvement of the application.
Similar content being viewed by others
References
Nilsson NJ (2009) The quest for artificial intelligence: a history of ideas and achievements, 1st edn. Cambridge University Press, New York
Haykin S (2008) Neural networks and learning machines, 2nd edn. Pearson, Porto Alegre
Baptista D, Abreu S, Freitas F et al (2013) A survey of software and hardware use in artificial neural networks. Neural Comput Appl 23:591–599. https://doi.org/10.1007/s00521-013-1406-y
Park DC, El-Sharkawy MA, Marks RJ et al (1991) Electric load forecasting using an artificial neural network. IEEE Trans Power Syst 6:442–449. https://doi.org/10.1109/59.76685
Krose B, Van Der Smagt P (1996) An introduction to neural networks, 8th edn. University of Amsterdan, Amsterdam
Mellit A, Pavan AM (2010) A 24-h forecast of solar irradiance using artificial neural network: application for performance prediction of a grid-connected PV plant at Trieste, Italy. Sol Energy 84:807–821. https://doi.org/10.1016/j.solener.2010.02.006
Omondi AR, Rajapakse JC, Bajger M (2006) FPGA Neurocomputers. In: FPGA implementations of neural networks. Springer, Dordrecht
Lapuschkin S (2016) The LRP toolbox for artificial neural networks. J Mach Learn Res 1:1–5
Arvind August D, Pingali K et al (2010) Programming multicores: do applications programmers need to write explicitly parallel programs? IEEE Micro 30:19–33. https://doi.org/10.1109/MM.2010.54
Belmonte DL, Linhares RR, Stadzisz PC, Simão JM (2016) A new method for dynamic balancing of workload and scalability in multicore systems. IEEE Lat Am Trans 14:3335–3344. https://doi.org/10.1109/TLA.2016.7587639
Valerievich BA, Anatolievna PT, Alekseevna BM, et al (2017) Modern approaches to the development parallel programs for modern multicore processors. In: 2017 6th Mediterranean conference on embedded computing (MECO). pp 1–4
Gabbrielli M, Martini S (2010) Programming languages: principles and paradigms, 1st edn. Springer, London
Seiffert U (2004) Artificial neural networks on massively parallel computer hardware. Neurocomputing 57:135–150. https://doi.org/10.1016/j.neucom.2004.01.011
Gupta G, Sohi GS (2011) Dataflow execution of sequential imperative programs on multicore architectures. In: Proceedings of the 44th annual IEEE/ACM international symposium on Microarchitecture. pp 59–70
Borkar S, Chien AA (2011) The future of microprocessors. Commun ACM 54:67. https://doi.org/10.1145/1941487.1941507
Linhares RR, Simão JM, Stadzisz PC (2015) NOCA: a notification-oriented computer architecture. IEEE Lat Am Trans 13:1593–1604. https://doi.org/10.1109/TLA.2015.7112020
Simao JM, Stadzisz PC (2009) Inference based on notifications: a holonic metamodel applied to control issues. IEEE Trans Syst Man Cybern A Syst Hum 39:238–250. https://doi.org/10.1109/TSMCA.2008.2006371
Simão JM, Banaszewski RF, Tacla CA, Stadzisz PC (2012) Notification oriented paradigm (NOP) and imperative paradigm: a comparative study. J Softw Eng Appl 5:402–416. https://doi.org/10.4236/jsea.2012.56047
Simão JM (2005) A contribution to the development of a HMS simulation tool and proposition of a meta-model for holonic control. Ph.D. Thesis. CPGEI at UTFPR, Curitiba, PR, Brazil and CRAN at UHP, France
Oliveira RN, Roth V, Henzen AF et al (2018) Notification oriented paradigm applied to ambient assisted living tool. IEEE Lat Am Trans 16:7. https://doi.org/10.1109/TLA.2018.8327425
Mendonça ITM, Simão JM, Wiecheteck LVB, Stadzisz PC (2015) Método para Desenvolvimento de Sistemas Orientados a Regras utilizando o Paradigma Orientado a Notificações. In: 12th Congresso Brasileiro de Inteligencia Computacional—CBIC. Curitiba, Paraná, Brazil
Nissen S (2017) Fast artificial neural network library. http://libfann.github.io/fann/docs/files/fann-h.html. Accessed 12 Dec 2017
Schütz F, Fabro JA, Lima CRE, et al (2015) Training of an artificial neural network with backpropagation algorithm using notification oriented paradigm. In: 2015 Latin-America congress on computational intelligence, LA-CCI 2015. IEEE Comput. Soc, Curitiba, Paraná, Brazil, pp 1–6
Heaton J (2015) Artificial Intelligence for Humans, Volume 3: Deep Learning and Neural Networks, 1st edn. Heaton Research Inc, Chesterfield, MO
Fisher RA (1936) The use of multiple measurements in taxonomic problems. Ann Hum Genet 7:179–188. https://doi.org/10.1111/j.1469-1809.1936.tb02137.x
Vyas S, Upadhyay D (2014) Identification of Iris plant using feed forward neural network on the basis of floral dimensions. IJIRSET 3:18200–18204. https://doi.org/10.15680/IJIRSET.2014.0312062
Simão JM, Stadzisz PC (2008) Notification Oriented Paradigm (NOP)—a notification oriented technique to software composition and execution. Patent Number PI0805518-1, INPI, Brazil
Ronszcka AF, Banaszewski RF, Linhares RR et al (2015) Notification-oriented and rete network inference: a comparative study. In: Proceedings of 2015 IEEE international conference on systems, man, and cybernetics, SMC 2015. IEEE, Hong-Kong, China, pp 807–814
Simão JM, Tacla CA, Stadzisz PC (2009) Holonic control meta-model. IEEE Trans Syst Man Cybern A Syst Hum 2(39):1126–1139. https://doi.org/10.1109/tsmca.2009.2022060
Ronszcka AF, Valença GZ, Linhares RR et al (2017) Notification-oriented paradigm framework 2.0: an implementation based on design patterns. IEEE Lat Am Trans 15:2220–2231
Ronszcka AF, Ferreira CA, Stadzisz PC et al (2017) Notification-oriented programming language and compiler. In: Simpósio Brasileiro de Engeharia de Sistemas Computacionais. Curitiba/PR
Ferreira CA (2015) Linguagem e compilador para o Paradigma Orientado a Notificações (PON): avanços e comparações. M.Sc. Thesis. PPGCA at UTFPR, Curitiba, PR
Peters E, Jasinski RP, Pedroni VA, Simão JM (2012) A new hardware coprocessor for accelerating notification-oriented applications. In: FPT 2012–2012 international conference on field-programmable technology. IEEE Computer Society, Seoul, Korea, pp 257–260
Kerschbaumer R, Linhares RR, Simão JM et al (2017) Notification-oriented paradigm to implement digital hardware. J Circuits Syst Comput. https://doi.org/10.1142/S0218126618501244
Simão JM, Linhares RR, de Witt FA et al (2012) Paradigma Orientado a Notificações em Hardware Digital. 13p. Patent number BR 10 2012 026429 3, INPI, Brazil
Pordeus LF, Kerschbaumer R, Linhares RR et al (2016) Notification oriented paradigm to digital hardware. Sodebras 11:116–122
Ronszcka AF (2012) Contribuição para a concepção de aplicações no paradigma orientado a notificações (PON) sob o viés de padrões. M.Sc. Thesis. CPGEI at UTFPR, Curitiba, PR
Sudha N (2015) Multicore processor: architecture and programming. In: 2015 19th international symposium on VLSI design and test. IEEE, pp 1–2
O’Sullivan B, Stewart D, Goerzen J (2008) Concurrent and multicore programming. In: Real world Haskell. O’Reilly Media, New York
(2000) IEEE Standard for Information Technology—portable operating system interface (POSIX)—part 1: system application program interface (API)-Amendment J: Advanced Real-time Extensions [C Language]. IEEE Std 10031j-2000 0_1-88. https://doi.org/10.1109/ieeestd.2000.91855
Izeboudjen N, Bouridane A, Farah A, Bessalah H (2012) Application of design reuse to artificial neural networks: case study of the back propagation algorithm. Neural Comput Appl 21:1531–1544. https://doi.org/10.1007/s00521-011-0764-6
Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back-propagating errors. Nature 323:533–536
Atanassov K, Krawczak M, Sotirov S (2008) Generalized net model for parallel optimization of feed-forward neural network with variable learning rate backpropagation algorithm. In: 2008 4th international IEEE conference intelligent systems, IS 2008, pp 1616–1619
Ludwig Jr O, Montgomery E (2007) Neural networks foundations and applications with C programs, 1st edn. Ed. Ciência Moderna
Raymond ES (2003) The art of unix programming, 1st edn. Addison-Wesley, Boston
Hughes C, Hughes T (2003) Parallel and distributed programming using C++, 1st edn. Addison-Wesley, Boston
Barreto WRM, Vendrami ACBK, Simao JM (2018) Notification oriented paradigm for distributed systems. In: Anais do Computer on the Beach. Florianópolis/SC
Pordeus LF (2017) Contribuição para o desenvolvimento de uma arquitetura de computação própria ao paradigma orientado a notificações. M.Sc. Thesis. CPGEI at UTFPR, Curitiba, PR
Yun H, Gondi S, Biswas S, Corporation B (2016) BWLOCK: a dynamic memory access control framework for soft real-time applications on multicore platforms. Trans Comput 66:1247–1252. https://doi.org/10.1109/TC.2016.2640961
Akhter S, Roberts J (2006) Multi-core programming: increasing performance through software multi-threading. Intel press, USA
Pethick M, Liddle M, Werstein P, Huang Z (2003) Parallelization of a backpropagation neural network on a cluster computer. In: 15th IASTED international conference on parallel and distributed computing and systems. IASTED/ACTA Press, Marina Del Rey, California, pp 574–582
Gargouri A, Krid M, Masmoudi DS (2013) Hardware implementation of new bell-shaped pulse mode neural network with on-chip learning and application to image processing. Int J High Perform Syst Archit 4:132–143. https://doi.org/10.1504/IJHPSA.2013.055224
Lakshmi KP, Subadra M (2013) A survey on FPGA based MLP realization for on-chip learning. Int J Sci Eng Res 4:1–9
Rezvani R, Katiraee M, Jamalian AH et al (2012) A new method for hardware design of multi-layer perceptron neural networks with online training. In: Proceedings of 11th conference on cognitive informatics & cognitive computing, pp 527–534. https://doi.org/10.1109/icci-cc.2012.6311205
Abrol S, Mahajan R (2015) Artificial neural network implementation on FPGA chip. Int J Comput Sci Inf Technol Res 3:11–18
Nedjah N, da Silva FP, de Sá AO et al (2016) A massively parallel pipelined reconfigurable design for M-PLN based neural networks for efficient image classification. Neurocomputing 183:39–55. https://doi.org/10.1016/j.neucom.2015.05.138
Baptista FD, Morgado-Dias F (2017) Automatic general-purpose neural hardware generator. Neural Comput Appl 28:25–36. https://doi.org/10.1007/s00521-015-2034-5
Acknowledgements
The authors would like to thank the original creators of NOP (Prof. J. M. Simão and Prof. P. C. Stadzisz) as well as the NOP research group particularly the developers of NOP Framework 2.0 (A. F. Ronszcka and G. Z. Valença). In addition, by funding researchers, the authors would like to acknowledge the Araucaria Foundation, Capes and UTFPR.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Schütz, F., Fabro, J.A., Ronszcka, A.F. et al. Proposal of a declarative and parallelizable artificial neural network using the notification-oriented paradigm. Neural Comput & Applic 30, 1715–1731 (2018). https://doi.org/10.1007/s00521-018-3517-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-018-3517-y