Neural Computing and Applications

, Volume 26, Issue 5, pp 1103–1115 | Cite as

SARASOM: a supervised architecture based on the recurrent associative SOM

  • David Gil
  • Jose Garcia-Rodriguez
  • Miguel Cazorla
  • Magnus Johnsson
Original Article
  • 134 Downloads

Abstract

We present and evaluate a novel supervised recurrent neural network architecture, the SARASOM, based on the associative self-organizing map. The performance of the SARASOM is evaluated and compared with the Elman network as well as with a hidden Markov model (HMM) in a number of prediction tasks using sequences of letters, including some experiments with a reduced lexicon of 15 words. The results were very encouraging with the SARASOM learning better and performing with better accuracy than both the Elman network and the HMM.

Keywords

Recurrent associative self-organizing map Supervised learning Prediction Sequence learning Elman network Recurrent neural network Hidden Markov model 

Notes

Acknowledgments

We want to express our acknowledgment to the Ministry of Science and Innovation (Ministerio de Ciencia e Innovación—MICINN) through the “José Castillejo” program from the Government of Spain and to the Swedish Research Council through the Swedish Linnaeus project Cognition, Communication and Learning (CCL) as funders of the work exhibited in this paper. This work was also partially funded by the Spanish Government DPI2013-40534-R.

References

  1. 1.
    Balkenius C, Morén J, Johansson B, Johnsson M (2010) Ikaros: building cognitive models for robots. Adv Eng Inf 24(1):40–48CrossRefGoogle Scholar
  2. 2.
    Bishop CM (1995) Neural networks for pattern recognition. Oxford University Press, OxfordGoogle Scholar
  3. 3.
    Blunsom P (2004) Hidden markov models. In: Lecture notes, vol 15, pp 18–19Google Scholar
  4. 4.
    Box GEP, Jenkins GM (1976) Time series analysis: forecasting and control, revised ed. Holden-Day, San Francisco, CAGoogle Scholar
  5. 5.
    De Gooijer JG, Hyndman RJ (2006) 25 years of time series forecasting. Int J Forecast 22(3):443–473CrossRefGoogle Scholar
  6. 6.
    Diaz AE, Hernandez CA (2009) Tool for the design and implementation of control systems with neural networks. In: IEEE international conference on intelligent computing and intelligent systems, 2009. ICIS 2009, vol 1Google Scholar
  7. 7.
    Elman JL (1990) Finding structure in time. Cogn Sci 14:179–211CrossRefGoogle Scholar
  8. 8.
    Elman JL (1991) Distributed representations, simple recurrent networks, and grammatical structure. Mach Learn 7(2):195–225Google Scholar
  9. 9.
    Erkki AL, Oja E, Simula O (2004) Time series prediction competition: the CATS benchmark. In: The 2004 international joint conference on neural networks (IJCNN), pp 2325–2329Google Scholar
  10. 10.
    Firat M, Erkan Turan M, Ali Yurdusev M (2010) Comparative analysis of neural network techniques for predicting water consumption time series. J Hydrol 384(1–2):46–51CrossRefGoogle Scholar
  11. 11.
    Gil D, Johnsson M, Garcia Chamizo JM, Soriano A, Ruiz D (2009) Application of artificial neural networks in the diagnosis of urological dysfunctions. Expert Syst Appl 36(3P2):5754–5760CrossRefGoogle Scholar
  12. 12.
    Gil D, Girela JL, De Juan J, Jose Gomez-Torres M, Johnsson M (2012) Predicting seminal quality with artificial intelligence methods. Expert Syst Appl 39(16):12564–12573CrossRefGoogle Scholar
  13. 13.
    Hassan R (2007) PhD thesis: hybrid HMM and soft computing modeling with applications to time series analysis. Computer Science and Software Engineering Department, University of Melbourne, AustraliaGoogle Scholar
  14. 14.
    Hassan R, Nath B (2005) Stockmarket forecasting using hidden Markov model: a new approach. In: IEEE computer societyGoogle Scholar
  15. 15.
    Johnsson M, Balkenius C (2008) Associating SOM representations of haptic submodalities. In: Proceedings of TAROS 2008, pp 124–129Google Scholar
  16. 16.
    Johnsson M, Balkenius C (2009) Experiments with self-organizing systems for texture and hardness perception. Glob J Comput Sci Technol 9(4):53–62Google Scholar
  17. 17.
    Johnsson M, Balkenius C, Hesslow G (2009) Associative self-organizing map. In: International joint conference on computational intelligence (IJCCI) 2009, pp 363–370Google Scholar
  18. 18.
    Johnsson M, Gil D, Balkenius C, Hesslow G (2010) Supervised architectures for internal simulation of perceptions and actions. In: Proceedings of brain inspired cognitive systems (BICS)Google Scholar
  19. 19.
    Johnsson M, Gil D, Balkenius C, Hesslow G (2011) Internal simulation in a bimodal system. In: Kofod-Petersen A, Heintz F, Langseth H (eds) Proceedings of the scandinavian conference on artificial intelligence (SCAI) 2011, Trondheim, Norway. IOS Press, pp 173–182, ISBN 978-1-60750-753-6Google Scholar
  20. 20.
    Johnsson M, Martinsson M, Gil D, Hesslow G (2011) Associative self-organizing map. In: Mwasiagi JI (ed) Self organising maps—applications and novel algorithm design. INTECH, pp 603–626, ISBN: 978-953-307-546-4Google Scholar
  21. 21.
    Kohonen T (1988) Self-organization and associative memory. Springer, BerlinMATHCrossRefGoogle Scholar
  22. 22.
    Lemke C, Gabrys B (2010) Meta-learning for time series forecasting in the NN GC1 competition. In: 2010 IEEE international conference on fuzzy systems (FUZZ). IEEE, pp 1–5Google Scholar
  23. 23.
    Mahamad AK, Hiyama T (2009) Improving Elman network using genetic algorithm for bearing failure diagnosis of induction motor. In: Diagnostics for electric machines, power electronics and drives, 2009. SDEMPED 2009. IEEE international symposium on. IEEE, pp 1–6Google Scholar
  24. 24.
    Markov A (1971) Extension of the limit theorems of probability theory to a sum of variables connected in a chain. In: Howard R (ed) Dynamic probabilistic systems (vol I: Markov models) chapter Appendix B. Wiley, New York City, pp 552–577Google Scholar
  25. 25.
    Masterson D, Kushmerick N (2003) Information extraction from multi-document threads. In: ECML-2003: workshop on adaptive text extraction and mining. Citeseer, pp 34–41Google Scholar
  26. 26.
    MathWorks I (2009) MATLAB & SIMULINK instrument control toolbox. Neural Network Toolbox. The MathWorks Inc. Google Scholar
  27. 27.
    Moody J, Utans J (1995) Architecture selection strategies for neural networks: application to corporate bond rating prediction. In: Neural networks in the capital markets, pp 277–300Google Scholar
  28. 28.
    Narasinga Rao MR, Sridhar GR, Madhu K, Rao AA (2010) A clinical decision support system using multi-layer perceptron neural network to predict quality of life in diabetes. Diabetes Metab Syndr Clin Res Rev 4(1):57–59CrossRefGoogle Scholar
  29. 29.
    Rabiner L, Juang B (1986) An introduction to hidden Markov models. ASSP Mag IEEE 3(1):4–16CrossRefGoogle Scholar
  30. 30.
    Rabiner LR (1989) A tutorial on hidden markov models and selected applications in speech recognition. Proc IEEE 77(2):257–286CrossRefGoogle Scholar
  31. 31.
    Rojas I, Valenzuela O, Rojas F, Guillen A, Herrera LJ, Pomares H, Marquez L, Pasadas M (2008) Soft-computing techniques and ARMA model for time series prediction. Neurocomputing 71(4–6):519–537CrossRefGoogle Scholar
  32. 32.
    Temurtas F (2009) A comparative study on thyroid disease diagnosis using neural networks. Expert Syst Appl 36(1):944–949CrossRefGoogle Scholar
  33. 33.
    Wang L, Shi X, Liang Y, Han X (2006) An improved elman neural network with profit factors and its applications. In: Huang D-S, Li K, Irwin GW (eds) Intelligent computing, volume 4113 of lecture notes in computer science. Springer, Berlin, pp 317–322, ISBN 978-3-540-37271-4Google Scholar
  34. 34.
    Wu CL, Chau KW (2010) Data-driven models for monthly streamflow time series prediction. Eng Appl Artif Intell 23(8):1350–1367CrossRefGoogle Scholar
  35. 35.
    Wyffels F, Schrauwen B (2010) A comparative study of reservoir computing strategies for monthly time series prediction. Neurocomputing 73(10–12):1958–1964CrossRefGoogle Scholar

Copyright information

© The Natural Computing Applications Forum 2014

Authors and Affiliations

  • David Gil
    • 1
  • Jose Garcia-Rodriguez
    • 2
  • Miguel Cazorla
    • 3
  • Magnus Johnsson
    • 4
  1. 1.Lucentia Research Group, Computing Technology and Data ProcessingUniversity of AlicanteAlicanteSpain
  2. 2.Computing Technology and Data ProcessingUniversity of AlicanteAlicanteSpain
  3. 3.Instituto de Investigación en Informática (IUII)University of AlicanteAlicanteSpain
  4. 4.Lund University Cognitive ScienceLundSweden

Personalised recommendations