Prediction of Learning Abilities Based on a Cross-Modal Evaluation of Non-verbal Mental Attributes Using Video-Game-Like Interfaces

  • Yiannis Laouris
  • Elena Aristodemou
  • Pantelis Makris
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5641)


The authors propose the thesis that today’s children immersed in cyberspace need to rely on different skills and mental attributes in order to interact successfully with knowledge. It is argued that learning pedagogies as well as corresponding assessment tools must comply with the multi-modality principle. The paper describes a multimodal evaluation of the learning potential and reading/learning abilities of young children’s brains. The method is based on the assessment of non-verbal abilities using video-game-like interfaces. The results show that the ability to orientate and navigate, to sequence or categorize objects or events, as well as to discriminate visual and auditory stimuli and the short-term visual and auditory memory can predict reading and learning abilities. Moreover, the combined assessment of several independent modalities significantly increases the predictive power.


Cognitive profile multimodal neuroscience instructional design navigation categorization sequencing lateralization auditory-visual discrimination computer assessment complex system video game 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Broadbent, D.E.: Perception and Communication. Pergamon, Oxford (1958)CrossRefGoogle Scholar
  2. 2.
    Calvert, G.: The Handbook of Multisensory Processes. MIT Press, New York (2004)Google Scholar
  3. 3.
    Ghazanfar, A.: Is neocortex essentially multisensory? Trends in Cognitive Science 10, 278–285 (2006)CrossRefGoogle Scholar
  4. 4.
    Driver, J.: Multisensory interplay reveals crossmodal influences on ‘sensory-specific’ brain regions, neural responses, and judgments. Neuron 11, 11–23 (2008)CrossRefGoogle Scholar
  5. 5.
    Shimojo, S.: Sensory modalities are not separate modalities: plasticity and interactions. Curr. Opin. Neurobiol. 11, 231–239 (2001)CrossRefGoogle Scholar
  6. 6.
    Schroeder, T.A., Smiley, C.E., Fu, J., McGinnis, K.G., O’Connell, T., Hackett, M.N.: Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing. Int. J. Psychophysiol. 50, 5–17 (2003)CrossRefGoogle Scholar
  7. 7.
    Learning Disabilities Association of Canada,
  8. 8.
    Mayer, R.E.: Cognitive constraints on multimedia learning: when presenting more material results in less understanding. J. Educ. Psychol. 93, 187–198 (2001)CrossRefGoogle Scholar
  9. 9.
    Montessori, M.: The Montessori Method. Frederick Stokes (1912)Google Scholar
  10. 10.
    Fernald, G.: The effect of kinesthetic factors in development of word recognition in the case of non-readers. J. Educ. Res. 4, 355–377 (1921)CrossRefGoogle Scholar
  11. 11.
    Orton, S.T.: Specific reading disability–strephosymbolia. J. Am. Med. Assoc. 90, 1095–1099 (1928)CrossRefGoogle Scholar
  12. 12.
    Nemenyi, E., Szendrei-Radnai, J., Varga, T.: The curriculum of mathematics for the grades from 1 to 8 of Hungary’s general educating schools. The Jhos Bolyai Math. Society, Budapest (1974)Google Scholar
  13. 13.
    Paivio, A.: Mental representations: a dual coding approach. Oxford University Press, Oxford (1986)Google Scholar
  14. 14.
    Paivio, A.: Mind and Its Evolution; A Dual Coding Theoretical Approach. Psychology Press, San Diego (2006)Google Scholar
  15. 15.
    Taraszow, T., Wahl, J., Laouris, Y., Scheiter, K., Gerjets, P.: Using Dynamic Visualization with Written Explanation to By-Pass Information Processing Deficits of Children with Reading Difficulties. In: EARLI Conference, Budapest (2007)Google Scholar
  16. 16.
    Laouris, Y., Makris, P.: M.A.P.S. Mental Attributes Profiling System computerized battery for dyslexia assessment. In: Multilingual & Cross-Cultural Perspectives on Dyslexia, Omni Shoreham Hotel, Washington, D.C. (2002)Google Scholar
  17. 17.
    Papadopoulos, T.C., Laouris, Y., Makris, P.: The validation of a computerised cognitive battery of tests for the diagnosis of children with dyslexia. In: IDA 54th Annual Conference, San Diego (2003)Google Scholar
  18. 18.
    Aristodemou, E., Taraszow, T., Laouris, Y., Papadopoulos, T., Makris, P.: Prediction of Reading Performance Using the MAPS (Mental Attributes Profiling System) Multimodal Interactive ICT Application. In: 7th European Conference on e-learning, Ayia Napa (2008)Google Scholar
  19. 19.
    Piaget, J.: Judgement and reasoning of the child. Paul Kegan, London (1928)CrossRefGoogle Scholar
  20. 20.
    Wimmer, H., Mayringer, H., Landerl, K.: The double deficit hypothesis and difficulties in learning to read a regular orthography. Journal of Educational Psychology 92, 668–680 (2002)CrossRefGoogle Scholar
  21. 21.
    Woodcock, R.W.: Woodcock reading mastery tests – Revised NU: Examiner’s manual. American Guidance Service, Circle Pines (1998)Google Scholar
  22. 22.
    Papadopoulos, T.C.: Phonological and cognitive correlates of word-reading acquisition under two different instructional approaches. European Journal of Psychology of Education 26, 549–568 (2001)CrossRefGoogle Scholar
  23. 23.
    Papadopoulos, T.C., Charalambous, A., Kanari, A., Loizou, M.: Kindergarten intervention for dyslexia: The PREP remediation in Greek. European Journal of Psychology of Education 19, 79–105 (2004)CrossRefGoogle Scholar
  24. 24.
    Matzel, L.D., Han, Y.R., Grossman, H., Karnik, M.S., Patel, D., Scott, N., Specht, S.M., Gandhi, C.C.: Individual Differences in the Expression of a “General” Learning Ability in Mice. The Journal of Neuroscience 23, 6423–6433 (2003)Google Scholar
  25. 25.
    Ohnishi, T., Matsuda, H., Hirakata, M., Ugawab, Y.: Navigation ability dependent neural activation in the human brain: An fMRI study. Neuroscience Research 55, 361–369 (2006)CrossRefGoogle Scholar
  26. 26.
    Laouris, Y., Laouri, R.: Can Information and mobile technologies serve close the economic, educational, digital and social gaps and accelerate development? World Futures 64, 254–275 (2008)CrossRefGoogle Scholar
  27. 27.
    Laouris, Y., Eteokleous, N.: We need an educationally relevant definition of mobile learning. In: 4th World Conference on Mobile Learning, Cape Town (2005)Google Scholar
  28. 28.
    Nyiri, K.: The mobile phone in 2005: Where are we now? In: Proceedings, Seeing Understanding, Learning in the Mobile Age, Budapest (2005)Google Scholar
  29. 29.
    Schnotz, W.: Visual learning with new technologies: Introduction. European Journal of Psychology of Education XIV, 245–265 (1999)CrossRefGoogle Scholar
  30. 30.
    Harp, S.F., Mayer, R.E.: How seductive details do their damage: a theory of cognitive interest in science learning. J. Educ. Psychol. 93, 187–198 (1998)Google Scholar
  31. 31.
    Clark, J.M., Paivio, A.: Dual coding theory and education. Educ. Psychol. 37, 250–263 (1991)Google Scholar
  32. 32.
    Bagui, S.: Reasons for increased learning using multimedia. J. Educ. Multimed. Hypermedia 7, 3–18 (1998)Google Scholar
  33. 33.
    Zhang, Y.: A review of recent evaluation methods for image segmentation. In: Int. Symposium on Signal Proc. and its Applications (ISSPA) (2001)Google Scholar
  34. 34.
    Ishibashi, H., et al.: Multimodality functional imaging evaluation in a patient with Rasmussen’s encephalitis. Brain and Development 24, 239–244 (2001)CrossRefGoogle Scholar
  35. 35.
    Cardenes, R., et al.: Multimodal Evaluation for Medical Image Segmentation. In: Kropatsch, W.G., Kampel, M., Hanbury, A. (eds.) CAIP 2007. LNCS, vol. 4673, pp. 229–236. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  36. 36.
    Stillman, S., Essa, I.: Towards reliable multimodal sensing in aware environments. In: ACM International Conference Proceeding Series, Orlando (2001)Google Scholar
  37. 37.
    Paleari, M., Lisetti, C.L.: Toward multimodal fusion of affective cues. In: International Multimedia Conference Proceedings of the 1st ACM international workshop on Human-centered multimedia, Santa Barbara (2006)Google Scholar
  38. 38.
    Thórisson, K.R.: Modeling Multimodal Communication as a Complex System. In: Wachsmuth, I., Knoblich, G. (eds.) ZiF Research Group International Workshop. LNCS (LNAI), vol. 4930, pp. 143–168. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  39. 39.
    Wang, F., McKenzie, E.: Multifunctional learning of a multi-agent based evolutionary artificial neural network with lifetime learning. In: Proceedings of IEEE International Symposium on Computational Intelligence in Robotics and Automation. CIRA (1999) Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Yiannis Laouris
    • 1
  • Elena Aristodemou
    • 1
  • Pantelis Makris
    • 1
    • 2
  1. 1.Cyprus Neuroscience & Technology InstituteNicosiaCyprus
  2. 2.Ministry of EducationNicosiaCyprus

Personalised recommendations