A Robot-Based Cognitive Assessment Model Based on Visual Working Memory and Attention Level

  • Ali Sharifara
  • Ashwin Ramesh Babu
  • Akilesh Rajavenkatanarayanan
  • Christopher Collander
  • Fillia Makedon
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10907)


Vocational assessment is the process of identifying and assessing an individual’s level of functioning in relation to vocational preparation. In this research, we have designed a framework to evaluate and train the visual working memory and attention level of users by using a humanoid robot and a brain headband sensor. The humanoid robot generates a sequence of colors and the user performs the task by arranging the colored blocks in the same order. In addition, a task-switching paradigm is used to switch between the tasks and colors to give a new instruction to the user by the robot. The humanoid robot displays guidance error detection information, observes the performance of users during the assessment and gives instructive feedback to them. This research describes the profile of cognitive and behavioral characteristics associated with visual working memory skills, selective attention and ways of supporting the learning needs of workers affected by this problem. Finally, the research concludes the relationships between visual working memory and attentional level during different level of the assessment.


Human robot interaction Visual working memory assessment Computer vision Sequence learning Socially assistive robots 



This work is supported in part by the National Science Foundation under Grant NSF-CNS 1338118. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.


  1. 1.
    Adjouadi, M., Cabrerizo, M., Yaylali, I., Jayakar, P.: Interpreting EEG functional brain activity. IEEE Potentials 23(1), 8–13 (2004)CrossRefGoogle Scholar
  2. 2.
    Belpaeme, T., Baxter, P.E., Read, R., Wood, R., Cuayáhuitl, H., Kiefer, B., Racioppa, S., Kruijff-Korbayová, I., Athanasopoulos, G., Enescu, V., Looije, R.: Multimodal child-robot interaction: building social bonds. J. Hum. Robot Interact. 1(2), 33–53 (2012)Google Scholar
  3. 3.
    Chang, C.H., Wang, S.C., Wang, C.C.: Vision-based cooperative simultaneous localization and tracking. In: IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China, pp. 5191–5197 (2011)Google Scholar
  4. 4.
    Colzato, L.S., van den Wildenberg, W.P., Zmigrod, S., Hommel, B.: Action video gaming and cognitive control: playing first person shooter games is associated with improvement in working memory but not action inhibition. Psychol. Res. 77(2), 234–239 (2013)CrossRefGoogle Scholar
  5. 5.
    Daneman, M., Carpenter, P.A.: Individual differences in working memory and reading. J. Verbal Learn. Verbal Behav. 19(4), 450–466 (1980)CrossRefGoogle Scholar
  6. 6.
    Dobnikar, A., Lotrič, U., Šter, B. (eds.): Adaptive and Natural Computing Algorithms: 10th International Conference, ICANNGA 2011, Proceedings. Springer, Heidelberg (2011). Scholar
  7. 7.
    Dollár, P., Zitnick, C.L.: Structured forests for fast edge detection. In: Proceedings of the IEEE International Conference on Computer Vision, Sydney, Australia, pp. 1841–1848 (2013)Google Scholar
  8. 8.
    Hamadicharef, B., Zhang, H., Guan, C., Wang, C., Phua, K.S., Tee, K.P., Ang, K.K.: Learning EEG-based spectral-spatial patterns for attention level measurement. In: IEEE International Symposium Circuits and Systems, ISCAS 2009, pp. 1465–1468. IEEE, Vancouver, May 2009Google Scholar
  9. 9.
    Hu, Y., Sirlantzis, K., Howells, G., Ragot, N., Rodríguez, P.: An online background subtraction algorithm deployed on a NAO humanoid robot based monitoring system. Robot. Auton. Syst. 85, 37–47 (2016)CrossRefGoogle Scholar
  10. 10.
    Ismail, L., Shamsuddin, S., Yussof, H., Hashim, H., Bahari, S., Jaafar, A., Zahari, I.: Face detection technique of Humanoid Robot NAO for application in robotic assistive therapy. In: IEEE International Conference on Control System, Computing and Engineering (ICCSCE), pp. 517–521. IEEE, Penang (2011)Google Scholar
  11. 11.
    Lay-Ekuakille, A., Vergallo, P., Griffo, G., Conversano, F., Casciaro, S., Urooj, S., Bhateja, V., Trabacca, A.: Entropy index in quantitative EEG measurement for diagnosis accuracy. IEEE Trans. Instrum. Meas. 63(6), 1440–1450 (2014)CrossRefGoogle Scholar
  12. 12.
    Mance, I., Vogel, E.K.: Visual working memory. Wiley Interdisc. Rev. Cogn. Sci. 4(2), 179–190 (2013)CrossRefGoogle Scholar
  13. 13.
    Monsell, S.: Task switching. Trends Cogn. Sci. 7(3), 134–140 (2003)CrossRefGoogle Scholar
  14. 14.
    Nguyen, T.L., Boukezzoula, R., Coquin, D., Benoit, E., Perrin, S.: Interaction between humans, NAO robot and multiple cameras for colored objects recognition using information fusion. In: 8th International Conference on Human System Interactions (HSI), pp. 322–328. IEEE, Warsaw, June 2015Google Scholar
  15. 15.
    Roy, P., Goswami, S., Chakraborty, S., Azar, A.T., Dey, N.: Image segmentation using rough set theory: a review. Int. J. Rough Sets Data Anal. (IJRSDA) 1(2), 62–74 (2014)CrossRefGoogle Scholar
  16. 16.
    Shamsuddin, S., Yussof, H., Ismail, L.I., Mohamed, S., Hanapiah, F.A., Zahari, N.I.: Humanoid robot NAO interacting with autistic children of moderately impaired intelligence to augment communication skills. Procedia Eng. 41, 1533–1538 (2012)CrossRefGoogle Scholar
  17. 17.
    Shamsuddin, S., Yussof, H., Miskam, M.A., Hamid, A.C., Malik, N.A., Hashim, H.: Humanoid robot NAO as HRI mediator to teach emotions using game-centered approach for children with autism. In: HRI 2013 Workshop on Applications for Emotional Robots (2013)Google Scholar
  18. 18.
    Shipstead, Z., Harrison, T.L., Engle, R.W.: Working memory capacity and the scope and control of attention. Atten. Percept. Psychophys. 77(6), 1863–1880 (2015)CrossRefGoogle Scholar
  19. 19.
    Simonov, M., Delconte, G.: Humanoid assessing rehabilitative exercises. Methods Inf. Med. 54(2), 114–121 (2015)CrossRefGoogle Scholar
  20. 20.
    Sousa, D.A.: How the Brain Learns. Corwin Press, Thousand Oaks (2016)Google Scholar
  21. 21.
    Veniero, D., Vossen, A., Gross, J., Thut, G.: Lasting EEG/MEG aftereffects of rhythmic transcranial brain stimulation: level of control over oscillatory network activity. Front. Cell. Neurosci. 9, 477 (2015)CrossRefGoogle Scholar
  22. 22.
    Westerberg, H., Jacobaeus, H., Hirvikoski, T., Clevberger, P., Östensson, M.L., Bartfai, A., Klingberg, T.: Computerized working memory training after stroke–a pilot study. Brain Inj. 21(1), 21–29 (2007)CrossRefGoogle Scholar
  23. 23.
    Wiechert, G., Triff, M., Liu, Z., Yin, Z., Zhao, S., Zhong, Z., Lingras, P.: Evolutionary semi-supervised rough categorization of brain signals from a wearable headband. In: 2016 IEEE Congress on Evolutionary Computation (CEC), pp. 3131–3138. IEEE, Vancouver (2016)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Ali Sharifara
    • 1
  • Ashwin Ramesh Babu
    • 1
  • Akilesh Rajavenkatanarayanan
    • 1
  • Christopher Collander
    • 1
  • Fillia Makedon
    • 1
  1. 1.Heracleia Human Centered Computing Laboratory, Department of Computer Science and EngineeringUniversity of Texas at ArlingtonArlingtonUSA

Personalised recommendations