Identifying Urban Mobility Challenges for the Visually Impaired with Mobile Monitoring of Multimodal Biosignals

  • Charalampos SaitisEmail author
  • Kyriaki Kalimeri
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9739)


In this study, we aim to better the user experience of the visually impaired when navigating in unfamiliar outdoor environments assisted by mobility technologies. We propose a framework for assessing their cognitive-emotional experience based on ambulatory monitoring and multimodal fusion of electroencephalography, electrodermal activity, and blood volume pulse signals. The proposed model is based on a random forest classifier which successfully infers in an automatic way the correct urban environment among eight predefined categories (AUROC 93 %). Geolocating the most predictive multimodal features that relate to cognitive load and stress, we provide further insights into the relationship of specific biomarkers with the environmental/situational factors that evoked them.


Visual impairment Multimodal Data fusion EEG EDA BVP Classification Stress Cognitive load Urban mobility 



The research leading to these results has received funding from the European Union’s Horizon 2020 Research and Innovation programme under grant agreement No 643636 “Sound of Vision.” The authors wish to thank the administration and O&M instructors at the National Institute for the Blind, Visually Impaired, and Deafblind in Iceland for their valuable input and generous assistance.


  1. 1.
    Badcock, N.A., Mousikou, P., Mahajan, Y., de Lissa, P., Thie, J., McArthur, G.: Validation of the emotiv EPOC EEG gaming system for measuring research quality auditory ERPs. PeerJ 1:e38 (2013)Google Scholar
  2. 2.
    Bao, F.S., Liu, X., Zhang, C.: PyEEG: an open source Python module for EEG/MEG feature extraction. Comput. Intell. Neurosci. 2011, e406391 (2011)CrossRefGoogle Scholar
  3. 3.
    Baumeister, J., Barthel, T., Geiss, K., Weiss, M.: Influence of phosphatidylserine on cognitive performance and cortical activity after induced stress. Nutr. Neurosci. 11(3), 103–110 (2008)CrossRefGoogle Scholar
  4. 4.
    Benedek, M., Kaernbach, C.: A continuous measure of phasic electrodermal activity. J. Neurosci. Methods 190, 80–91 (2010)CrossRefGoogle Scholar
  5. 5.
    Boucsein, W.: Electrodermal Activity, 2nd edn. Springer, New York (2012)CrossRefGoogle Scholar
  6. 6.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Cacioppo, J.T., Tassinary, L.G.: Inferring psychological significance from physiological signals. Am. Psychol. 45(1), 16–28 (1990)CrossRefGoogle Scholar
  8. 8.
    Cattaneo, Z., Vecchi, T., Cornoldi, C., Mammarella, I., Bonino, D., Ricciardi, E., Pietrini, P.: Imagery and spatial processes in blindness and visual impairement. Neurosci. Biobehav. Rev. 32, 1346–1360 (2008)CrossRefGoogle Scholar
  9. 9.
    David, H.W., Whitaker, K.W., Ries, A.J., Vettel, J.M., Cortney, B.J., Kerick, S.E., McDowell, K.: Usability of four commercially-oriented EEG systems. J. Neural Eng. 11, 046018 (2014)CrossRefGoogle Scholar
  10. 10.
    Debener, S., Minow, F., et al.: How about taking a low-cost, small, and wireless eeg for a walk? Psychophysiology 49, 1449–1453 (2012)CrossRefGoogle Scholar
  11. 11.
    Ekandem, J.I., Davis, T.A., Alvarez, I., James, M.T., Gilbert, J.E.: Evaluating the ergonomics of BCI devices for research and experimentation. Ergonomics 55, 592–598 (2012)CrossRefGoogle Scholar
  12. 12.
    Garbarino, M., Lai, M., Bender, D., Picard, R.W., Tognetti, S.: Empatica E3 - a wearable wireless multi-sensor device for real-time computerized biofeedback and data acquisition. In: EAI 4th International Conference Wireless Mobile Communication Healthcare (Mobihealth), pp. 39–42 (2014)Google Scholar
  13. 13.
    Geruschat, D.R., Smith, A.J.: Low vision for orientation and mobility. In: Wiener, W.R., Welsh, R.L., Blasch, B.B. (eds.) Foundations of Orientation and Mobility. History and Theory, vol. I, 3rd edn. AFB Press, New York (2010)Google Scholar
  14. 14.
    Giudice, N.A., Legge, G.E.: Blind navigation and the role of technology. In: Helal, A., Mokhtari, M., Abdulrazak, B. (eds.) The Engineering Handbook of Smart Technology for Aging, Disability, and Independence, pp. 479–500. John Willey & Sons, Hoboken (2008)CrossRefGoogle Scholar
  15. 15.
    Hosseini, S.A., Naghibi-Sistani, M.B.: Classification of emotional stress using brain activity. In: Gargiulo, G.D., McEwan, A. (eds.) Applied Biomedical Engineering, pp. 313–336. InTech, Rijeka (2011)Google Scholar
  16. 16.
    Jena, S.K.: Examination stress and its effect on EEG. Int. J. Med. Sci. Public Health 11(4), 1493–1497 (2015)Google Scholar
  17. 17.
    Marston, J.R., Golledge, R.G.: The hidden demand for participation in activities and travel by persons who are visually impaired. J. Vis. Impairment Blindness 97(8), 475–488 (2003)Google Scholar
  18. 18.
    Massot, B., Baltenneck, N., Gehin, C., Dittmar, A., McAdams, E.: EmoSense: an ambulatory device for the assessment of ANS activity–application in the objective evaluation of stress with the blind. IEEE Sensors J. 12(3), 543–551 (2012)CrossRefGoogle Scholar
  19. 19.
    Mavros, P., Skroumpelou, K., Smith, A.H.: Understanding the urban experience of people with visual impairments. In: Proceedings of GIS Research UK 2015, pp. 401–406. Leeds, 15–17 April 2015Google Scholar
  20. 20.
    Millar, S.: Understanding and Representing Space: Theory and Evidence from Studies with Blind and Sighted Children. Clarendon, Oxford (1994)CrossRefGoogle Scholar
  21. 21.
    Peake, P., Leonard, J.A.: The use of heart rate as an index of stress in blind pedestrians. Ergonomics 14(2), 189–204 (1971)CrossRefGoogle Scholar
  22. 22.
    Peper, E., Harvey, R., Lin, I.M., Tylova, H., Moss, D.: Is there more to blood volume pulse than heart rate variability, respiratory sinus arrhythmia, and cardiorespiratory synchrony? Biofeedback 35(2), 54–61 (2007)Google Scholar
  23. 23.
    Quiroga, R.Q., Blanco, S., Rosso, O.A., Garcia, H., Rabinowicz, A.: Searching for hidden information with Gabor Transform in generalized tonic-clonic seizures. Electroencephalogr. Clin. Neurophysiol. 103, 434–439 (1997)CrossRefGoogle Scholar
  24. 24.
    Quiñones, P.A., Greece, T.C., Yang, R., Newman, M.W.: Supporting visually impaired navigation: a needs-finding study. In: ACM CHI Conference Human Factors Computing Systems, pp. 1645–1650. Vancouver, BC, 7–12 May 2011Google Scholar
  25. 25.
    Turner, J.R.: Cardiovascular Reactivity and Stress: Patters of Physiological Response. Springer, New York (1994)CrossRefGoogle Scholar
  26. 26.
    Wand, M.P., Jones, M.C.: Kernel Smoothing. Chapman & Hall, London (1995)CrossRefzbMATHGoogle Scholar
  27. 27.
    Welsh, R.L.: Improving psychosocial functioning for orientation and mobility. In: Wiener, W.R., Welsh, R.L., Blasch, B.B. (eds.) Foundations of Orientation and Mobility. Instructional Strategies and Practical Applications, vol. 2, 3rd edn. AFB Press, New York (2010)Google Scholar
  28. 28.
    Wycherley, R.J., Nicklin, B.H.: The heart rate of blind and sighted pedestrians on a town route. Ergonomics 13(2), 181–192 (1970)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.ISI FoundationTurinItaly

Personalised recommendations