Symbiotic Adaptive Interfaces: A Case Study Using BrainX3

  • Ryszard Cetnarski
  • Alberto Betella
  • Andrea Miotto
  • Riccardo Zucca
  • Xerxes D. Arsiwalla
  • Pedro Omedas
  • Jonathan Freeman
  • Paul F. M. J. Verschure
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9359)

Abstract

Modern symbiotic and adaptive HCI paradigms include real-time algorithms capable of inferring user’s states. We present an experimental interface which aims to understand the information processing capacity of a human user and use this information to improve interaction in a database exploration task. We collected the electrodermal activity and pupillometry signals in tasks of increasing difficulty and used their features to infer whether the subject was performing the task correctly or not. By combining principal component analysis and logistic regression methods, we successfully inferred the accuracy of users’ responses from the signal after the response was made. This study provides a quantitative framework for modelling user internal states and evaluates it in a practical human computer interaction task.

Keywords

Human-computer-interaction Adaptive interfaces Affective computing EDA Pupillometry 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Arsiwalla, X.D., Zucca, R., Betella, A., Martinez, E., Dalmazzo, D., Omedas, P., Deco, G., Verschure, P.F.: Network dynamics with Brain X3: a large-scale simulation of the human brain network with real-time interaction. Frontiers in Neuroinformatics 9 (2015)Google Scholar
  2. 2.
    Beatty, J., Lucero-Wagoner, B.: The pupillary system. Handbook of Psychophysiology 2, 142–162 (2000)Google Scholar
  3. 3.
    Betella, A., Cetnarski, R., Zucca, R., Arsiwalla, X.D., Martínez, E., Omedas, P., Mura, A., Verschure, P.F.: Brain X3: embodied exploration of neural data. In: Proceedings of the 2014 Virtual Reality International Conference, p. 37. ACM (2014)Google Scholar
  4. 4.
    Betella, A., Martínez, E., Zucca, R., Arsiwalla, X.D., Omedas, P., Wierenga, S., Mura, A., Wagner, J., Lingenfelser, F., André, E., et al.: Advanced interfaces to stem the data deluge in mixed reality: placing human (un) consciousness in the loop. In: ACM SIGGRAPH 2013 Posters, p. 68. ACM (2013)Google Scholar
  5. 5.
    Boucsein, W.: Electrodermal activity. Springer Science & Business Media (2012)Google Scholar
  6. 6.
    Clark, L., Crooks, B., Clarke, R., Aitken, M.R., Dunn, B.D.: Physiological responses to near-miss outcomes and personal control during simulated gambling. Journal of Gambling Studies 28(1), 123–137 (2012)CrossRefGoogle Scholar
  7. 7.
    Ekman, P., Levenson, R.W., Friesen, W.V.: Autonomic nervous system activity distinguishes among emotions. Science 221(4616), 1208–1210 (1983)CrossRefGoogle Scholar
  8. 8.
    Fichtenholtz, H.M., Dean, H.L., Dillon, D.G., Yamasaki, H., McCarthy, G., LaBar, K.S.: Emotion-attention network interactions during a visual oddball task. Cognitive Brain Research 20(1), 67–80 (2004)CrossRefGoogle Scholar
  9. 9.
    Frith, C.D., Frith, U.: The neural basis of mentalizing. Neuron 50(4), 531–534 (2006)CrossRefGoogle Scholar
  10. 10.
    Granholm, E., Asarnow, R.F., Sarkin, A.J., Dykes, K.L.: Pupillary responses index cognitive resource limitations. Psychophysiology 33(4), 457–461 (1996)CrossRefGoogle Scholar
  11. 11.
    Jacucci, G., Spagnolli, A., Freeman, J., Gamberini, L.: Symbiotic interaction: a critical definition and comparison to other human-computer paradigms. In: Jacucci, G., Gamberini, L., Freeman, J., Spagnolli, A. (eds.) Symbiotic 2014. LNCS, vol. 8820, pp. 3–20. Springer, Heidelberg (2014) Google Scholar
  12. 12.
    Lanatà, A., Valenza, G., Scilingo, E.P.: A novel EDA glove based on textile-integrated electrodes for affective computing. Medical & Biological Engineering & Computing 50(11), 1163–1172 (2012)CrossRefGoogle Scholar
  13. 13.
    Lessiter, J., Miotto, A., Freeman, J., Verschure, P., Bernardet, U.: CEEDs: unleashing the power of the subconscious. Procedia Computer Science 7, 214–215 (2011)CrossRefGoogle Scholar
  14. 14.
    McDonald, S., Flanagan, S.: Social perception deficits after traumatic brain injury: interaction between emotion recognition, mentalizing ability, and social communication. Neuropsychology 18(3), 572 (2004)CrossRefGoogle Scholar
  15. 15.
    McKinney, W., Perktold, J., Seabold, S.: Time Series Analysis in Python with statsmodels. Jarrodmillman.Com, pp. 96–102, July 2011Google Scholar
  16. 16.
    Omedas, P., Betella, A., Zucca, R., Arsiwalla, X.D., Pacheco, D., Wagner, J., Lingenfelser, F., Andre, E., Mazzei, D., Lanatà, A., et al.: Xim-engine: a software framework to support the development of interactive applications that uses conscious and unconscious reactions in immersive mixed reality. In: Proceedings of the 2014 Virtual Reality International Conference, p. 26. ACM (2014)Google Scholar
  17. 17.
    Paas, F., Tuovinen, J.E., Tabbers, H., Van Gerven, P.W.: Cognitive load measurement as a means to advance cognitive load theory. Educational Psychologist 38(1), 63–71 (2003)CrossRefGoogle Scholar
  18. 18.
    Pantic, M., Pentland, A., Nijholt, A., Huang, T.S.: Human computing and machine understanding of human behavior: a survey. In: Huang, T.S., Nijholt, A., Pantic, M., Pentland, A. (eds.) ICMI/IJCAI Workshops 2007. LNCS (LNAI), vol. 4451, pp. 47–71. Springer, Heidelberg (2007) CrossRefGoogle Scholar
  19. 19.
    Setz, C., Arnrich, B., Schumm, J., La Marca, R., Troster, G., Ehlert, U.: Discriminating stress from cognitive load using a wearable EDA device. IEEE Transactions on Information Technology in Biomedicine 14(2), 410–417 (2010)CrossRefGoogle Scholar
  20. 20.
    Wagner, J., Lingenfelser, F., André, E., Mazzei, D., Tognetti, A., Lanatà, A., De Rossi, D., Betella, A., Zucca, R., Omedas, P., et al.: A sensing architecture for empathetic data systems. In: Proceedings of the 4th Augmented Human International Conference, pp. 96–99. ACM (2013)Google Scholar
  21. 21.
    Wagner, J., Lingenfelser, F., Baur, T., Damian, I., Kistler, F., André, E.: The social signal interpretation (ssi) framework: multimodal signal processing and recognition in real-time. In: Proceedings of the 21st ACM International Conference on Multimedia, pp. 831–834. ACM (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Ryszard Cetnarski
    • 1
  • Alberto Betella
    • 1
  • Andrea Miotto
    • 2
  • Riccardo Zucca
    • 1
  • Xerxes D. Arsiwalla
    • 1
  • Pedro Omedas
    • 1
  • Jonathan Freeman
    • 2
  • Paul F. M. J. Verschure
    • 1
    • 3
  1. 1.SPECS, N-RAS, DTICUniversitat Pompeu FabraBarcelonaSpain
  2. 2.Department of PsychologyGoldsmiths - University of LondonLondonUK
  3. 3.Institució Catalana de Recerca i Estudis Avançats (ICREA)BarcelonaSpain

Personalised recommendations