Skip to main content
SpringerLink
Log in

Taxonomy of Physiologically Adaptive Systems and Design Framework

  • Conference paper
  • First Online:
Adaptive Instructional Systems. Design and Evaluation (HCII 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12792))

Included in the following conference series:

Abstract

The design of physiologically adaptive systems entails several complex steps from acquiring human body signals to create responsive adaptive behaviors that can be used to enhance conventional communication pathways between human and technological systems. Categorizing and classifying the computing techniques used to create intelligent adaptation via physiological metrics is an important step towards creating a body of knowledge that allows the field to develop and mature accordingly. This paper proposes the creation of a taxonomy that groups several physiologically adaptive (also called biocybernetic) systems that have been previously designed and reported. The taxonomy proposes two subcategories of adaptive techniques: control theoretics and machine learning, which have multiple sub-categories that we illustrate with systems created in the last decades. Based on the proposed taxonomy, we also propose a design framework that considers four fundamental aspects that should be defined when designing physiologically adaptive systems: the medium, the application area, the psychophysiological target state, and the adaptation technique. We conclude the paper by discussing the importance of the proposed taxonomy and design framework as well as suggesting research areas and applications where we envision biocybernetic systems will evolve in the following years.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.nasa.gov/topics/technology/features/mindshift.html.

  2. 2.

    https://www.weforum.org/reports/the-internet-of-bodies-is-here-tackling-new-challenges-of-technology-governance.

  3. 3.

    https://www.neuropype.io/.

  4. 4.

    https://www.neuromore.com/.

References

  1. Alpaydin, E.: Introduction to Machine Learning. Adaptive Computation and Machine Learning Series. The MIT Press, Cambridge (2009)

    Google Scholar 

  2. Amores, J., Richer, R., Zhao, N., Maes, P., Eskofier, B.M.: Promoting relaxation using virtual reality, olfactory interfaces and wearable EEG. In: 2018 IEEE 15th International Conference on Wearable and Implantable Body Sensor Networks (BSN), pp. 98–101. IEEE (2018)

    Google Scholar 

  3. Barzilay, O., Wolf, A.: Adaptive rehabilitation games. J. Electromyogr. Kinesiol. 23(1), 182–189 (2013)

    Article  Google Scholar 

  4. Boucsein, W.: Electrodermal Activity. Springer, Boston (2012). https://doi.org/10.1007/978-1-4614-1126-0

  5. Cacioppo, J.T., Tassinary, L.G., Berntson, G.: Handbook of Psychophysiology. Cambridge University Press, Cambridge (2007)

    Google Scholar 

  6. Chanel, G., Rebetez, C., Betrancourt, M., Pun, T.: Emotion assessment from physiological signals for adaptation of game difficulty. Syst. Man Cybern. 41(6), 1052–1063 (2011)

    Google Scholar 

  7. Dobrovsky, A., Borghoff, U.M., Hofmann, M.: Improving adaptive gameplay in serious games through interactive deep reinforcement learning. In: Klempous, R., Nikodem, J., Baranyi, P.Z. (eds.) Cognitive Infocommunications, Theory and Applications. TIEI, vol. 13, pp. 411–432. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-95996-2_19

    Chapter  Google Scholar 

  8. Eberlin, P., Mulholland, T.: Bilateral differences in parietal-occipital EEG induced by contingent visual feedback. Psychophysiology 13(3), 212–218 (1976)

    Article  Google Scholar 

  9. Ernst, G.: Heart-rate variability-more than heart beats? Front. Public Health 5, 240 (2017)

    Article  Google Scholar 

  10. Ewing, K.C., Fairclough, S.H., Gilleade, K.: Evaluation of an adaptive game that uses EEG measures validated during the design process as inputs to a biocybernetic loop. Front. Hum. Neurosci. 10, 223 (2016)

    Article  Google Scholar 

  11. Fairclough, S., Gilleade, K.: Construction of the biocybernetic loop: a case study. In: Proceedings of the 14th ACM International Conference on Multimodal Interaction, pp. 571–578 (2012)

    Google Scholar 

  12. Fairclough, S.H.: Physiological computing and intelligent adaptation. In: Emotions and Affect in Human Factors and Human-Computer Interaction, pp. 539–556. Elsevier, Amsterdam (2017)

    Google Scholar 

  13. Fortin-Côté, A., et al.: Predicting video game players’ fun from physiological and behavioural data. In: Arai, K., Kapoor, S., Bhatia, R. (eds.) FICC 2018. AISC, vol. 886, pp. 479–495. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-03402-3_33

    Chapter  Google Scholar 

  14. Fuchs, S.: Session overview: adaptation strategies and adaptation management. In: Schmorrow, D.D., Fidopiastis, C.M. (eds.) AC 2018. LNCS (LNAI), vol. 10915, pp. 3–8. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91470-1_1

    Chapter  Google Scholar 

  15. Fuchs, S., Schwarz, J.: Towards a dynamic selection and configuration of adaptation strategies in augmented cognition. In: Schmorrow, D.D., Fidopiastis, C.M. (eds.) AC 2017. LNCS (LNAI), vol. 10285, pp. 101–115. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-58625-0_7

    Chapter  Google Scholar 

  16. Hardy, S., Dutz, T., Wiemeyer, J., Göbel, S., Steinmetz, R.: Framework for personalized and adaptive game-based training programs in health sport. Multimedia Tools Appl. 74(14), 5289–5311 (2015)

    Article  Google Scholar 

  17. Hou, M., Banbury, S., Burns, C.: Intelligent Adaptive Systems: An Interaction-Centered Design Perspective. CRC Press, Boca Raton (2014)

    Book  Google Scholar 

  18. Jacob, R.J., Leggett, J.J., Myers, B.A., Pausch, R.: Interaction styles and input/output devices. Behav. Inf. Technol. 12(2), 69–79 (1993)

    Article  Google Scholar 

  19. Jacucci, G., Fairclough, S., Solovey, E.T.: Physiological computing. Computer 48(10), 12–16 (2015)

    Article  Google Scholar 

  20. Jagodnik, K.M., Thomas, P.S., van den Bogert, A.J., Branicky, M.S., Kirsch, R.F.: Training an actor-critic reinforcement learning controller for arm movement using human-generated rewards. IEEE Trans. Neural Syst. Rehabil. Eng. 25(10), 1892–1905 (2017)

    Article  Google Scholar 

  21. Kosunen, I., et al.: Exploring the dynamics of the biocybernetic loop in physiological computing. Series of publications A/Department of Computer Science, University of Helsinki (2018)

    Google Scholar 

  22. Kramer, A.F., Weber, T.: Applications of psychophysiology to human factors. In: Handbook of Psychophysiology, vol. 2, pp. 794–814 (2000)

    Google Scholar 

  23. Labonte-Lemoyne, E., Courtemanche, F., Louis, V., Fredette, M., Sénécal, S., Léger, P.M.: Dynamic threshold selection for a biocybernetic loop in an adaptive video game context. Front. Hum. Neurosci. 12, 282 (2018)

    Article  Google Scholar 

  24. Loewe, N., Nadj, M.: Physio-adaptive systems-a state-of-the-art review and future research directions. In: ECIS (2020)

    Google Scholar 

  25. Luong, T., Martin, N., Raison, A., Argelaguet, F., Diverrez, J.M., Lécuyer, A.: Towards real-time recognition of users mental workload using integrated physiological sensors into a VR HMD. In: 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 425–437 (2020)

    Google Scholar 

  26. Marín-Morales, J., et al.: Affective computing in virtual reality: emotion recognition from brain and heartbeat dynamics using wearable sensors. Sci. Rep. 8(1), 1–15 (2018)

    Article  Google Scholar 

  27. Montoya, M.F., Muñoz, J.E., Henao, O.A.: Enhancing virtual rehabilitation in upper limbs with biocybernetic adaptation: the effects of virtual reality on perceived muscle fatigue, game performance and user experience. IEEE Trans. Neural Syst. Rehabil. Eng. 28(3), 740–747 (2020)

    Article  Google Scholar 

  28. Mulholland, T.: Biofeedback as scientific method. In: Biofeedback: Theory and Research, pp. 9–28 (1977)

    Google Scholar 

  29. Muñoz, J., Gonçalves, A., Vieira, T., Cró, D., Chisik, Y., i Badia, S.B.: Space connection-a multiplayer collaborative biofeedback game to promote empathy in teenagers: a feasibility study. In: International Conference on Physiological Computing Systems, vol. 2, pp. 88–97. SciTePress (2016)

    Google Scholar 

  30. Muñoz, J., Gouveia, E., Cameirao, M., Bermudez, I., Badia, S.: The biocybernetic loop engine: an integrated tool for creating physiologically adaptive videogames. In: Proceedings of the 4th International Conference on Physiological Computing Systems, pp. 45–54 (2017)

    Google Scholar 

  31. Muñoz, J.E., Cameirão, M., Bermúdez i Badia, S., Gouveia, E.R.: Closing the loop in exergaming-health benefits of biocybernetic adaptation in senior adults. In: Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play, pp. 329–339 (2018)

    Google Scholar 

  32. Muñoz, J.E., Pope, A.T., Velez, L.E.: Integrating biocybernetic adaptation in virtual reality training concentration and calmness in target shooting. In: Holzinger, A., Pope, A., Plácido da Silva, H. (eds.) PhyCS 2016-2018. LNCS, vol. 10057, pp. 218–237. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-27950-9_12

    Chapter  Google Scholar 

  33. Muñoz, J.E., Quintero, L., Stephens, C.L., Pope, A.T.: A psychophysiological model of firearms training in police officers: a virtual reality experiment for biocybernetic adaptation. Front. Psychol. 11, (2020)

    Google Scholar 

  34. Nikzad-Khasmakhi, N., Balafar, M., Reza Feizi-Derakhshi, M.: The state-of-the-art in expert recommendation systems. Eng. Appl. Artif. Intell. 82, 126–147 (2019)

    Article  Google Scholar 

  35. Oskoei, M.A., Hu, H.: Adaptive myoelectric control applied to video game. Biomed. Signal Process. Control 18, 153–160 (2015)

    Article  Google Scholar 

  36. Palsson, O.S., Harris Sr., R.L., Pope, A.T.: Method and apparatus for encouraging physiological self-regulation through modulation of an operator’s control input to a video game or training simulator. US Patent 6,450,820, 17 Sep 2002

    Google Scholar 

  37. Palsson, O., Pope, A.: Stress counter response training of pilots via instrument functionality feedback. Abstract. In: Proceedings of the 1999 Association for Applied Psychophysiology and Biofeedback Meeting (1999)

    Google Scholar 

  38. Parent, M., et al.: PASS: a multimodal database of physical activity and stress for mobile passive body/brain-computer interface research. Front. Neurosci. 14, 1274 (2020)

    Article  Google Scholar 

  39. Parnandi, A., Gutierrez-Osuna, R.: A comparative study of game mechanics and control laws for an adaptive physiological game. J. Multimodal User Interfaces 9(1), 31–42 (2014). https://doi.org/10.1007/s12193-014-0159-y

    Article  Google Scholar 

  40. Pope, A.T., Bogart, E.H., Bartolome, D.S.: Biocybernetic system evaluates indices of operator engagement in automated task. Biol. Psychol. 40(1–2), 187–195 (1995)

    Article  Google Scholar 

  41. Pope, A.T., Stephens, C.L., Blanson, N.M.: Physiologically modulating videogames or simulations which use motion-sensing input devices. US Patent 8,827,717, 9 Sep 2014

    Google Scholar 

  42. Pope, A.T., Stephens, C.L., Gilleade, K.: Biocybernetic adaptation as biofeedback training method. In: Fairclough, S.H., Gilleade, K. (eds.) Advances in Physiological Computing. HIS, pp. 91–115. Springer, London (2014). https://doi.org/10.1007/978-1-4471-6392-3_5

    Chapter  Google Scholar 

  43. Pope, A.T., Stephens, C.L., Jones, C.A.: Method and system for physiologically modulating action role-playing open world video games and simulations which use gesture and body image sensing control input devices. US Patent 9,084,933, 21 Jul 2015

    Google Scholar 

  44. Prinzel III, L.J., Pope, A.T., Palsson, O.S., Turner, M.J.: Method and apparatus for performance optimization through physical perturbation of task elements. US Patent 8,628,333, 14 Jan 2014

    Google Scholar 

  45. Raol, J.R., Ayyagari, R.: Control systems: classical, modern, and AI-based approaches. CRC Press, Boca Raton (2019)

    Book  Google Scholar 

  46. Rodriguez-Guerrero, C., Knaepen, K., Fraile-Marinero, J.C., Perez-Turiel, J., Gonzalez-de Garibay, V., Lefeber, D.: Improving challenge/skill ratio in a multimodal interface by simultaneously adapting game difficulty and haptic assistance through psychophysiological and performance feedback. Front. Neurosci. 11, 242 (2017)

    Article  Google Scholar 

  47. Ros, T., J Baars, B., Lanius, R.A., Vuilleumier, P.: Tuning pathological brain oscillations with neurofeedback: a systems neuroscience framework. Front. Hum. Neurosci. 8, 1008 (2014)

    Article  Google Scholar 

  48. Roy, R.N., Drougard, N., Gateau, T., Dehais, F., Chanel, C.P.: How can physiological computing benefit human-robot interaction? Robotics 9(4), 100 (2020)

    Article  Google Scholar 

  49. Saeed, A., Ozcelebi, T., Lukkien, J., van Erp, J.B.F., Trajanovski, S.: Model adaptation and personalization for physiological stress detection. In: 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA), pp. 209–216 (2018)

    Google Scholar 

  50. Schwartz, M.S., Andrasik, F.: Biofeedback: A Practitioner’s Guide. Guilford Publications, New York (2017)

    Google Scholar 

  51. Shirzad, N., der Loos, H.F.M.V.: Adaptation of task difficulty in rehabilitation exercises based on the user’s motor performance and physiological responses. In: 2013 IEEE 13th International Conference on Rehabilitation Robotics (ICORR), vol. 2013, pp. 1–6 (2013)

    Google Scholar 

  52. Shu, L., et al.: A review of emotion recognition using physiological signals. Sensors 18(7), 2074 (2018)

    Article  Google Scholar 

  53. Sinclair, J., Hingston, P., Masek, M., Nosaka, K.: Testing an exergame for effectiveness and attractiveness. In: 2010 2nd International IEEE Consumer Electronics Society’s Games Innovations Conference, pp. 1–8. IEEE (2010)

    Google Scholar 

  54. Stephens, C., et al.: Biocybernetic adaptation strategies: machine awareness of human engagement for improved operational performance. In: Schmorrow, D.D., Fidopiastis, C.M. (eds.) AC 2018. LNCS (LNAI), vol. 10915, pp. 89–98. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91470-1_9

    Chapter  Google Scholar 

  55. Stephens, C., et al.: Crew state monitoring and line-oriented flight training for attention management. In: 19th International Symposium on Aviation Psychology, p. 196 (2017)

    Google Scholar 

  56. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. The MIT Press, Cambridge (2018)

    MATH  Google Scholar 

  57. Treacy Solovey, E., Afergan, D., Peck, E.M., Hincks, S.W., Jacob, R.J.: Designing implicit interfaces for physiological computing: guidelines and lessons learned using fNIRS. ACM Trans. Comput. Hum. Interact. (TOCHI) 21(6), 1–27 (2015)

    Article  Google Scholar 

  58. Verhulst, A., Yamaguchi, T., Richard, P.: Physiological-based dynamic difficulty adaptation in a theragame for children with cerebral palsy. In: PhyCS, pp. 164–171 (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Systems Design Engineering, University of Waterloo, Waterloo, ON, Canada

    John E. Muñoz

  2. Department of Computer and Systems Sciences, Stockholm University, Stockholm, Sweden

    Luis Quintero

  3. NASA Langley Research Center, Hampton, VA, USA

    Chad L. Stephens

  4. Learning Engagement Technologies, Poquoson, VA, USA

    Alan Pope

Authors
  1. John E. Muñoz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luis Quintero
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chad L. Stephens
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alan Pope
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to John E. Muñoz .

Editor information

Editors and Affiliations

  1. Soar Technology, Inc., Orlando, FL, USA

    Robert A. Sottilare

  2. Fraunhofer FKIE, Wachtberg, Germany

    Jessica Schwarz

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Muñoz, J.E., Quintero, L., Stephens, C.L., Pope, A. (2021). Taxonomy of Physiologically Adaptive Systems and Design Framework. In: Sottilare, R.A., Schwarz, J. (eds) Adaptive Instructional Systems. Design and Evaluation. HCII 2021. Lecture Notes in Computer Science(), vol 12792. Springer, Cham. https://doi.org/10.1007/978-3-030-77857-6_40

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-77857-6_40

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-77856-9

  • Online ISBN: 978-3-030-77857-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation