Skip to main content
SpringerLink
Log in

Assessing the cognitive load associated with ambient displays

  • Original Paper
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

In ubiquitous computing, there is a trade-off between adequate display of information and how the layering of data from multiple devices may result in increased imposition upon cognitive load. An emerging class of ubiquitous technologies referred to as ambient displays aim to reduce this imposition through trading off highly detailed visualisation and notification capabilities in the pursuit of systems which impose low levels of workload. What is yet to be explored in detail is how these systems may impose upon cognitive load in a quantified manner. In response, this study focuses on two research questions: RQ1—What methods may be appropriate in the measure of Ambient Display induced Cognitive Load and RQ2—How does one prototype display contribute to levels of Cognitive Load in a laboratory setting. These questions are investigated through a dual-task measure of cognitive load. A repeated measures within-groups laboratory experiment was conducted where participants were required to complete an n-back task in a display-present and display-absent condition. The n-back task was found to be an appropriate method of manipulating performance in both the primary and secondary tasks, although the presence of the evaluated ambient display was not found to induce additional workload. This is despite participants indicating in a subjective measure that they were able to perceive events visualised on the ambient display.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Weiser M (1991) The computer for the 21st century. Sci Am 265:94–105

    Article  Google Scholar 

  2. Weiser M, Brown J (1997) The coming age of calm technology. Beyond Calculation 75-85. https://doi.org/10.1007/978-1-4612-0685-9_6

  3. Hazlewood WR, Stolterman E, Connelly K (2011) Issues in evaluating ambient displays in the wild: two case studies. Conference on Human Factors in Computing Systems - Proceedings. ACM Press, New York, New York, USA, pp 877–886

    Chapter  Google Scholar 

  4. Ames MG, Dey AK (2002) Description of design dimensions and evaluation for Ambient Displays. Computer Science Division, University of California

  5. Vande Moere A (2008) Beyond the tyranny of the pixel: exploring the physicality of information visualization. In: Proceedings of the International Conference on Information Visualisation. pp 469–474

  6. Mankoff J, Dey AK, Hsieh G et al (2003) Heuristic evaluation of ambient displays. In: Press ACM (ed) Conference on Human Factors in Computing Systems - Proceedings. New York, New York, USA, pp 169–176

    Chapter  Google Scholar 

  7. Hemmert F, Hamann S, Löwe M, et al (2010) Shape-changing mobiles: tapering in two-dimensional deformational displays in mobile phones. In: CHI ’10 Extended Abstracts on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, pp 3075–3080

  8. Pielot M, De Oliveira R (2013) Peripheral vibro-tactile displays. In: MobileHCI 2013 - Proceedings of the 15th International Conference on Human-Computer Interaction with Mobile Devices and Services. Association for Computing Machinery, New York, NY, USA, pp 1–10

  9. Yu B, Bongers N, van Asseldonk A, et al (2016) LivingSurface: Biofeedback through shape-changing display. In: Proceedings of the TEI ’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction. Association for Computing Machinery, New York, NY, USA, pp 168–175

  10. Torres C, O’Leary J, Paulos E (2016) LiveObjects: leveraging theatricality for an expressive internet of things. In: Proceedings of the 2016 ACM Conference Companion Publication on Designing Interactive Systems. Association for Computing Machinery, New York, NY, USA, pp 5–8

  11. Hong J, Song S, Cho J, Bianchi A (2015) Better posture awareness through flower-shaped ambient avatar. In: Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction. Association for Computing Machinery, New York, NY, USA, pp 337–340

  12. Tabuenca B, Börner D, Kalz M (2021) Effects of an ambient learning display on noise levels and perceived learning in a secondary school. IEEE Trans Learn Technol 14:69–80. https://doi.org/10.1109/TLT.2021.3056038

    Article  Google Scholar 

  13. Löcken A, Frison A-K, Fahn V, et al (2020) Increasing user experience and trust in automated vehicles via an ambient light display. In: 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services. Association for Computing Machinery, New York, NY, USA, pp 1–10

  14. An P, Bakker S, Ordanovski S, et al (2019) Unobtrusively enhancing reflection-in-action of teachers through spatially distributed ambient information. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, pp 1–14

  15. Occhialini V, van Essen H, Eggen B (2011) Design and evaluation of an ambient display to support time management during meetings. In: Campos P, Graham N, Jorge J et al (eds) Human-Computer Interaction - INTERACT 2011. Springer, Berlin Heidelberg, Berlin, Heidelberg, pp 263–280

    Chapter  Google Scholar 

  16. Murnane EL, Jiang X, Kong A, et al (2020) Designing ambient narrative-based interfaces to reflect and motivate physical activity. In: Proceedings of the SIGCHI conference on human factors in computing systems. Association for Computing Machinery, New York, NY, USA, pp 1–14

  17. Alphen E van, Bakker S (2016) Lernanto: using an ambient display during differentiated instruction. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, pp 2334–2340

  18. Steinberger F, Proppe P, Schroeter R, Alt F (2016) CoastMaster: an ambient speedometer to gamify safe driving. In: Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. Association for Computing Machinery, New York, NY, USA, pp 83–90

  19. Alemán CA, Favela J (2016) Ambient displays to assist caregivers monitoring the sleep of people with dementia. In: García CR, Caballero-Gil P, Burmester M, Quesada-Arencibia A (eds) Ubiquitous Computing and Ambient Intelligence. Springer International Publishing, Cham, pp 40–45

    Chapter  Google Scholar 

  20. Wu J, van Essen H, Eggen B (2016) Designing sculpting light systems for information decoration. In: Proceedings of the TEI ’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction. Association for Computing Machinery, New York, NY, USA, pp 609–614

  21. Hammerschmidt J, Hermann T, Walender A, Krömker N (2015) InfoPlant: multimodal augmentation of plants for enhanced human-computer interaction. In: 2015 6th IEEE International Conference on Cognitive Infocommunications (CogInfoCom). IEEE, pp 511–516

  22. Shelton B, Nesbitt K (2020) A systematic review of ambient display modalities, physical forms and interactivity. In: AMBIENT 2020: The Tenth International Conference on Ambient Computing, Applications, Services and Technologies. IARIA, pp 9–15

  23. Shelton, B, Nesbitt, K (2019) Categorised ambient displays published between 1996 and 2016 and identified in a systematic review. Mendeley Data. https://doi.org/10.17632/m8zwv4jdwr.1

  24. Nielsen J, Molich R (1990) Heuristic evaluation of user interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, pp 249–256

  25. Sweller J, Van Merrienboer JJG, Paas FGWC (1998) Cognitive architecture and instructional design. Educ Psychol Rev 10:251–296. https://doi.org/10.1023/A:1022193728205

    Article  Google Scholar 

  26. Sweller J (1988) Cognitive load during problem solving: effects on learning. Cogn Sci 12:257–285. https://doi.org/10.1016/0364-0213(88)90023-7

    Article  Google Scholar 

  27. Albers MJ (2011) Tapping as a measure of cognitive load and website usability. In: SIGDOC’11 - Proceedings of the 29th ACM International Conference on Design of Communication. Association for Computing Machinery, New York, New York, USA, pp 25–32

  28. Pousman Z, Stasko J (2006) A taxonomy of ambient information systems: four patterns of design. In: Proceedings of the Workshop on Advanced Visual Interfaces. ACM Press, New York, New York, USA, pp 67–74

  29. McKendrick RD, Cherry E (2018) A deeper look at the NASA TLX and where it falls short. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting. Sage, pp 44–48

  30. Hart S (2006) NASA-task load index (NASA-TLX); 20 years later. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting. Sage, pp 904–908

  31. Di Stasi LL, Antolí A, Gea M, Cañas JJ (2011) A neuroergonomic approach to evaluating mental workload in hypermedia interactions. Int J Ind Ergon 41:298–304. https://doi.org/10.1016/j.ergon.2011.02.00

    Article  Google Scholar 

  32. Paas F, Tuovinen JE, Tabbers H, Van Gerven PWM (2003) Cognitive load measurement as a means to advance cognitive load theory. Educ Psychol 38:63–71. https://doi.org/10.1207/S15326985EP3801_8

    Article  Google Scholar 

  33. Debue N, van de Leemput C (2014) What does germane load mean? An empirical contribution to the cognitive load theory. Front Psychol 5:1099. https://doi.org/10.3389/fpsyg.2014.01099

    Article  Google Scholar 

  34. Heard J, Harriott CE, Adams JA (2018) A survey of workload assessment algorithms. IEEE Trans Human-Machine Syst 48:434–451. https://doi.org/10.1109/THMS.2017.2782483

    Article  Google Scholar 

  35. Ahmad MI, Keller I, Robb DA, Lohan KS (2020) A framework to estimate cognitive load using physiological data. Pers Ubiquitous Comput. https://doi.org/10.1007/s00779-020-01455-7

    Article  Google Scholar 

  36. Whelan R (2007) Neuroimaging of cognitive load in instructional multimedia. Educ Res Rev 2:1. https://doi.org/10.1016/j.edurev.2006.11.001

    Article  Google Scholar 

  37. Cegarra J, Chevalier Aline A (2008) The use of Tholos software for combining measures of mental workload: toward theoretical and methodological improvements. Behav Res Methods 40:988–1000. https://doi.org/10.3758/BRM.40.4.988

    Article  Google Scholar 

  38. Hossain D, Salimullah S, Chowdhury A, Hasan S, Kabir E, Mahmudi R, Islam M (2019) Measurement of cognitive load for writing tasks using galvanic skin response. In: Proceedings of the 6th International Conference on Networking, Systems and Security (NSysS ’19), Dhaka Bangladesh, December 2019. Association for Computing Machinery, New York, NY, USA, pp 122–126

  39. Van Dillen LF, Heslenfeld DJ, Koole SL (2009) Tuning down the emotional brain: an fMRI study of the effects of cognitive load on the processing of affective images. Neuroimage 45:1212–1219. https://doi.org/10.1016/j.neuroimage.2009.01.016

    Article  Google Scholar 

  40. Ayaz H, Shewokis P, Bunce S et al (2012) Optical brain monitoring for operator training and mental workload assessment. Neuroimage 59:36–47. https://doi.org/10.1016/j.neuroimage.2011.06.023

    Article  Google Scholar 

  41. Chen S, Epps J, Ruiz N, Chen F (2011) Eye activity as a measure of human mental effort in HCI. In: International conference on Intelligent user interfaces, Palo Alto. Association for Computing Machinery, New York, NY, USA, pp 315–318

  42. Hart SG, Staveland LE (1988) Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. Adv Psychol 52:139–183. https://doi.org/10.1016/S0166-4115(08)62386-9

    Article  Google Scholar 

  43. Reid GB, Nygren TE (1988) The subjective workload assessment technique: a scaling procedure for measuring mental workload. Adv Psychol 52:185–218. https://doi.org/10.1016/S0166-4115(08)62387-0

    Article  Google Scholar 

  44. Cooper G, Harper Jr R (1969) The use of pilot rating in the evaluation of aircraft handling qualities. Washington DC Advisory Group for Aerospace Research And Development Neuilly-sur-seine

  45. Zumbach J, Mohraz M (2008) Cognitive load in hypermedia reading comprehension: Influence of text type and linearity. Comput Human Behav 24:875–887. https://doi.org/10.1016/j.chb.2007.02.015

    Article  Google Scholar 

  46. Harbluk JL, Noy YI, Trbovich PL, Eizenman M (2007) An on-road assessment of cognitive distraction: Impacts on drivers’ visual behavior and braking performance. Accid Anal Prev 39:372–379. https://doi.org/10.1016/j.aap.2006.08.013

    Article  Google Scholar 

  47. Eggemeier F, Crabtree M, Zingg J et al. (1982) Subjective workload assessment in a memory update task. In: Human Factors and Ergonomics Society Annual Meeting. SAGE Publications, pp 643–647

  48. Brünken R, Steinbacher S, Plass JL, Leutner D (2002) Assessment of cognitive load in multimedia learning using dual-task methodology. Exp Psychol 49:109–119. https://doi.org/10.1027/1618-3169.49.2.109

    Article  Google Scholar 

  49. ISO - ISO 17488:2016 - Road vehicles - Transport information and control systems - Detection-Response Task (DRT) for assessing attentional effects of cognitive load in driving. https://www.iso.org/standard/59887.html. Accessed 12 Sep 2020

  50. Bruyas M, Dumont L, Bron F (2013) Sensitivity of Detection Response Task (DRT) to the driving demand and task difficulty. In: Seventh International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design. pp 64–70

  51. Conti A, Dlugosch C, Vilimek R et al. (2012) An assessment of cognitive workload using detection response tasks. In: Proceedings of the Fourth International Conference on Applied Human Factors and Ergonomics

  52. Harbluk J, Burns P, Hernandez S et al. (2013) Detection response tasks: using remote, headmounted and Tactile signals to assess cognitive demand while driving. In: 7th International Driving Symposium on Human Factors in Driver Assessment, Training, and Vehicle Design. pp 78–84

  53. Thorpe A, Nesbitt K, Eidels A (2019) Assessing game interface workload and usability: a cognitive science perspective. In: ACSW 2019: Proceedings of the Australasian Computer Science Week Multiconference, Sydney, January 2019. ACM, pp 441–448

  54. Strayer DL, Cooper JM, Goethe RM et al (2019) Assessing the visual and cognitive demands of in-vehicle information systems. Cogn Res Princ Implic 4:1–22. https://doi.org/10.1186/s41235-019-0166-3

    Article  Google Scholar 

  55. Strayer, D. L., Drews, F. A., Albert, R. W., & Johnston, W. A. (2001). Cell phone induced perceptual impairments during simulated driving. In: Driving Assessment 2001: International Symposium on Human Factors in Driver Assessment, Training, and Vehicle Design. pp. 14–19

  56. Strayer DL, Johnston WA (2001) Driven to distraction: Dual-task studies of simulated driving and conversing on a cellular telephone. Psychol Sci 12:462–466. https://doi.org/10.1111/1467-9280.00386

    Article  Google Scholar 

  57. Shelton B, Nesbitt K (2017) Evaluating WaveWatch: an ambient display of web traffic. In: ACSW ‘17: Proceedings of the Australasian Computer Science Week Multiconference, Geelong Australia, January 2017. ACM, pp 91–99

  58. Jaeggi S, Studer-Luethi B, Buschkuehl M et al (2010) The relationship between n-back performance and matrix reasoning-implications for training and transfer. Intelligence 38:625–635. https://doi.org/10.1016/j.intell.2010.09.001

    Article  Google Scholar 

  59. Millisecond (2020) Inquisit (Version 5) [Computer Software]

  60. JASP Team (2020) JASP (Version 0.13.1) [Computer software]

  61. Jeffreys H (1961) Theory of probability. Oxford, Clarendon Press

  62. Liu Y, Pfaff M (2014) WiredIn: Using visual feedback to support task resumption. In: Proceedings on Human Factors in Computing Systems. Association for Computing Machinery, New York, New York, USA, pp 2539–2544

  63. Müller H, Kazakova A, Heuten W, Boll S (2015) Lighten up! - an ambient light progress bar using individually controllable LEDs. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer Verlag, pp 109–124

  64. Probst K, Haller M, Yasu K, et al (2014) Move-it sticky notes providing active physical feedback through motion. In: Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction. Association for Computing Machinery, New York, New York, USA, pp 29–36

  65. Wierwille WW, Casali JG (1983) A validated rating scale for global mental workload measurement applications. Proc Hum Factors Soc Annu Meet 27:129–133. https://doi.org/10.1177/154193128302700203

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Information and Physical Sciences, University of Newcastle, University Drive Callaghan, Newcastle, NSW, 2308, Australia

    Ben Shelton & Keith Nesbitt

  2. School of Psychological Sciences, University of Newcastle, University Drive Callaghan, Newcastle, NSW, 2308, Australia

    Alexander Thorpe & Ami Eidels

Authors
  1. Ben Shelton
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Keith Nesbitt
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexander Thorpe
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ami Eidels
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ben Shelton.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shelton, B., Nesbitt, K., Thorpe, A. et al. Assessing the cognitive load associated with ambient displays. Pers Ubiquit Comput 26, 185–204 (2022). https://doi.org/10.1007/s00779-021-01662-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-021-01662-w

Keywords

Search

Navigation