, Volume 33, Issue 4, pp 599–619 | Cite as

It’s like holding a human heart”: the design of Vital + Morph, a shape-changing interface for remote monitoring

  • Alberto BoemEmail author
  • Hiroo Iwata
Open Forum


Based on the concept of data physicalization, we developed Vital + Morph, an interactive surface for remote connection and awareness of clinical data. It enables users located in remote places to monitor and feel the vital signs measured from a hospitalized person through shape-change. We propose shape-changing interfaces as a way of making data physicalization a richer, intriguing and memorable experience that communicates complex information and insights about data. To demonstrate and validate our proposed concept, we developed an exploratory study about the design and its implications. For evaluating the social impact of shape-changing interfaces in the context of remote monitoring, we presented Vital + Morph in several Media Art festivals. We collected and analyzed the feedback from the visitors during the exhibitions, and discussed the possibilities of the proposed system. A preliminary evaluation shows how shape-changing displays are perceived by users, which establishes not only the potential benefits but also highlights the concerns that several users have raised. Through this study, we aim to contribute to the design of remote monitoring systems by providing a novel approach for displaying clinical data that consider the richness of the physical world. In today’s information-driven society, we should not just focus on how abstract data are collected and analyzed, but also on how it can be presented and incorporated into our daily lives.


Shape-changing interfaces Remote monitoring Information technologies Haptics Vital signs 



We would like to thank Kai Sasaki and Shori Kano for the help provided during the development of Vital + Morph. In addition, we want to thank Prof. Hideaki Kuzuoka, Masakazu Hirokawa, and Alessio Chierico for the fruitful discussions and comments provided during the development of this manuscript.


  1. Barrass S (2016) Diagnosing blood pressure with acoustic sonification singing bowls. Int J Hum Comput Stud 85(C):68–71. doi: 10.1016/j.ijhcs.2015.08.007 CrossRefGoogle Scholar
  2. Bitterman N (2006) Technologies and solutions for data display in the operating room. J Clin Monit Comput 20(3):165–173. doi: 10.1007/s10877-006-9017-0 CrossRefGoogle Scholar
  3. Brown C, Hurst A (2012) VizTouch: automatically generated tactile visualizations of coordinate spaces. In: Proceedings of of TEI’12. doi: 10.1145/2148131.2148160
  4. Cermack M (2006) Monitoring and telemedicine support in remote environments and in human space flight. Br J Anaesth 97(1):107–114. doi: 10.1093/bja/ael132 CrossRefGoogle Scholar
  5. Coelho M, Zigelbaum J (2011) Shape-changing interfaces. Pers Ubi Comput 15(2):161–173. doi: 10.1007/s00779-010-0311-y CrossRefGoogle Scholar
  6. Drews FA, Westenskow DR (2016) The right picture is worth a thousand numbers: data displays in anesthesia. Hum Factors 48(1):59–71. doi: 10.1518/001872006776412270 CrossRefGoogle Scholar
  7. Dubus G, Bresin R (2014) A systematic review of mapping strategies for the sonification of physical quantities. PLoS One 9:4. doi: 10.1371/journal.pone.0096018 CrossRefGoogle Scholar
  8. Follmer S, Leithinger D, Olwal A, Hogge A, Ishii H (2013) inFORM: dynamic physical affordances and constraints through shape and object actuation. In: Proceedings of UIST’13. doi: 10.1145/2501988.2502032
  9. Garnæs K, Grünberger O, Kjeldskov J, Skov MB (2007) Designing technologies for presence-in-absence: illustrating the cube and the picture frame. Pers Ubi Comput 11(5):403–408. doi: 10.1007/s00779-006-0072-9 CrossRefGoogle Scholar
  10. Görges M, Staggers N (2008) Evaluations of physiological monitoring displays: a systematic review. J Clin Monit Comput. doi: 10.1007/s10877-007-9106-8 CrossRefGoogle Scholar
  11. Greenberg S, Kuzuoka H (1999) Using digital but physical surrogates to mediate awareness, communication and privacy in media spaces. Pers Technol 3(4):182–198. doi: 10.1007/BF01540552 CrossRefGoogle Scholar
  12. Hardy J, Weichel C, Taher F, Vidler J, Alexander J (2015) ShapeClip: towards rapid prototyping with shape-changing displays for designers. In: Proceedings of CHI’15. doi: 10.1145/2702123.2702599
  13. Heiner JM, Hudson SE, Tanaka K (1999) The information percolator: ambient information display in a decorative object. In: Proceedings of UIST’99. doi: 10.1145/320719.322595
  14. Hemmert F, Löwe M, Wohlauf A, Joost G (2013) Animate mobiles: proxemically reactive posture actuation as a means of relational interaction with mobile phones. In: Proceedings of TEI’12. doi: 10.1145/2460625.2460669
  15. Iaconesi S, Persico O (2016) La cura. Codice EdizioniGoogle Scholar
  16. Ishii H, Ullmer B (1997) Tangible bits: towards seamless interfaces between people, bits and atoms. In: Proceedings of CHI’97. doi: 10.1145/258549.258715
  17. Ishii H, Lakatos D, Bonanni L, Labrune JB (2012) Radical atoms: beyond tangible bits, toward transformable materials. Interactions 19(1):38–51. doi: 10.1145/2065327.2065337 CrossRefGoogle Scholar
  18. Iwata H (2005) Art and technology in interface devices. In: Proceedings of VRST’05. doi: 10.1145/1101616.1101617
  19. Iwata H, Yano H, Nakaizumi F, Kawamura R (2001) Project FEELEX: adding haptic surface to graphics. In: Proceedings of SIGGRAPH’01. doi: 10.1145/383259.383314
  20. Iwata H, Yano H, Ono N (2005) Volflex. In: Emerging technologies SIGGRAPH’05. doi: 10.1145/1187297.1187329
  21. Jansen Y, Dragicevic P, Fekete JD (2013) Evaluating the efficiency of physical visualizations. In: Proceedings of CHI’13. doi: 10.1145/2470654.2481359
  22. Jansen Y et al (2015) Opportunities and challenges for data physicalization. In: Proceedings of CHI’15. doi: 10.1145/2702123.2702180
  23. Janssen JH, Bailenson JN, Ijsselsteijn WA, Westerink JHDM (2010) Intimate heartbeats: opportunities for affective communication technology. IEEE Trans Affect Comput 1(2):72–80. doi: 10.1109/T-AFFC.2010.13 CrossRefGoogle Scholar
  24. Johnson GJ, Ambrose PJ (2006) Neo-tribes: the power and potential of online communities in health care. Commun ACM 49(1):107–113. doi: 10.1145/1107458.1107463 CrossRefGoogle Scholar
  25. Jovanon E, Starcevic D, Radivojevic V (2001) Perceptualization of biomedical data. In: Akay M, Marsh A (eds) Information technologies in medicine, vol I. Medical simulation and education. Wiley, New York, pp 189–204Google Scholar
  26. Kaltenbrunner M, Jorda S, Geiger G, Alonso M (2006) The reacTable*: a collaborative musical instrument. In: Proceedings of IEEE WETICE’06. doi: 10.1109/WETICE.2006.68
  27. Korhonen I, Parkka J, Van Gils M (2003) Health monitoring in the home of the future. IEEE Eng Med Biol 22(3):66–73. doi: 10.1109/MEMB.2003.1213628 CrossRefGoogle Scholar
  28. Kusunoki DS, Aleksandra S, Zhang Z, Burd RS (2013) Understanding visual attention of teams in dynamic medical settings through vital signs monitor use. In: Proceedings of CSCW’13. doi: 10.1145/2441776.2441836
  29. Leithinger D, Follmer S, Olwal A, Ishii H (2014) Physical telepresence: shape capture and display for embodied, computer-mediated remote collaboration. In: Proceedings of UIST’14. doi: 10.1145/2642918.2647377
  30. Lindlbauer D, Grønbæk JE, Birk M, Halskov K, Alexa M, Müller J (2016) Combining shape-changing interfaces and spatial augmented reality enables extended object appearance. In: Proceedings of CHI’16. doi: 10.1145/2858036.2858457
  31. Lynn G (1999) Animated form. Princeton Architectural Press, New YorkGoogle Scholar
  32. Maciejewski R, Choi S, Ebert DS, Tan HZ (2005) Multi-modal perceptualization of volumetric data and its application to molecular docking. In: Proceedings of world haptics conference. doi: 10.1109/WHC.2005.97
  33. Mankoff J, Dey AK, Hsieh G, Kientz J, Lederer S, Ames M (2003) Heuristic evaluation of ambient displays. In: Proceedings of CHI’03. doi: 10.1145/642611.642642
  34. Manovich L (2008) Introduction to info-aesthetics. Accessed 15 June 2017CrossRefGoogle Scholar
  35. McLanders M, Santomauro C, Tran J, Sanderson P (2014) Tactile displays of pulse oximetry in integrated and separated configurations. In: Proceedings of HFES’14. doi: 10.1177/1541931214581158 CrossRefGoogle Scholar
  36. Nafus D (2016) The domestication of data: why embracing digital data means embracing bigger questions. In: Proceedings of EPIC’16. doi: 10.1111/1559-8918.2016.01097 CrossRefGoogle Scholar
  37. Nakamichi D, Nishio S, Ishiguro H (2014) Training of telecommunication through teleoperated android “Telenoid” and its effect. In: Proceedings of IEEE RO-MAN. doi: 10.1109/ROMAN.2014.6926396
  38. Nangalia V, Prytherch DR, Smith GB (2010) Health technology assessment review: remote monitoring of vital signs—current status and future challenges. Crit Care 14(5):233. doi: 10.1186/cc9208 CrossRefGoogle Scholar
  39. Pangaro G, Maynes-Aminzade D, Ishii H (2002) The actuated workbench: computer-controlled actuation in tabletop tangible interfaces. In: Proceedings of UIST’02. doi: 10.1145/571985.572011
  40. Park YW, Park J, Nam TJ (2015) The trial of Bendi in a coffeehouse: use of a shape-changing device for a tactile-visual phone conversation. In: Proceedings of CHI’15. doi: 10.1145/2702123.2702326
  41. Paul C (2015) From immateriality to neomateriality: art and the conditions of digital materiality. In: Proceedings of ISEA 2015. Accessed 15 June 2017
  42. Poupyrev I, Nashida T, Okabe M (2007) Actuation and tangible user interfaces: the Vaucanson duck, robots, and shape displays. In: Proceedings of TEI’07. doi: 10.1145/1226969.1227012
  43. Rasmussen MK, Pedersen EW, Petersen MG, Hornbæk K (2012) Shape-changing interfaces: a review of the design space and open research questions. In: Proceedings of CHI’12. doi: 10.1145/2207676.2207781
  44. Schmitz M (2010) Concepts for life-like interactive objects. In: Proceedings of TEI’11. doi: 10.1145/1935701.1935732
  45. Sekiguchi D, Inami M, Tachi S (2001) RobotPHONE: RUI for interpersonal communication. In: Proceedings of CHI EA’01. doi: 10.1145/634067.634231
  46. Slovák P, Janssen J, Fitzpatrick G (2012) Understanding heart rate sharing: towards unpacking physiological space. In: Proceedings of CHI’12. doi: 10.1145/2207676.2208526
  47. Sommerer C, Mignonneau L (2004) Mobile feelings-wireless communication of heartbeat and breath for mobile art. In: Proceedings of ICAT’04Google Scholar
  48. Sugiyama Y, Hirai S (2006) Crawling and jumping by a deformable robot. Int J Robot Res 25(5–6):603–620. doi: 10.1177/0278364906065386 CrossRefGoogle Scholar
  49. Taher F et al (2017) Investigating the use of a dynamic physical bar chart for data exploration and presentation. IEEE Trans Vis Comput Gr 23(1):451–460. doi: 10.1109/TVCG.2016.2598498 CrossRefGoogle Scholar
  50. Thompson D (1992) On growth and form. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  51. Van de Moere A (2008) Beyond the tyranny of the pixel: exploring the physicality of information visualization. In: Proceedings of IV’08. doi: 10.1109/IV.2008.84
  52. Vertegaal R, Poupyrev I (eds) (2008) Special issue: organic user interfaces. Commun ACM 51(6)Google Scholar
  53. Wakita A, Nakano A, Kobayashi N (2010) Programmable blobs: a rheologic interface for organic shape design. In: Proceedings of TEI’11. doi: 10.1145/1935701.1935760
  54. Weiser M (1991) The computer for the 21st century. Sci Am 265(3):94–104CrossRefGoogle Scholar
  55. Willett W, Jansen Y, Dragicevic P (2017) Embedded data representations. IEEE Trans Vis Comput Gr 23(1):461–470. doi: 10.1109/TVCG.2016.2598608 CrossRefGoogle Scholar
  56. Wisneski C et al (1998) Ambient displays: turning architectural space into an interface between people and digital information. In: Streitz NA, Konomi S, Burkhardt HJ (eds) Cooperative buildings: integrating information, organization, and architecture. Cobuild 1998, Lecture Notes in Computer Science, vol 1370. Springer, New York, pp 22–32Google Scholar
  57. Yao L et al (2015) bioLogic: natto cells as nanoactuators for shape changing interfaces. In: Proceedings of CHI’15. doi: 10.1145/2702123.2702611

Copyright information

© Springer-Verlag London Ltd. 2017

Authors and Affiliations

  1. 1.Empowerment Informatics, School of Integrative and Global MajorsUniversity of TsukubaTsukubaJapan
  2. 2.Faculty of Engineering, Information and SystemsUniversity of TsukubaTsukubaJapan

Personalised recommendations