Touch the 3rd Dimension! Understanding Stereoscopic 3D Touchscreen Interaction

  • Ashley Colley
  • Jonna Häkkilä
  • Johannes Schöning
  • Florian Daiber
  • Frank Steinicke
  • Antonio Krüger
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8433)


The penetration of stereoscopic 3D (S3D) output devices is becoming widespread. S3D screens range in size from large cinema screens, to tabletop displays, TVs in living rooms, and even mobile devices are nowadays equipped with autostereoscopic 3D screens. As a consequence, the requirement for interacting with 3D content is also increasing, with “touch” being one of the dominant input methods. However, the design requirements and best practices for interaction with S3D touch screen user interfaces are only now evolving. To understand the challenges and limitations S3D technology brings to interaction design, and in particular the additional demands they place on the users of such interfaces, we present our research that considers stereoscopic touch screens of different sizes in static and mobile usage contexts. We report on perceptual, cognitive and behavioral aspects revealed by user studies and present interaction design requirements based on the findings.


Stereoscopy 3D user interfaces Tabletops Mobile user interfaces User studies 


  1. 1.
    Benko, H., Feiner, S.: Balloon selection: a multi-finger technique for accurate low-fatigue 3D selection. In: Proceeding IEEE 3DUI, pp. 79–86 (2007)Google Scholar
  2. 2.
    Benzeroual, K., Allison, R.S., Wilcox, L.M.: 3D Display size matters: compensating for the perceptual effects of S3D display scaling. In: Proceedings of CVPRW 2012, pp. 45–52. IEEE (2012)Google Scholar
  3. 3.
    Bergstrom-Lehtovirta, J., Oulasvirta, A., Brewster, S.: The effects of walking speed on target acquisition on a touchscreen interface. In: Proceedings of MobileHCI 2011, ACM (2011)Google Scholar
  4. 4.
    Broy, N., André, E., Schmidt, A.: Is stereoscopic 3D a better choice for information representation in the car? In: Proceedings of the 4th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, pp. 93–100. ACM (2012)Google Scholar
  5. 5.
    Broy, N., Schneegass, S., Alt, F., Schmidt, A.: Framebox and mirrorbox: tools and guidelines to support designers in prototyping interfaces for 3D displays. In: Proceedings of CHI 2014, pp. 2037–2046. ACM (2014)Google Scholar
  6. 6.
    Bruder, G., Steinicke, F., Stuerzlinger, W.: Effects of visual conflicts on 3D selection task performance in stereoscopic display environments. In: Proceedings of 3DUI’10, pp. 115–118. IEEE (2013)Google Scholar
  7. 7.
    Bruder, G., Steinicke, F., Sturzlinger, W.: To touch or not to touch?: comparing 2D touch and 3D mid-air interaction on stereoscopic tabletop surfaces. In: Proceedings of SUI 2013. ACM (2013)Google Scholar
  8. 8.
    Chan, L.W., Kao, H.S., Chen, M.Y., Lee, M.S., Hsu, J., Hung, Y.P.: Touching the void: direct-touch interaction for intangible displays. In: Proceedings CHI 2010, pp. 2625–2634. ACM (2010)Google Scholar
  9. 9.
    Colley, A., Häkkilä, J., Schöning, J., Posti, M.: Investigating mobile stereoscopic 3D touchscreen interaction. In: Proceedings of OzCHI 2013. ACM (2013)Google Scholar
  10. 10.
    Daiber, F., Speicher, M., Gehring, S., Löchtefeld, M., Krüger, A.: Interacting with 3D content on stereoscopic displays. In: Proceedings of PerDis 2014. ACM (2014)Google Scholar
  11. 11.
    Daiber, F., Li, L., Krüger, A.: Designing gestures for mobile 3D gaming. In: Proceedings of MUM 2012. ACM (2012)Google Scholar
  12. 12.
    Dodgson, N.A.: Variation and extrema of human interpupillary distance. Stereoscopic Displays and Applications. In: Proceedings of SPIE 2004 5291, pp. 36–46 (2004)Google Scholar
  13. 13.
    Fitts, P.M.: The information capacity of the human motor system in controlling the amplitude of movement. J. Exp. Psychol. 47, 381–391 (1954)CrossRefGoogle Scholar
  14. 14.
    Grossman, T., Wigdor, D.: Going deeper: a taxonomy of 3D on the tabletop. In: Second Annual IEEE International Workshop on Horizontal Interactive Human-Computer Systems, 2007, TABLETOP 2007, pp. 137–144. IEEE (2007)Google Scholar
  15. 15.
    Häkkilä, J., Kytökorpi, K., Karukka, M.: Mobile stereoscopic 3D user experience - calling for user centric design. In: Proceedings of 3DCHI Workshop at CHI 2012 (2012)Google Scholar
  16. 16.
    Häkkilä, J., Posti, M., Schneegass, S., Alt, F., Gultekin, K., Schmidt, A.: Let me catch this! experiencing interactive 3D cinema through collecting content with a mobile phone. In: Proceedings of CHI 2014. ACM (2014)Google Scholar
  17. 17.
    Häkkilä, J., Posti, M., Ventä-Olkkonen, L., Koskenranta, O., Colley, A.: Mobile photo sharing through collaborative space in stereoscopic 3D. In: Proceedings of MUM 2013. ACM (2013)Google Scholar
  18. 18.
    Häkkilä, J., Posti, M., Koskenranta, O., Ventä-Olkkonen, L.: Design and evaluation of mobile phonebook application with stereoscopic 3D user interface. In: CHI 2013 Extended Abstracts on Human Factors in Computing Systems, pp. 1389–1394. ACM (2013)Google Scholar
  19. 19.
    Häkkinen, J., Kawaid, T., Takatalo, J., Mitsuya, R., Nyman, G.: What do people look at when they watch stereoscopic movies? In: Proceedings of SPIE 2010, pp. 7524 (2010)Google Scholar
  20. 20.
    Hall, A.D., Cunningham, J.B., Roache, R.P., Cox, J.W.: Factors affecting performance using touch entry systems: Tactual recognition fields and system accuracy. J. Appl. Psychol. 4, 711–720 (1988)CrossRefGoogle Scholar
  21. 21.
    Hilliges, O., Izadi, S., Wilson, A.D., Hodges, S., Garcia-Mendoza, A., Butz, A.: Interactions in the air: adding further depth to interactive tabletops. In: Proceedings of ACM UIST 2009, pp. 139–148 (2009)Google Scholar
  22. 22.
    Holtz, C., Baudisch, P.: Understanding touch. In: Proceedings of CHI 2011, pp. 2501–2510. ACM (2011)Google Scholar
  23. 23.
    Huhtala, J., Karukka, M., Salmimaa, M., Häkkilä, J.: Evaluating depth illusion as method of adding emphasis in autostereoscopic mobile displays. In: Proceedings of MobileHCI 2011. ACM (2011)Google Scholar
  24. 24.
    Jumisko-Pyykkö, S., Dominik Strohmeier, D., Utriainen, T., Kunze, K.: Descriptive quality of experience for mobile 3D video. In: Proceedings of NordiCHI, 2010, pp. 266–275. ACM (2010)Google Scholar
  25. 25.
    Kane, S., Wobbrock, J., Smith, I.: Getting off the treadmill: evaluating walking user interfaces for mobile devices in public spaces. In: Proceedings of MHCI08, pp. 109–118 (2008)Google Scholar
  26. 26.
    Kerber, F., Lessel, P., Mauderer, M., Daiber, F., Oulasvirta, A., Krüger, A.: Is autostereoscopy useful for handheld AR? In: Proceedings of MUM 2013, ACM (2013)Google Scholar
  27. 27.
    Kommerell, G., Schmitt, C., Kromeier, M., Bach, M.: Ocular prevalence versus ocular dominance. Vision. Res. 2003(43), 1397–1403 (2003)CrossRefGoogle Scholar
  28. 28.
    Kooi, F., Toet, A.: Visual comfort of binocular and 3-D displays. Displays 25(2004), 99–108 (2004)CrossRefGoogle Scholar
  29. 29.
    Lambooij, M.T.M., Wijnand, A., IJsselsteijn, W.A., Heynderickx, I.: Visual discomfort in stereoscopic displays: a review. In: Stereoscopic Displays and Virtual Reality Systems XIV, SPIE-IS&T (2007)Google Scholar
  30. 30.
    Loomis, J.M., Knapp, J.M.: Visual perception of egocentric distance in real and virtual environments. In: Hettinger, L.J., Haas, M.W., (eds.) Virtual and Adaptive Environ-ments, volume Virtual and adaptive environments. Mahwah (2003)Google Scholar
  31. 31.
    Matero, J., Colley, A.: Identifying unintentional touches on handheld touch screen devices. In: Proceedings of DIS 2012, pp. 506–509. ACM (2012)Google Scholar
  32. 32.
    Mauderer, M., Conte, S., Nacenta, M.A., Vishwanath, D.: Depth perception with gaze-contingent depth of field. In: Proceedings of CHI 2014, pp. 217–226. ACM (2014)Google Scholar
  33. 33.
    Mikkola, M., Boev, A., Gotchev, A.: Relative importance of depth cues on portable autostereoscopic display. In: Proceedings of MoViD at ACM Multimedia 2010, pp. 63–68 (2010)Google Scholar
  34. 34.
    Mizobuchi, S., et al.: The effect of stereoscopic viewing in a word search task with a layered background. J. Soc. Inform. Display 16(11), 1105–1113 (2008)CrossRefGoogle Scholar
  35. 35.
    Oulasvirta, A., Tamminen, S., Roto, V., Kuorelahti, J.: Interaction in 4-second bursts: the fragmented nature of attentional resources in mobile HCI. In: Proceedings of CHI 2005. ACM (2005)Google Scholar
  36. 36.
    Parhi, P., Karlson, A.K., Bederson, B.B.: Target size study for one-handed thumb use on small touchscreen devices. In: Proceedings of MobileHCI 2006, ACM (2006)Google Scholar
  37. 37.
    Perry, K., Hourcade, J.P.: Evaluating one handed thumb tapping on mobile touchscreen devices. In: Proceedings of Graphics Interface, pp. 57–64 (2008)Google Scholar
  38. 38.
    Posti, M., Ventä-Olkkonen, L., Colley, A., Koskenranta, O., Häkkilä, J.: Exploring gesture based interaction with a layered stereoscopic 3D interface. In: Proceedings of PerDis 2014. ACM (2014)Google Scholar
  39. 39.
    Pölönen, M., Salmimaa, M., Häkkinen, J.: Effect of ambient illumination level on perceived autostereoscopic display quality and depth perception. Displays 32, pp. 135–141. Elsevier (2011)Google Scholar
  40. 40.
    Schöning, J., Steinicke, F., Krüger, A., Hinrichs, K., Valkov, D.: Bimanual interaction with interscopic multi-touch surfaces. In: Gross, T., Gulliksen, J., Kotzé, P., Oestreicher, L., Palanque, P., Prates, R.O., Winckler, M. (eds.) INTERACT 2009. LNCS, vol. 5727, pp. 40–53. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  41. 41.
    Sivak, B., MacKenzie, C.L.: Integration of visual information and motor output in reaching and grasping: the contributions of peripheral and central vision. Neuropsychologia 28(10), 1095–1116 (1990)CrossRefGoogle Scholar
  42. 42.
    Strothoff, S., Valkov, D., Hinrichs, K.H.: Triangle cursor: interactions with objects above the tabletop. In: Proceedings of ACM ITS 2011, pp. 111–119 (2011)Google Scholar
  43. 43.
    Sunnari, M., Arhippainen, L., Pakanen, M., Hickey, S.: Studying user experiences of autostereoscopic 3D menu on touch screen mobile device. In: Proceedings of OZCHI 2012, ACM (2012)Google Scholar
  44. 44.
    Teather, R.J., Stuerzlinger, W.: Pointing at 3D targets in a stereo head-tracked virtual environment. In: Proceedings of 3DUI 2011, pp. 87–94. IEEE Computer Society (2011)Google Scholar
  45. 45.
    Teather, R.J., Stuerzlinger, W., Pavlovych, A.: Fishtank fitts: a desktop VR testbed for evaluating 3D pointing techniques. In: CHI 2014 Extended Abstracts on Human Factors in Computing Systems, pp. 519–522. ACM (2014)Google Scholar
  46. 46.
    Valkov, D., Giesler, A., Hinrichs, K.H.: Imperceptible depth shifts for touch interaction with stereoscopic objects. In: Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems, pp. 227–236. ACM (2014)Google Scholar
  47. 47.
    Valkov, D., Steinicke, F., Bruder, G., Hinrichs, K.: 2D Touching of 3D stereoscopic objects. In: Proceedings of of CHI 2011, pp. 1353–1362. ACM (2011)Google Scholar
  48. 48.
    Valkov, D., Steinicke, F., Bruder, G., Hinrichs, K., Schöning, J., Daiber, F., Krüger, A.: Touching floating objects in projection-based virtual reality environments. Proc. EGVE/EuroVR/VEC 2010, 17–24 (2010)Google Scholar
  49. 49.
    Ventä-Olkkonen, L., Posti, M., Häkkilä, J.: How to use 3D in stereoscopic mobile user interfaces – study of initial user perceptions. In: Proceedings of Academic MindTrek 2013. ACM (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Ashley Colley
    • 1
  • Jonna Häkkilä
    • 1
  • Johannes Schöning
    • 2
  • Florian Daiber
    • 3
  • Frank Steinicke
    • 4
  • Antonio Krüger
    • 3
  1. 1.CIEUniversity of OuluOuluFinland
  2. 2.Hasselt University, tUL – iMindsDiepenbeekBelgium
  3. 3.DFKI GmbHSaarbruckenGermany
  4. 4.Department of InformaticsUniversity of HamburgHamburgGermany

Personalised recommendations