Advertisement

A Comparative Study of Pointing Techniques for Eyewear Using a Simulated Pedestrian Environment

  • Quentin RoyEmail author
  • Camelia Zakaria
  • Simon Perrault
  • Mathieu Nancel
  • Wonjung Kim
  • Archan Misra
  • Andy Cockburn
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11748)

Abstract

Eyewear displays allow users to interact with virtual content displayed over real-world vision, in active situations like standing and walking. Pointing techniques for eyewear displays have been proposed, but their social acceptability, efficiency, and situation awareness remain to be assessed. Using a novel street-walking simulator, we conducted an empirical study of target acquisition while standing and walking under different levels of street crowdedness. We evaluated three phone-based eyewear pointing techniques: indirect touch on a touchscreen, and two in-air techniques using relative device rotations around forward and a downward axes. Direct touch on a phone, without eyewear, was used as a control condition. Results showed that indirect touch was the most efficient and socially acceptable technique, and that in-air pointing was inefficient when walking. Interestingly, the eyewear displays did not improve situation awareness compared to the control condition. We discuss implications for eyewear interaction design.

Notes

Acknowledgements

This research was partially supported by Singapore Ministry of Education Academic Research Fund Tier 2 under research grant MOE2014-T2-1063, the University of Waterloo, the University of Canterbury, and INRIA.

References

  1. 1.
    Bailly, G., Müller, J., Rohs, M., Wigdor, D., Kratz, S.: ShoeSense: a new perspective on gestural interaction and wearable applications. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1239–1248. ACM (2012).  https://doi.org/10.1145/2207676.2208576
  2. 2.
    Barnard, L., Yi, J.S., Jacko, J.A., Sears, A.: An empirical comparison of use-in-motion evaluation scenarios for mobile computing devices. Int. J. Hum.-Comput. Stud. 62(4), 487–520 (2005).  https://doi.org/10.1016/j.ijhcs.2004.12.002CrossRefGoogle Scholar
  3. 3.
    Basch, C.H., Ethan, D., Zybert, P., Basch, C.E.: Pedestrian behavior at five dangerous and busy Manhattan intersections. J. Commun. Health 40(4), 789–792 (2015).  https://doi.org/10.1007/s10900-015-0001-9CrossRefGoogle Scholar
  4. 4.
    Bergstrom-Lehtovirta, J., Oulasvirta, A., Brewster, S.: The effects of walking speed on target acquisition on a touchscreen interface. In: Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services, pp. 143–146. ACM (2011).  https://doi.org/10.1145/2037373.2037396
  5. 5.
    Chan, L., et al.: FingerPad: private and subtle interaction using fingertips. In: Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, pp. 255–260. ACM (2013).  https://doi.org/10.1145/2501988.2502016
  6. 6.
    Chen, K.Y., Lyons, K., White, S., Patel, S.: uTrack: 3D input using two magnetic sensors. In: Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, pp. 237–244. ACM (2013).  https://doi.org/10.1145/2501988.2502035
  7. 7.
    Costanza, E., Inverso, S.A., Allen, R.: Toward subtle intimate interfaces for mobile devices using an EMG controller. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 481–489. ACM (2005).  https://doi.org/10.1145/1054972.1055039
  8. 8.
    Dobbelstein, D., Hock, P., Rukzio, E.: Belt: an unobtrusive touch input device for head-worn displays. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 2135–2138. ACM (2015).  https://doi.org/10.1145/2702123.2702450
  9. 9.
    Ens, B., Byagowi, A., Han, T., Hincapié-Ramos, J.D., Irani, P.: Combining ring input with hand tracking for precise, natural interaction with spatial analytic interfaces. In: Proceedings of the 2016 Symposium on Spatial User Interaction, pp. 99–102. ACM (2016).  https://doi.org/10.1145/2983310.2985757
  10. 10.
    Fiala, M.: Designing highly reliable fiducial markers. IEEE Trans. Pattern Anal. Mach. Intell. 32(7), 1317–1324 (2010).  https://doi.org/10.1109/TPAMI.2009.146CrossRefGoogle Scholar
  11. 11.
    Fitts, P.M.: The information capacity of the human motor system in controlling the amplitude of movement. J. Exp. Psychol. 74, 381–391 (1954).  https://doi.org/10.1037/h0055392CrossRefGoogle Scholar
  12. 12.
    Goffman, E.: The Presentation of Self in Everyday Life 1959. Doubleday, Garden City (2002)Google Scholar
  13. 13.
    Guiard, Y.: The kinematic chain as a model for human asymmetrical bimanual cooperation. In: Colley, A.M., Beech, J.R. (eds.) Cognition and Action in Skilled Behaviour, vol. 55, pp. 205–228. North-Holland, Amsterdam (1988).  https://doi.org/10.1016/S0166-4115(08)60623-8. B.T.A.i.PCrossRefGoogle Scholar
  14. 14.
    Harrison, C., Benko, H., Wilson, A.D.: OmniTouch: wearable multitouch interaction everywhere. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, pp. 441–450. ACM (2011).  https://doi.org/10.1145/2047196.2047255
  15. 15.
    Harrison, C., Tan, D., Morris, D.: Skinput: appropriating the body as an input surface. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 453–462. ACM (2010).  https://doi.org/10.1145/1753326.1753394
  16. 16.
    Holleis, P., Schmidt, A., Paasovaara, S., Puikkonen, A., Häkkilä, J.: Evaluating capacitive touch input on clothes. In: Proceedings of the 10th International Conference on Human Computer Interaction with Mobile Devices and Services, pp. 81–90. ACM (2008).  https://doi.org/10.1145/1409240.1409250
  17. 17.
    Hsieh, Y.T., Jylhä, A., Orso, V., Gamberini, L., Jacucci, G.: Designing a willing-to-use-in-public hand gestural interaction technique for smart glasses. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 4203–4215. ACM (2016).  https://doi.org/10.1145/2858036.2858436
  18. 18.
    Hühn, A.E., Khan, V.J., Lucero, A., Ketelaar, P.: On the use of virtual environments for the evaluation of location-based applications. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2569–2578. ACM (2012).  https://doi.org/10.1145/2207676.2208646
  19. 19.
    IPSOS: Distracted Walking Study: Topline Summary Findings, pp. 1–6. American Academy of Orthopedic Surgeons (2015)Google Scholar
  20. 20.
    Jain, M., Balakrishnan, R.: User learning and performance with bezel menus. In: CHI 2012, pp. 2221–2230. ACM (2012).  https://doi.org/10.1145/2207676.2208376
  21. 21.
    Jalaliniya, S., Mardanbeigi, D., Pederson, T., Hansen, D.W.: Head and eye movement as pointing modalities for eyewear computers. In: 2014 11th International Conference on Wearable and Implantable Body Sensor Networks Workshops, pp. 50–53, June 2014.  https://doi.org/10.1109/BSN.Workshops.2014.14
  22. 22.
    Katsuragawa, K., Pietroszek, K., Wallace, J.R., Lank, E.: Watchpoint: freehand pointing with a smartwatch in a ubiquitous display environment. In: Proceedings of the International Working Conference on Advanced Visual Interfaces, pp. 128–135. ACM (2016).  https://doi.org/10.1145/2909132.2909263
  23. 23.
    Kim, W., Choo, K.T.W., Lee, Y., Misra, A., Balan, R.K.: Empath-D: VR-based empathetic app design for accessibility. In: Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services, MobiSys 2018, pp. 123–135. ACM (2018).  https://doi.org/10.1145/3210240.3210331
  24. 24.
    Kjeldskov, J., Skov, M.B.: Was it worth the hassle?: Ten years of mobile HCI research discussions on lab and field evaluations. In: Proceedings of the 16th International Conference on Human-Computer Interaction with Mobile Devices & #38; Services, pp. 43–52. ACM (2014).  https://doi.org/10.1145/2628363.2628398
  25. 25.
    Koelle, M., El Ali, A., Cobus, V., Heuten, W., Boll, S.C.J.: All about acceptability?: Identifying factors for the adoption of data glasses. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 295–300. ACM (2017).  https://doi.org/10.1145/3025453.3025749
  26. 26.
    Koh, F.: Singapore trials LED lights on pavements: what other places are doing to keep smartphone zombies safe. The Straits Times (4), 789–792 (2017)Google Scholar
  27. 27.
    Lauber, F., Butz, A.: In-your-face, yet unseen?: Improving head-stabilized warnings to reduce reaction time. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 3201–3204. ACM (2014).  https://doi.org/10.1145/2556288.2557063
  28. 28.
    Liu, M., Nancel, M., Vogel, D.: Gunslinger: subtle arms-down mid-air interaction. In: Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology, pp. 63–71. ACM (2015).  https://doi.org/10.1145/2807442.2807489
  29. 29.
    Lopes, P., Baudisch, P.: Muscle-propelled force feedback: bringing force feedback to mobile devices. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2577–2580. ACM (2013).  https://doi.org/10.1145/2470654.2481355
  30. 30.
    Lucero, A., Vetek, A.: NotifEye: using interactive glasses to deal with notifications while walking in public. In: Proceedings of the 11th Conference on Advances in Computer Entertainment Technology. pp. 17:1–17:10. ACM (2014).  https://doi.org/10.1145/2663806.2663824
  31. 31.
    Lumsden, J., Brewster, S.: A paradigm shift: alternative interaction techniques for use with mobile & wearable devices. In: Proceedings of CASCON, pp. 197–210. IBM Press (2003)Google Scholar
  32. 32.
    Lumsden, J., Kondratova, I., Durling, S.: Investigating microphone efficacy for facilitation of mobile speech-based data entry. In: Proceedings of British HCI, pp. 89–97. British Computer Society (2007)Google Scholar
  33. 33.
    McCallum, D.C., Irani, P.: ARC-pad: absolute+relative cursor positioning for large displays with a mobile touchscreen. In: Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology, pp. 153–156. ACM (2009).  https://doi.org/10.1145/1622176.1622205
  34. 34.
    McCullough, M.: On attention to surroundings. Interactions 19, 40–49 (2012)CrossRefGoogle Scholar
  35. 35.
    Melzer, J.E.: Head-Mounted Displays: Designing for the User. McGraw-Hill Professional, New York (1997)Google Scholar
  36. 36.
    Mistry, P., Maes, P., Chang, L.: WUW - Wear Ur World: a wearable gestural interface. In: CHI 2009 Extended Abstracts on Human Factors in Computing Systems, pp. 4111–4116. ACM (2009).  https://doi.org/10.1145/1520340.1520626
  37. 37.
    Montero, C.S., Alexander, J., Marshall, M.T., Subramanian, S.: Would you do that?: Understanding social acceptance of gestural interfaces. In: Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services, pp. 275–278. ACM (2010).  https://doi.org/10.1145/1851600.1851647
  38. 38.
    Mustonen, T., Berg, M., Kaistinen, J., Kawai, T., Häkkinen, J.: Visual task performance using a monocular see-through head-mounted display (HMD) while walking (2013).  https://doi.org/10.1037/a0034635Google Scholar
  39. 39.
    Nancel, M., Pietriga, E., Chapuis, O., Beaudouin-Lafon, M.: Mid-air pointing on ultra-walls. ACM Trans. Comput.-Hum. Interact. 22(5), 21:1–21:62 (2015).  https://doi.org/10.1145/2766448CrossRefGoogle Scholar
  40. 40.
    Nasar, J.L., Troyer, D.: Pedestrian injuries due to mobile phone use in public places. Accid. Anal. Prev. 57, 91–95 (2013).  https://doi.org/10.1016/j.aap.2013.03.021CrossRefGoogle Scholar
  41. 41.
    Ng, A., Williamson, J.H., Brewster, S.A.: Comparing evaluation methods for encumbrance and walking on interaction with touchscreen mobile devices. In: Proceedings of ICMI, pp. 23–32. ACM (2014).  https://doi.org/10.1145/2628363.2628382
  42. 42.
    Oulasvirta, A., Tamminen, S., Roto, V., Kuorelahti, J.: Interaction in 4-second Bursts: the fragmented nature of attentional resources in mobile HCI. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 919–928. ACM (2005).  https://doi.org/10.1145/1054972.1055101
  43. 43.
    Pirhonen, A., Brewster, S., Holguin, C.: Gestural and audio metaphors as a means of control for mobile devices. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 291–298. ACM (2002).  https://doi.org/10.1145/503376.503428
  44. 44.
    Rahman, M., Gustafson, S., Irani, P., Subramanian, S.: Tilt techniques: investigating the dexterity of wrist-based input. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1943–1952 (2009).  https://doi.org/10.1145/1518701.1518997
  45. 45.
    Rico, J., Brewster, S.: Usable gestures for mobile interfaces: evaluating social acceptability. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 887–896. ACM (2010).  https://doi.org/10.1145/1753326.1753458
  46. 46.
    Ronkainen, S., Häkkilä, J., Kaleva, S., Colley, A., Linjama, J.: Tap input as an embedded interaction method for mobile devices. In: Proceedings of the 1st International Conference on Tangible and Embedded Interaction, pp. 263–270. ACM (2007).  https://doi.org/10.1145/1226969.1227023
  47. 47.
    Saponas, T.S., Harrison, C., Benko, H.: PocketTouch: through-fabric capacitive touch input. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, pp. 303–308. ACM (2011).  https://doi.org/10.1145/2047196.2047235
  48. 48.
    Sawhney, N., Schmandt, C.: Nomadic radio: speech and audio interaction for contextual messaging in nomadic environments. ACM Trans. Comput.-Hum. Interact. 7(3), 353–383 (2000).  https://doi.org/10.1145/355324.355327CrossRefGoogle Scholar
  49. 49.
    Schwebel, D.C., Gaines, J., Severson, J.: Validation of virtual reality as a tool to understand and prevent child pedestrian injury. Accid. Anal. Prev. 40(4), 1394–1400 (2008).  https://doi.org/10.1016/j.aap.2008.03.005CrossRefGoogle Scholar
  50. 50.
    Schwebel, D.C., Stavrinos, D., Byington, K.W., Davis, T., O’Neal, E.E., de Jong, D.: Distraction and pedestrian safety: how talking on the phone, texting, and listening to music impact crossing the street. Accid. Anal. Prev. 45, 266–271 (2012).  https://doi.org/10.1016/j.aap.2011.07.011CrossRefGoogle Scholar
  51. 51.
    Serrano, M., Ens, B.M., Irani, P.P.: Exploring the use of hand-to-face input for interacting with head-worn displays. In: Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems, pp. 3181–3190. ACM (2014).  https://doi.org/10.1145/2556288.2556984
  52. 52.
    Shneiderman, B.: Direct manipulation for comprehensible, predictable and controllable user interfaces. In: Proceedings of the 2nd International Conference on Intelligent User Interfaces - IUI 1997, pp. 33–39 (1997).  https://doi.org/10.1145/238218.238281
  53. 53.
    Soukoreff, R.W., MacKenzie, S.: Towards a standard for pointing device evaluation, perspectives on 27 years of Fitts’ law research in HCI. Int. J. Hum.-Comput. Stud. 61(6), 751–789 (2004).  https://doi.org/10.1016/j.ijhcs.2004.09.001CrossRefGoogle Scholar
  54. 54.
    Sutherland, I.E.: A head-mounted three dimensional display. In: Proceedings of the Fall Joint Computer Conference, Part I, 9–11 December 1968, pp. 757–764. ACM (1968).  https://doi.org/10.1145/1476589.1476686
  55. 55.
    Vogel, D., Balakrishnan, R.: Distant freehand pointing and clicking on very large, high resolution displays. In: Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology, pp. 33–42. ACM (2005).  https://doi.org/10.1145/1095034.1095041
  56. 56.
    Yang, X.D., Grossman, T., Wigdor, D., Fitzmaurice, G.: Magic finger: always-available input through finger instrumentation. In: Proceedings of UIST 2012, pp. 147–156. ACM (2012).  https://doi.org/10.1145/2380116.2380137
  57. 57.
    Zhao, S., Dragicevic, P., Chignell, M., Balakrishnan, R., Baudisch, P.: EarPod: eyes-free menu selection using touch input and reactive audio feedback. In: Proceedings of CHI, pp. 1395–1404. ACM (2007).  https://doi.org/10.1145/1240624.1240836

Copyright information

© IFIP International Federation for Information Processing 2019

Authors and Affiliations

  • Quentin Roy
    • 1
    • 2
    Email author
  • Camelia Zakaria
    • 1
  • Simon Perrault
    • 3
  • Mathieu Nancel
    • 4
  • Wonjung Kim
    • 1
  • Archan Misra
    • 1
  • Andy Cockburn
    • 5
  1. 1.Singapore Management UniversitySingaporeSingapore
  2. 2.School of Computer ScienceUniversity of WaterlooWaterlooCanada
  3. 3.Singapore University of Technology and Design (SUTD)SingaporeSingapore
  4. 4.Inria and University of Lille, UMR 9189 - CRIStALLilleFrance
  5. 5.University of CanterburyChristchurchNew Zealand

Personalised recommendations