Personal and Ubiquitous Computing

, Volume 17, Issue 2, pp 399–409 | Cite as

Usability of one-handed interaction methods for handheld projection-based augmented reality

  • Jinhyuk Choi
  • Gerard J. KimEmail author
Original Article


With the advent of the portable projector (also embeddable in a smart phone), projection-based augmented reality (AR) will be an attractive form of AR as the augmentation is made directly in the real space (instead of on the video screen). Several interaction methods for such “Procam”-based projection AR systems have been developed, but their comparative usability has not been studied in depth. In this paper, we compared the usability of four representative interaction methods, as applied to the menu selection task, for the handheld projection-based AR. In particular, we explored the possibility of using just one hand for enhanced convenience and mobility. As such, the four menu selection methods chosen for the study were formed by combinations of two types of cursor control (projector cursor vs. on-device touch screen), and two types of object selection (explicit click vs. crossing), all feasible with only one hand. Other considerations included the need for maintaining the stability of the handheld projector and effectively taking advantage of the smart phone as the interaction device. Experimental results have shown that the menu selection task was the most efficient, usable and preferred when the projector cursor with the crossing widget was used. Furthermore, the task performance was not statistically different among using the dominant, non-dominant hand and even both hands.


Augmented reality Projection based Interaction Menu selection 



This research was supported by the MKE/KIAT (Strategic Technology Laboratory Support Program) and the Korea Science Foundation (Research Grant NRF-2010-32A B00183 and the ERC Program in Holo-Digilog Human Media).

Supplementary material

Supplementary material 1 (AVI 2000 kb)


  1. 1.
    Ballagas R, Rohs M, Sheridan JG (2005) Sweep, point and shoot: phonecam-based interactions for large public displays. In: Conference on human factors in computing systems, pp 1200–1203Google Scholar
  2. 2.
    Beardsley P, Baar J, Raskar R, Forlines C (2005) Interaction using a handheld projector. IEEE Comput Graph Appl 25(1):39–43CrossRefGoogle Scholar
  3. 3.
    Cao X (2009) Handheld projector interaction, PhD thesis, University of TorontoGoogle Scholar
  4. 4.
    Fitzmaurice G (1993) Situated information spaces and spatially aware palmtop computers. Commun ACM 36(7):38–49CrossRefGoogle Scholar
  5. 5.
    Fitzmaurice G, Zhai S, Chignell MH (1993) Virtual reality for palmtop computers. ACM Transact Infor Syst 11(3):197–218CrossRefGoogle Scholar
  6. 6.
    Forlines C, Balakrishnan R, Beardsley P, Baar JJ, Raskar R (2005) Zoom-and-pick: facilitating visual zooming and precision pointing with interactive handheld projectors. In: proceedings of the 18th annual ACM symposium on user interface software and technology. ACM Press, Seattle, WA, USA, pp 73–82Google Scholar
  7. 7.
    Hachet M, Pouderoux J, Guitton P (2005) A camera-based interface for interaction with mobile handheld computers. In: proceedings of the symposium on interactive 3D graphics and games, pp 65–72Google Scholar
  8. 8.
    Hachet M, Kitamura Y (2005) 3D Interaction with and from handheld computers. In: proceedings of the IEEE VR 2005 workshop: new directions in 3D user interfaces, pp 11–14Google Scholar
  9. 9.
    Hansen T, Eriksson E, Lykke-Olesen A (2005) Mixed interaction space: designing for camera based interaction with mobile devices. In: proceedings of ACM CHI 2005 conference on human factors in computing systems, pp 1933–1936Google Scholar
  10. 10.
    Kato H, Billinghurst M (1999) Marker tracking and HMD calibration for a video-based augmented reality conferencing system. In: proceedings of the 2nd international workshop on augmented reality, pp 85–94Google Scholar
  11. 11.
    Kukimoto N, Furusho Y, Nonaka J, Koyamada K, Kanazawa M (2003) PDA-based visualization control and annotation interface for virtual environment. In: proceeding of 3rd IASTED international conference visualization, image and image processingGoogle Scholar
  12. 12.
    Mantyla V, Mantyjarvi J, Seppanen T, Tuulari E (2000) Hand gesture recognition of a mobile device user. In: proceedings of the IEEE international conference on multi-media and expo, pp 281–284Google Scholar
  13. 13.
    Paelke V, Reimann C, Stichling D (2004) Foot-based mobile interaction with games. In: ACM SIGCHI international conference on advances in computer entertainment technology (ACE), pp 321–324Google Scholar
  14. 14.
    Pinhanez C (2001) The everywhere displays projector: a device to create ubiquitous graphical interfaces. Conference on ubiquitous computing, pp 315–331Google Scholar
  15. 15.
    Rapp S, Michelitsch G, Osen M, Williams J, Barbisch M, Bohan R, Valsan Z, Emele M (2004) Spotlight navigation: interaction with a handheld projection device. International conference on pervasive computing, Video paperGoogle Scholar
  16. 16.
    Rohs M, Zweifel P (2005) A conceptual framework for camera phone-based interaction techniques. Lect Notes Comput Sci 3468:171–189CrossRefGoogle Scholar
  17. 17.
    Sugimoto M, Miyahara K, Inoue H, Tsunesada Y (2005) Hotaru: intuitive manipulation techniques for projected displays of mobile devices. Proceeding of INTERACT, LNCS 3585, pp 57–68Google Scholar
  18. 18.
    Wagner D, Schmalstieg D (2003) First steps towards handheld augmented reality. In: proceedings of the 7th international conference on wearable computers, p 127Google Scholar
  19. 19.
    Watsen K, Darken RP, Capps M (1999) A handheld computer as an interaction device to a virtual environment. In: proceedings of the third immersive projection technology workshopGoogle Scholar
  20. 20.
    Willis K, Poupyrev I, Shiratori T (2011) MotionBeam: a metaphor for character interaction with handheld projectors. Proceedings of ACM SIGCHIGoogle Scholar
  21. 21.
    Winograd T, Guimbretière F (1999) Visual instruments for an interactive mural. Proceedings of ACM SIGCHI, Extended Abstracts, pp 234–235Google Scholar
  22. 22.
    Yoshida T, Hirobe Y, Nii H, Kawakami N, Tachi S (2010) Twinkle: interacting with physical surfaces using handheld projector. Proceedings of IEEE virtual reality conferenceGoogle Scholar
  23. 23.
    ARToolKit (2011) Accessed 28 Dec 2011

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  1. 1.Digital Experience Laboratory, College of Information and CommunicationsKorea UniversitySeoulKorea

Personalised recommendations