Comparing Free Hand Menu Techniques for Distant Displays Using Linear, Marking and Finger-Count Menus

  • Gilles Bailly
  • Robert Walter
  • Jörg Müller
  • Tongyan Ning
  • Eric Lecolinet
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6947)


Distant displays such as interactive public displays (IPD) or interactive television (ITV) require new interaction techniques as traditional input devices may be limited or missing in these contexts. Free hand interaction, as sensed with computer vision techniques, presents a promising interaction technique. This paper presents the adaptation of three menu techniques for free hand interaction: Linear menu, Marking menu and Finger-Count menu. The first study based on a Wizard-of-Oz protocol focuses on Finger-Counting postures in front of interactive television and public displays. It reveals that participants do not choose the most efficient gestures neither before nor after the experiment. Results are used to develop a Finger-Count recognizer. The second experiment shows that all techniques achieve satisfactory accuracy. It also shows that Finger-Count requires more mental demand than other techniques.


Finger-Counting Depth-Camera Public display ITV Menus 


  1. 1.
    Bailly, G., Lecolinet, E., Nigay, L.: Wave menus: improving the novice mode of hierarchical marking menus. In: Baranauskas, C., Abascal, J., Barbosa, S.D.J. (eds.) INTERACT 2007. LNCS, vol. 4662, pp. 475–488. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  2. 2.
    Bailly, G., Lecolinet, E., Guiard, Y.: Finger-count & radial-stroke shortcuts: 2 techniques for augmenting linear menus on multi-touch surfaces. In: Proc. CHI 2010, pp. 591–594. ACM, NY (2010)Google Scholar
  3. 3.
    Benko, H., Wilson, A.D.: Pinch-the-sky dome: freehand multi-point interactions with immersive omni-directional data. In: Proc. CHI EA 2010, pp. 3045–3050. ACM, NY (2010)Google Scholar
  4. 4.
    Baudel, T., Beaudouin-Lafon, M., Charade, M.: Remote control of objects using free-hand gestures. Commun. ACM 36(7), 28–35 (1993)CrossRefGoogle Scholar
  5. 5.
    Beyer, G., Alt, F., Müller, J., Schmidt, A., Haulsen, I., Klose, S., Isakovic, K., Schiewe, M.: Audience Behavior around Large Interactive Cylindrical Screens. In: Proc. CHI 2011. ACM, NY (2011)Google Scholar
  6. 6.
    Bolt, B.R.: ”Put-that-there”: Voice and gesture at the graphics interface. In: ACM SIGGRAPH Computer Graphics, vol. 14(3), pp. 262–270. ACM, NY (1980)Google Scholar
  7. 7.
    Callahan, J., Hopkins, D., Weiser, M., Shneiderman, B.: An empirical comparison of pie vs. linear menus. In: Proc. CHI 1988, pp. 95–100. ACM, NY (1988)Google Scholar
  8. 8.
    Cesar, P., Chorianopoulos, K.: The evolution of TV systems, content and users toward interactivity. Now Foundation and Trends in HCI 2(4), 373–395 (2009)Google Scholar
  9. 9.
    Cockburn, A., Gutwin, C., Greenberg, S.: A predictive model of menu performance. In: Proc.of CHI 2007, pp. 627–636. ACM, NY (2007)Google Scholar
  10. 10.
    Dachselt, R., Hübner, A.: Virtual Environments: Three-dimensional menus: A survey and taxonomy. IEEE Computers & Graphics 31(1), 53–65 (2007)CrossRefGoogle Scholar
  11. 11.
    Fitts, P.M.: The Information Capacity of The Human Mote; System in Controlling The Amplitude of Movement. Journal of Experimental Psychology 47, 381–391 (1954)CrossRefGoogle Scholar
  12. 12.
    Francone, j., Bailly, G., Lecolinet, E., Mandran, N., Nigay, L.: Wavelet menus on handheld devices: stacking metaphor for novice mode and eyes-free selection for expert mode. In: Proc. AVI 2010, pp. 173–180. ACM, NY (2010)Google Scholar
  13. 13.
    Freeman, W.T., Weissman, C.: Television control by hand gestures. In: Int. Workshop on Automatic Face- and Gesture-Recognition, pp. 179–183. IEEE, Los Alamitos (1995)Google Scholar
  14. 14.
    Jones, E., Alexander, J., Andreas Andreou, A., Irani, P., Subramanian, S.: GesText: accelerometer-based gestural text-entry systems. In: Proc. of CHI 2010, pp. 2173–2182. ACM, NY (2010)Google Scholar
  15. 15.
    Kurtenbach, G., Buxton, W.: The limits of expert performance using hierarchic marking menus. In: Proc. of INTERACT 1993 and CHI 1993, pp. 482–487. ACM, NY (1993)Google Scholar
  16. 16.
    Kurtenbach, G., Buxton, W.: User learning and performance with marking menus. In: Proc. ACM CHI 1994, pp. 258–264 (1994)Google Scholar
  17. 17.
    Lenman, S., Bretzner, L., Thuresson, B.: Using marking menus to develop command sets for computer vision based hand gesture interfaces. In: Proc. Of NordiCHI 2002, pp. 239–242. ACM, NY (2002)CrossRefGoogle Scholar
  18. 18.
    Lenman, S., Bretzner, L., Thuresson, B.: Computer Vision Based Recognition of Hand Gestures for Human-Computer Interaction. Technical report TRITANA-D0209 (2002)Google Scholar
  19. 19.
    Lepinski, g.J., Grossman, T., Fitzmaurice, G.: The design and evaluation of multitouch marking menus. In: Proc. of CHI 2010, pp. 2233–2242. ACM, NY (2010)Google Scholar
  20. 20.
    Malik, S., Ranjan, A., Balakrishnan, R.: Interacting with large displays from a distance with vision-tracked multi-finger gestural input. In: Proc. UIST 2005, pp. 43–52. ACM, NY (2005)CrossRefGoogle Scholar
  21. 21.
    Michelis, D., Müller, J.: The Audience Funnel: Observations of Gesture Based Interaction with multiple large displays in a city center. Int. Journal of HCI (to appear)Google Scholar
  22. 22.
    Pavlovic, V., Sharma, R., Huang, T.S.: Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 677–695 (1997)CrossRefGoogle Scholar
  23. 23.
    Peltonen, P., Kurvinen, E., Salovaara, A., Jacucci, G., Ilmonen, T., Evans, J., Oulasvirta, A., Saarikko, P.: It’s Mine, Don’t Touch!: interactions at a large multi-touch display in a city centre. In: Proc. CHI 2008, pp. 1285–1294. ACM, NY (2008)Google Scholar
  24. 24.
    Roudaut, A., Bailly, G., Lecolinet, E., Nigay, L.: Leaf menus: Linear menus with stroke shortcuts for small handheld devices. In: Gross, T., Gulliksen, J., Kotzé, P., Oestreicher, L., Palanque, P., Prates, R.O., Winckler, M. (eds.) INTERACT 2009. LNCS, vol. 5726, pp. 616–619. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  25. 25.
    Schick, A., van de Camp, F., Ijsselmuiden, F., Stiefelhagen, R.: Extending touch: towards interaction with large-scale surfaces. In: Proc. of ITS 2009, pp. 117–124. ACM, NY (2009)Google Scholar
  26. 26.
    Tanvir, E., Cullen, J., Irani, P., Cockburn, A.: AAMU: adaptive activation area menus for improving selection in cascading pull-down menus. In: Proc. of CHI 2008, pp. 1381–1384. ACM, NY (2008)Google Scholar
  27. 27.
    Vogel, D.: Balakrishnan. R. Interactive public ambient displays: transitioning from implicit to explicit, public to personal, interaction with multiple users. In: Proc. of UIST 2004, pp. 137–146. ACM, NY (2004)CrossRefGoogle Scholar
  28. 28.
    Wu, M., Balakrishnan, R.: Multi-finger and whole hand gestural interaction techniques for multi-user tabletop displays. In: Proc. of UIST 2003, pp. 193–202. ACM, NY (2003)CrossRefGoogle Scholar
  29. 29.
    Zhao, S., Balakrishnan, R.: Simple vs. compound mark hierarchical marking menus. In: Proc. UIST 2004, pp. 33–42. ACM, NY (2004)CrossRefGoogle Scholar

Copyright information

© IFIP International Federation for Information Processing 2011

Authors and Affiliations

  • Gilles Bailly
    • 1
    • 2
  • Robert Walter
    • 1
  • Jörg Müller
    • 1
  • Tongyan Ning
    • 1
  • Eric Lecolinet
    • 2
  1. 1.Deutsche Telekom LaboratoriesTU BerlinGermany
  2. 2.Telecom Paristech – CNRS LTCIFrance

Personalised recommendations