Control with Hand Gestures in Home Environment: A Review

Conference paper
First Online:

Abstract

With many advances made in the area of automatic gesture recognition, gestural control has gradually gained its acceptance and popularity. However, the research to date has tended to focus on recognition technologies rather than human behaviors. With an emphasis on users, this paper reviews recent research literature on hand gestural control in home environment. The aim is to summarize and analyze current development processes of gesture vocabularies for commanding home appliances. A semiotic dual triadic model is proposed for the review of the control commands for the appliances, the types of hand gestures, the derivation processes of designer-defined and user-defined gestures. A typical derivation process for a user-defined gesture set was first collecting raw data by inquiring potential users to perform the most suitable gestures that they thought for triggering the given commands or functions, and then applying algorithms for the selection of final gesture vocabularies. A brief comparison among the research results of user-defined freehand gestures for TV control commands was provided as an example to show a need of research in this area. Further research direction includes the exploration of broader user population and the refinement of current gesture selection algorithms.

Keywords

Gestural user interface Human computer interaction Human centered design Gesture set 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgments

This work is funded by the National Science Council of Taiwan (NSC 101-2221-E-027-005-MY3).

References

  1. Aoki R, Karatsu Y, Ihara M, Maeda A, Kobayashi M, Kagami S (2011a) Gesture identification based on zone entry and axis crossing. In: Paper presented at the 14th international conference on human-computer interaction. Orlando, USA, 9–14 July 2011 LNCS 6762-2: 194–203Google Scholar
  2. Aoki R, Karatsu Y, Ihara M, Maeda A, Kobayashi M, Kagami S (2011b) Unicursal gesture interface for TV remote with touch screens. In: Paper presented at the 2011 IEEE international conference on consumer electronics (ICCE). Las Vegas, USA, 9–12 Jan 2011, pp 99–100Google Scholar
  3. Aoki R, Karatsu Y, Ihara M, Maeda A, Kobayashi M, Kagami S (2011c) Expanding kinds of gestures for hierarchical menu selection by unicursal gesture interface. IEEE T Consum Electr 57–2:731–737CrossRefGoogle Scholar
  4. Aoki R, Ihara M, Kobayashi T, Kobayashi M, Chan B, Kagami S (2012) A gesture recognition algorithm for vision-based unicursal gesture interfaces. In: Paper presented at the 10th European conference on interactive TV and video (EuroITV’12). Berlin, Germany, 4–6 July 2012, pp 53–56Google Scholar
  5. Bailly G, Vo D-B, Lecolinet E, Guiard Y (2011a) Gesture-aware remote controls: Guidelines and interaction techniques. In: Paper presented at the 13th international conference on multimodal interfaces (ICMI’11). Alicante, Spain, 14–18 Nov 2011, pp 263–270Google Scholar
  6. Bailly G, Walter R, Müller J, Ning T, Lecolinet E (2011b) Comparing free hand menu techniques for distant displays using linear, marking and finger-count menus. In: Paper presented at the 13th IFIP TC 13 international conference on human-computer interaction (INTERACT’11). Lisbon, Portugal, 5–9 Sept 2011 LNCS 6947-2: 248–262Google Scholar
  7. Bobeth J, Schmehl S, Kruijff E, Deutsch S, Tscheligi M (2012) Evaluating performance and acceptance of older adults using freehand gestures for TV menu control. In: Paper presented at the 10th European conference on interactive TV and video (EuroITV’12). Berlin, Germany, 4–6 July 2012, pp 35–44Google Scholar
  8. Chandler D (2002) Semiotics: the basics. Routledge, New YorkCrossRefGoogle Scholar
  9. Choi E, Kwon S, Lee D, Lee H, Chung MK (2012) Can user-defined gesture be considered as the best gesture for a command? Focusing on the commands for smart home system. In: Paper presented at the human factors and ergonomics society 56th annual meeting. Boston, USA, 22–26 Oct 2012, pp 1253–1257Google Scholar
  10. De Miranda LC, Hornung HH, Baranauskas C (2010) Adjustable interactive rings for iDTV. IEEE T Consum Electr 56(3):1988–1996CrossRefGoogle Scholar
  11. De Miranda LC, Hornung HH, Baranauskas C (2011a) Adjustable interactive rings for iDTV: first results of an experiment with end-users. In: Paper presented at the 14th international conference on human-computer interaction. Orlando, USA, 9–14 July 2011 LNCS 6776: 262–271Google Scholar
  12. De Miranda LC, Hornung HH, Baranauskas C (2011b) Prospecting a new physical artifact of interaction for iDTV: results of participatory practices. In: Paper presented at the 14th international conference on human-computer interaction. Orlando, USA, 9–14 July 2011 LNCS 6770: 167–176Google Scholar
  13. Dezfuli N, Khalilbeigi M, Huber J, Müller FB, Mühlhäuser M (2012) PalmRC: imaginary palm-based remote control for eyes-free television interaction. In: Paper presented at the 10th European conference on interactive TV and video (EuroITV’12). Berlin, Germany, 4–6 July 2012, pp 27–34Google Scholar
  14. Freeman D, Vennelakanti R, Madhvanath S (2012) Freehand pose-based gestural interaction: Studies and implications for interface design. In: Paper presented at the 4th international conference on intelligent human computer interaction. IIT Kharagpur, India, 27–29 Dec 2012, pp 1–6Google Scholar
  15. Guesgen H, Kessell D (2012) Gestural control of household appliances for the physically impaired. In: Paper presented at the 25th international Florida artificial intelligence research society conference. Macro Island, USA, 23–25 May 2012, pp 353–358Google Scholar
  16. Ishikawa T, Horry Y, Hoshino T (2005) Touchless input device and gesture commands. In: Paper presented at the IEEE international conference on consumer electronics. Las Vegas, USA, 8–12 Jan 2005, pp 205–206Google Scholar
  17. Jeong S, Jin J, Song T, Kwon K, Jeon JW (2012) Single-camera dedicated television control system using gesture drawing. IEEE T Consum Electr 58–4:1129–1137CrossRefGoogle Scholar
  18. Jing L, Yamagishi K, Wang J, Zhou Y, Huang T, Cheng Z (2011) A unified method for multiple home appliances control through static finger gestures. In: Paper presented at the IEEE/IPSJ 11th international symposium on applications and the internet. Munich, Germany, 18–21 July 2011, pp 82–90Google Scholar
  19. Kela J, Korpipää P, Mäntyjärvi J, Kallio S, Savino G, Jozzo L, Di Marca S (2006) Accelerometer-based gesture control for a design environment. Pers Ubiquit Comput 10–5:285–299CrossRefGoogle Scholar
  20. Kühnel C, Westermann T, Hemmert F, Kratz S, Müller A, Möller S (2011) I’m home: defining and evaluating a gesture set for smart-home control. Int J Hum Comput St 69:693–704CrossRefGoogle Scholar
  21. Lee S-S, Maeng S, Kim D, Lee K-P, Lee W, Kim S, Jung S (2011) FlexRemote: exploring the effectiveness of deformable user interface as an input device for TV. In: Paper presented at the 14th international conference on human-computer interaction. Orlando, USA, 9–14 July 2011 CCIS 174: 62–65Google Scholar
  22. Liang SFM (2006) Analyzing icon design by the axiomatic method: a case study of alarm icons in process control displays. Asian J Ergon, 7(1–2):11–28Google Scholar
  23. Löcken A, Hesselmann T, Pielot M, Henze N, Boll S (2012) User-centred process for the definition of free-hand gestures applied to controlling music playback. Multimedia Syst 18:15–31CrossRefGoogle Scholar
  24. Morris MR (2012) Web on the wall: insights from a multimodal interaction elicitation study. In: Paper presented at the ACM international conference on interactive tabletops and surfaces. Cambridge, USA, 11–14 Nov 2012, pp 95–104Google Scholar
  25. Norman DA (1983) Some observations on mental models. In: Gentner D, Stevens AL (eds) Mental models. Erlbaum, Hillsdale, pp 7–14Google Scholar
  26. Obaid M, Häring M, Kistler F, Bühling R, André E (2012) User-defined body gestures for navigational control of a humanoid robot. In: Paper presented at the international conference on social robotics. Chengdu, China, 29–31 Oct 2012 LNAI 7621: 367–377Google Scholar
  27. Pan G, Wu J, Zhang D, Wu Z, Yang Y, Li S (2010) GeeAir: a universal multimodal remote control device for home appliances. Pers Ubiquit Comput 14:723–735CrossRefGoogle Scholar
  28. Shimada A, Yamashita T, Taniguchi R (2013) Hand gesture based TV control system. In: Paper presented at the 19th Korea-Japan joint workshop on frontiers of computer vision. Incheon, Korea, 30 Jan–01 Feb 2013, pp 121–126Google Scholar
  29. Stern HI, Wachs JP, Edan Y (2008) Designing hand gesture vocabularies for natural interaction by combining psycho-physiological and recognition factors. Int J Semantic Comput 2(1):137–160CrossRefGoogle Scholar
  30. Tahir M, Bailly G, Lecolinet E (2007) ARemote: A tangible interface for selecting TV channels. In: Paper presented at the 17th international conference on artificial reality and telexistence. Esbjerg, Denmark, 28–30 Nov 2007, pp 298–299Google Scholar
  31. Vatavu R-D (2012) User-defined gestures for free-hand TV control. In: Paper presented at the 10th European conference on interactive TV and video (EuroITV’12). Berlin, Germany, 4–6 July 2012, pp 45–48Google Scholar
  32. Wobbrock JO, Aung HH, Rothrock B, Myers BA (2005) Maximizing the guessability of symbolic input. In: Paper presented at the conference on human factors in computing (CHI). Oregon, USA, 2–7 April 2005 1869-1872Google Scholar
  33. Wu H, Wang J (2012) User-defined body gestures for TV-based applications. In: Paper presented at the 4th international conference on digital home. Guangzhou, China, 23–25 Nov 2012, pp 415–420Google Scholar
  34. Wu H, Wang J (2013) Understanding user preferences for freehand gestures in the TV viewing environment. AISS 5–4:709–717Google Scholar
  35. Wu H, Chen X, Li G (2012) Simultaneous tracking and recognition of dynamic digit gestures for smart TV systems. In: Paper presented at the 4th international conference on digital home. Guangzhou, China, 23–25 Nov 2012, pp 351–356Google Scholar

Copyright information

© Springer Science+Business Media Singapore 2013

Authors and Affiliations

  1. 1.National Taipei University of TechnologyTaipeiTaiwan, Republic of China

Personalised recommendations