Personal and Ubiquitous Computing

, Volume 19, Issue 3–4, pp 551–567

Touch-less interactive augmented reality game on vision-based wearable device

Original Article

Abstract

There is an increasing interest in creating pervasive games based on emerging interaction technologies. In order to develop touch-less, interactive and augmented reality games on vision-based wearable device, a touch-less motion interaction technology is designed and evaluated in this work. Users interact with the augmented reality games with dynamic hands/feet gestures in front of the camera, which triggers the interaction event to interact with the virtual object in the scene. Three primitive augmented reality games with eleven dynamic gestures are developed based on the proposed touch-less interaction technology as proof. At last, a comparing evaluation is proposed to demonstrate the social acceptability and usability of the touch-less approach, running on a hybrid wearable framework or with Google Glass, as well as workload assessment, user’s emotions and satisfaction.

Keywords

Wearable device Smartphone game Hand free Pervasive game Augmented reality game Touch-less 

References

  1. 1.
    Alaa H, Haibo L (2013) Fingerink: turn your glass into a digital board. In: Australian computer-human interaction conference, OzCHI’13, 25–29 November 2013, Adelaide, AustraliaGoogle Scholar
  2. 2.
    Ballagas RA, Kratz SG, Borchers J, Yu E, Walz SP, Fuhr CO, Hovestadt L, Tann M (2007) Rexplorer: a mobile, pervasive spell-casting game for tourists. In: CHI’07 extended abstracts on human factors in computing systems. ACM, pp 1929–1934Google Scholar
  3. 3.
    Bell M, Chalmers M, Barkhuus L, Hall M, Sherwood S, Tennent P, Brown B, Rowland D, Benford S, Capra M, et al (2006) Interweaving mobile games with everyday life. In: Proceedings of the SIGCHI conference on Human Factors in computing systems. ACM, pp 417–426Google Scholar
  4. 4.
    Benford S, Magerkurth C, Ljungstrand P (2005) Bridging the physical and digital in pervasive gaming. Commun ACM 48(3):54–57CrossRefGoogle Scholar
  5. 5.
    Birk M, Mandryk RL (2013) Control your game-self: effects of controller type on enjoyment, motivation, and personality in game. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’13, 2013. ACM, New York, pp 685–694Google Scholar
  6. 6.
    Broll W, Ohlenburg J, Lindt I, Herbst I, Braun A-K (2006) Meeting technology challenges of pervasive augmented reality games. In: Proceedings of 5th ACM SIGCOMM workshop on network and system support for games. ACM, p 28Google Scholar
  7. 7.
    Capra M, Radenkovic M, Benford S, Oppermann L, Drozd A, Flintham M (2005) The multimedia challenges raised by pervasive games. In: Proceedings of the 13th annual ACM international conference on multimedia. ACM, pp 89–95Google Scholar
  8. 8.
    Chan K (2011) Playing in traffic: pervasive gaming for commuters. In: Proceedings of the 7th Australasian conference on interactive entertainment. Interactive Entertainment, p 4Google Scholar
  9. 9.
    Cheok AD, Yang X, Ying ZZ, Billinghurst M, Kato H (2002) Touch-space: mixed reality game space based on ubiquitous, tangible, and social computing. Pers Ubiquitous Comput 6(5–6):430–442CrossRefGoogle Scholar
  10. 10.
    Coenen T, Mostmans L, Naessens K (2013) Museus: case study of a pervasive cultural heritage serious game. J Comput Cultural Herit (JOCCH) 6(2):8Google Scholar
  11. 11.
    Crossan A, Brewster S, Ng A (2010) Foot tapping for mobile interaction. In: Proceedings of the 24th BCS interaction specialist group conference, BCS ’10. British Computer Society, Swinton, pp 418–422Google Scholar
  12. 12.
    de Freitas AA, Dey AK (2015) The group context framework: an extensible toolkit for opportunistic grouping and collaboration. In: Proceedings of the 18th ACM conference on computer supported cooperative work & #38; Social Computing, CSCW ’15, 2015. ACM, New York, pp 1602–1611Google Scholar
  13. 13.
    Duffner S, Garcia C (December 2013) Pixeltrack: a fast adaptive algorithm for tracking non-rigid objects. In: 2013 IEEE international conference on computer vision (ICCV), pp 2480–2487Google Scholar
  14. 14.
    Ermi L, Mäyrä F (2015) Challenges for pervasive mobile game design: examining players’ emotional responses. In: Proceedings of the 2005 ACM SIGCHI international conference on advances in computer entertainment technology. ACM, pp 371–372Google Scholar
  15. 15.
    Erol A, Bebis G, Nicolescu M (2007) Vision-based hand pose estimation: a review. Special issue on vision for human-computer interaction 108(12):52–73 Special Issue on Vision for Human-Computer InteractionCrossRefGoogle Scholar
  16. 16.
    Faulkner X (2002) Usability engineering. Palgrave Macmillan, New YorkGoogle Scholar
  17. 17.
    Felzenszwalb P, Zabih R (2011) Dynamic programming and graph algorithms in computer vision. IEEE Trans Pattern Anal Mach Intell 33(4):721–740CrossRefGoogle Scholar
  18. 18.
    Felzenszwalb PF, Zabih R (2011) Dynamic programming and graph algorithms in computer vision. PAMI 33(4):721–740CrossRefGoogle Scholar
  19. 19.
    Fitz-Walter Z, Tjondronegoro D, Koh D, Zrobok M (2012) Mystery at the library: encouraging library exploration using a pervasive mobile game. In: Proceedings of the 24th Australian computer-human interaction conference. ACM, pp 142–145Google Scholar
  20. 20.
    Gentes A, Guyot-Mbodji A, Demeure I (2010) Gaming on the move: urban experience as a new paradigm for mobile pervasive game design. Multimed Syst 16(1):43–55CrossRefGoogle Scholar
  21. 21.
    Godec M, Roth P, Bischof H (2013) Hough-based tracking of non-rigid objects. Comput Vis Image Underst 117(10):1245–1256CrossRefGoogle Scholar
  22. 22.
    Guo B, Yu Z, Zhang D, He H, Tian J, Zhou X (2014) Toward a group-aware smartphone sensing system. Pervasive Comput IEEE 13(4):80–88CrossRefGoogle Scholar
  23. 23.
    Herbst I, Braun A-K, McCall R, Broll W (2008) Timewarp: interactive time travel with a mobile mixed reality game. In: Proceedings of the 10th international conference on human computer interaction with mobile devices and services. ACM, pp 235–244Google Scholar
  24. 24.
    Jiang D, Xu Z, Li W, Chen Z (2015) Network coding-based energy-efficient multicast routing algorithm for multi-hop wireless networks. J Syst Softw 104:152–165CrossRefGoogle Scholar
  25. 25.
    Jiang D, Xu Z, Zhang P Zhu T (2014) A transform domain-based anomaly detection approach to network-wide traffic. J Netw Comput ApplGoogle Scholar
  26. 26.
    Jonsson S, Waern A (2008) The art of game-mastering pervasive games. In: Proceedings of the 2008 international conference on advances in computer entertainment technology. ACM, pp 224–231Google Scholar
  27. 27.
    Kalal Z, Mikolajczyk K, Matas J (2012) Tracking-learning-detection. IEEE Trans Pattern Anal Mach Intell 34(7):1409–1422CrossRefGoogle Scholar
  28. 28.
    Kasapakis V, Gavalas D (2014) Blending history and fiction in a pervasive game prototype. In: Proceedings of the 13th international conference on mobile and ubiquitous multimedia, MUM ’14. ACM, New York, pp 116–122Google Scholar
  29. 29.
    Kasapakis V, Gavalas D, Bubaris N (2013) Pervasive games research: a design aspects-based state of the art report. In: Proceedings of the 17th panhellenic conference on informatics. ACM, pp 152–157Google Scholar
  30. 30.
    Kasapakis V, Gavalas D, Chatzidimitris T (2015) Evaluation of pervasive games: recruitment of qualified participants through preparatory game phases. arXiv preprint. arXiv:1501.02661
  31. 31.
    Lehner U, Baldauf M, Eranti V, Reitberger W, Fröhlich P (2014) Civic engagement meets pervasive gaming: towards long-term mobile participation. In: CHI’14 extended abstracts on human factors in computing systems. ACM, pp 1483–1488Google Scholar
  32. 32.
    Li X, Lv Z, Hu J, Zhang B, Yin L, Zhong C, Wang W, Feng S (2015) Traffic management and forecasting system based on 3d gis. In: 15th IEEE/ACM international conference on cluster, cloud and grid computing (CCGrid), 2015. IEEEGoogle Scholar
  33. 33.
    Li X, Lv Z, Zhang B, Wang W, Feng S, Hu J (2015) Webvrgis based city bigdata 3d visualization and analysis. In: Visualization symposium (PacificVis), 2015 IEEE Pacific. IEEEGoogle Scholar
  34. 34.
    Lindt I, Ohlenburg J, Pankoke-Babatz U, Ghellal S (2007) A report on the crossmedia game epidemic menace. Comput Entertain (CIE) 5(1):8CrossRefGoogle Scholar
  35. 35.
    Lv Z (2013) Wearable smartphone: wearable hybrid framework for hand and foot gesture interaction on smartphone. In: proceedings of the 2013 IEEE international conference on computer vision workshops, ICCVW ’13. IEEE, SydneyGoogle Scholar
  36. 36.
    Lv Z, Chen G, Zhong C, Han Y, Qi YY (2012) A framework for multi-dimensional webgis based interactive online virtual community. Adv Sci Lett 7(1):215–219CrossRefGoogle Scholar
  37. 37.
    Lv Z, Esteve C, Chirivella J, Gagliardo P (2015) A game based assistive tool for rehabilitation of dysphonic patients. In: 3rd workshop on virtual and augmented assistive technology (VAAT), 2015. IEEEGoogle Scholar
  38. 38.
    Lv Z, Esteve C, Chirivella J, Gagliardo P (2015) Serious game based dysphonic rehabilitation tool. In 2015 international conference on virtual rehabilitation (ICVR). IEEEGoogle Scholar
  39. 39.
    Lv Z et al (2013) Finger in air: touch-less interaction on smartphone. In: Proceedings of the 12th ACM international conference on mobile and ubiquitous multimedia, MUM ’13, 2013. ACM, LuleåGoogle Scholar
  40. 40.
    Lv Z, Feng L, Feng S, Li H (2015) Extending touch-less interaction on vision based wearable device. In: Virtual reality (VR), 2015 IEEE. IEEEGoogle Scholar
  41. 41.
    Lv Z, Feng L, Li H, Feng S (2014) Hand-free motion interaction on Google glass. In: SIGGRAPH Asia 2014 mobile graphics and interactive applications. ACMGoogle Scholar
  42. 42.
    Lv Z, Halawani A, Feng S, Li H, Ur Réhman S (2014) Multimodal hand and foot gesture interaction for handheld devices. ACM Trans Multimed Comput Commun Appl (TOMM) 11(1s):1–19Google Scholar
  43. 43.
    Lv Z, Khan MSL, Réhman S (2013) Hand and foot gesture interaction for handheld devices. In: Proceedings of the 21st ACM international conference on multimedia, MM ’13, 2013. ACM, BarcelonaGoogle Scholar
  44. 44.
    Lv Z, Li H (2015) Imagining in-air interaction for hemiplegia sufferer. In: 2015 international conference on virtual rehabilitation (ICVR). IEEEGoogle Scholar
  45. 45.
    Lv Z, Li X, Zhang B, Wang W, Feng S, Hu J (2015) Big city 3d visual analysis. In: Eurographics 2015Google Scholar
  46. 46.
    Lv Z, Su T (2014) 3d seabed modeling and visualization on ubiquitous context. In: SIGGRAPH Asia 2014 posters. ACM, p 33Google Scholar
  47. 47.
    Lv Z, Tek A, Da Silva F, Empereur-mot C, Chavent M, Baaden M (2013) Game on, science-how video game technology may help biologists tackle visualization challenges. PloS one 8(3):e57990CrossRefGoogle Scholar
  48. 48.
    Lv Z, Yin T, Han Y, Chen Y, Chen G (2011) Webvr-web virtual reality engine based on p2p network. J Netw 6(7):990–998Google Scholar
  49. 49.
    Magnusson C, Waern A, Gröhn KR, Bjernryd Å, Bernhardsson H, Jakobsson A, Salo J, Wallon M, Hedvall P-O (2011) Navigating the world and learning to like it: mobility training through a pervasive game. In: Proceedings of the 13th international conference on human computer interaction with mobile devices and services . ACM, pp 285–294Google Scholar
  50. 50.
    Mayo MJ (2007) Games for science and engineering education. Commun ACM 50(7):30–35CrossRefGoogle Scholar
  51. 51.
    Mistry P, Maes P, Chang L (2009) Wuw-wear ur world: a wearable gestural interface. In: CHI ’09 extended abstracts on human factors in computing systems, CHI EA ’09. ACM, New York, pp 4111–4116Google Scholar
  52. 52.
    Mittal A, Zisserman A, Torr PHS (2011) Hand detection using multiple proposals. In: British machine vision conferenceGoogle Scholar
  53. 53.
    Montola M, Stenros J, Waern A (2009) Pervasive games: theory and design. Morgan Kaufmann Publishers Inc., San FranciscoGoogle Scholar
  54. 54.
    Nohre R (1996) Deformed template matching by the viterbi algorithm. Pattern Recogn Lett 17(14):1423–1428CrossRefGoogle Scholar
  55. 55.
    Oliveira T, Carvalho L, Ferreira E (2013) Ectodiegesis as immersive effect in pervasive games. In: Proceedings of international conference on making sense of converging media. ACM, p 281Google Scholar
  56. 56.
    Paelke V, Reimann C, Stichling D (2004) Foot-based mobile interaction with games. In: Proceedings of the 2004 ACM SIGCHI international conference on advances in computer entertainment technology, ACE ’04. ACM, New York, pp 321–324Google Scholar
  57. 57.
    Petrakis EGM, Diplaros A, Milios E (2002) Matching and retrieval of distorted and occluded shapes using dynamic programming. IEEE Trans Pattern Anal Mach Intell 24(11):1501–1516CrossRefGoogle Scholar
  58. 58.
    Qian C, Sun X, Wei Y, Tang X, Sun J (2014) Realtime and robust hand tracking from depth. In: IEEE conference on computer vision and pattern recognition (CVPR)Google Scholar
  59. 59.
    Réhman S, Khan A, Li H (2012) Interactive feet for mobile immersive interaction. In: ACM international, workshop MobiVis Workshop at MobileHCIGoogle Scholar
  60. 60.
    Rico J, Brewster S (2010) Usable gestures for mobile interfaces: evaluating social acceptability. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’10. ACM, New York, pp 887–896Google Scholar
  61. 61.
    Ross J (2011) Pervasive negabehavior games for environmental sustainability. In: CHI’11 extended abstracts on human factors in computing systems. ACM, pp 1085–1088Google Scholar
  62. 62.
    Scherer K (January 2005) What are emotions? And how can they be measured? Social science informationGoogle Scholar
  63. 63.
    Simon J, Jahn M, Al-Akkad A (2012) Saving energy at work: the design of a pervasive game for office spaces. In: Proceedings of the 11th international conference on mobile and ubiquitous multimedia. ACM, p 9Google Scholar
  64. 64.
    Su T, Lv Z, Gao S, Li X, Lv H (2014) 3d seabed: 3d modeling and visualization platform for the seabed. In: 2014 IEEE international conference on multimedia and expo workshops (ICMEW). IEEE, pp 1–6Google Scholar
  65. 65.
    Tan J, Fan X, Ren Y (2014) Methodology for geographical data evolution: three-dimensional particle-based real-time snow simulation with remote-sensing data. J Appl Remote Sens 8(1):084598–084598CrossRefGoogle Scholar
  66. 66.
    Tek A, Laurent B, Piuzzi M, Lu Z, Chavent M, Baaden M, Delalande O, Martin C, Piccinali L, Katz B, et al (2012) Advances in human-protein interaction-interactive and immersive molecular simulations. In: Biochemistry, genetics and molecular biology ‘protein–protein interactions-computational and experimental tools’, pp 27–65Google Scholar
  67. 67.
    Tutzschke J-P, Zukunft O (2009) Frap: a framework for pervasive games. In: Proceedings of the 1st ACM SIGCHI symposium on engineering interactive computing systems. ACM, pp 133–142Google Scholar
  68. 68.
    Wachs JP, Kölsch M, Stern H, Edan Y (2011) Vision-based hand-gesture applications. Commun ACM 54(2):60–71CrossRefGoogle Scholar
  69. 69.
    Waern A, Ahmet Z, Sundström D (2009) An in-game reporting tool for pervasive games. In: Proceedings of the international conference on advances in computer entertainment technology. ACM, pp 240–248Google Scholar
  70. 70.
    Waern A, Montola M, Stenros J (2009) The three-sixty illusion: designing for immersion in pervasive games. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, pp 1549–1558Google Scholar
  71. 71.
    Walther BK (2005) Atomic actions-molecular experience: theory of pervasive gaming. Comput Entertain (CIE) 3(3):4–4CrossRefGoogle Scholar
  72. 72.
    Wang Y, Nandi A, Agrawal G (2014) Saga: array storage as a db with support for structural aggregations. In: SSDBM ’14. ACM, New YorkGoogle Scholar
  73. 73.
    Wigdor D, Forlines C, Baudisch P, Barnwell J, Shen C (2007) Lucid touch: a see-through mobile device. In: Proceedings of the 20th annual ACM symposium on user interface software and technology, UIST ’07. ACM, New York, pp 269–278Google Scholar
  74. 74.
    Wilcoxon F (1945) Individual comparisons by ranking methods. Biom Bull 1(6):80–83CrossRefGoogle Scholar
  75. 75.
    Xu C, Cheng L (2013) Efficient hand pose estimation from a single depth image. In: International conference on computer vision (ICCV)Google Scholar
  76. 76.
    Yang J, Ding Z, Guo F, Wang H (2014) Multiview image rectification algorithm for parallel camera arrays. J Electron Imaging 23(3):033001–033001CrossRefGoogle Scholar
  77. 77.
    Yang J, Liu Y, Meng Q, Chu R (2015) Objective evaluation criteria for stereo camera shooting quality under different shooting parameters and shooting distances. IEEE Sens J (99):1. doi:10.1109/JSEN.2015.2421518
  78. 78.
    Yousefi S, Abedan Kondori F, Li H (2013) Experiencing real 3d gestural interaction with mobile devices. Pattern Recogn Lett 34(8):912–921CrossRefGoogle Scholar
  79. 79.
    Zhang M, Lv Z, Zhang X, Chen G, Zhang K (2009) Research and application of the 3d virtual community based on WEBVR and RIA. Comput Inform Sci 2(1):P84Google Scholar
  80. 80.
    Zhang S, Jing H (2014) Fast log-gabor-based nonlocal means image denoising methods. In: 2014 IEEE international conference on image processing (ICIP). IEEE, pp 2724–2728Google Scholar
  81. 81.
    Zou S, Xiao H, Wan H, Zhou X (2009) Vision-based hand interaction and its application in pervasive games. In: Proceedings of the 8th international conference on virtual reality continuum and its applications in industry. ACM, pp 157–162Google Scholar

Copyright information

© Springer-Verlag London 2015

Authors and Affiliations

  1. 1.SIATChinese Academy of ScienceShenzhenChina
  2. 2.Department of Applied Physics & ElectronicsUmeå UniversityUmeåSweden
  3. 3.Royal Institute of Technology (KTH)StockholmSweden
  4. 4.Computer Engineering and Science DepartmentPalestine Polytechnic UniversityHebronPalestine

Personalised recommendations