Gesture Profile for Web Services: An Event-Driven Architecture to Support Gestural Interfaces for Smart Environments

  • Radu-Daniel Vatavu
  • Cătălin-Marian Chera
  • Wei-Tek Tsai
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7683)


Gestural interfaces have lately become extremely popular due to the introduction on the market of low-cost acquisition devices such as iPhone, Wii, and Kinect. Such devices allow practitioners to design, experiment, and evaluate novel interfaces and interactions for new smart environments. However, gesture recognition algorithms are currently the appanage of machine learning experts which sometimes leaves AmI practitioners dealing with complex pattern recognition techniques instead of focusing on prototyping ambient interactions. To address this problem, we propose GPWS (Gesture Profile for Web Services), a service-oriented architecture (SOA) designed to assist implementation of gestural interfaces. By providing gesture recognition as a web service, we leverage easy and fast adoption of gestural interfaces for various platforms and environments through simple service discovery and composition mechanisms. We discuss two GPWS designs based on SOA 1.0 and SOA 2.0 standards, analyze their performance, and demonstrate GPWS for a gesture-controlled smart home application.


Gesture gesture-based control service-oriented computing event-driven architecture smart home web services gesture recognition SOA EDA 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aarts, E., de Ruyter, B.: New research perspectives on Ambient Intelligence. J. Ambient Intell. Smart Environ. 1(1), 5–14 (2009)Google Scholar
  2. 2.
    Aarts, E., Grotenhuis, F.: Ambient Intelligence 2.0: Towards Synergetic Prosperity. J. Ambient Intell. Smart Environ. 3(1), 3–11 (2011)Google Scholar
  3. 3.
    Anthony, L., Wobbrock, J.O.: A lightweight multistroke recognizer for user interface prototypes. In: Proceedings of Graphics Interface 2010 (GI 2010), pp. 245–252. Canadian Information Processing Society, Toronto (2010)Google Scholar
  4. 4.
    Bernhaupt, R., Obrist, M., Weiss, A., Beck, E., Tscheligi, M.: Trends in the living room and beyond: results from ethnographic studies using creative and playful probing. Comput. Entertain. 6(1), Article 5 (2008)Google Scholar
  5. 5.
    Bonino, D., Corno, F.: What would you ask to your home if it were intelligent? Exploring user expectations about next-generation homes. J. Ambient Intell. Smart Environ. 3(2), 111–126 (2011)Google Scholar
  6. 6.
    Bott, J.N., Crowley, J.G., LaViola Jr., J.J.: Exploring 3D gestural interfaces for music creation in video games. In: Proceedings of the 4th International Conference on Foundations of Digital Games (FDG 2009), pp. 18–25. ACM, New York (2009)CrossRefGoogle Scholar
  7. 7.
    Chakraborty, D., Joshi, A., Finin, T., Yesha, Y.: Service composition for mobile environments. Mob. Netw. Appl. 10(4), 435–451 (2005), doi:10.1145/1160162.1160168CrossRefGoogle Scholar
  8. 8.
    Chera, C.M., Tsai, W.T., Vatavu, R.D.: Gesture Ontology for Informing Service-oriented Architectures. In: IEEE Multi-Conference on Systems and Control (2012)Google Scholar
  9. 9.
    Ghosh, D., Dube, T., Shivaprasad, A.: Script Recognition: A Review. IEEE Trans. Pattern Anal. Mach. Intell. 32(12), 2142–2161 (2010)CrossRefGoogle Scholar
  10. 10.
    Gold, N., Knight, C., Mohan, A., Munro, M.: Understanding Service-Oriented Software. IEEE Softw. 21(2), 71–77 (2004), doi:10.1109/MS.2004.1270766CrossRefGoogle Scholar
  11. 11.
    Harley, D., Fitzpatrick, G., Axelrod, L., White, G., McAllister, G.: Making the Wii at Home: Game Play by Older People in Sheltered Housing. In: Leitner, G., Hitz, M., Holzinger, A. (eds.) USAB 2010. LNCS, vol. 6389, pp. 156–176. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  12. 12.
    Hoffman, M., Varcholik, P., LaViola Jr., J.J.: Breaking the status quo: Improving 3D gesture recognition with spatially convenient input devices. In: Proceedings of the 2010 IEEE Virtual Reality Conference (VR 2010), pp. 59–66. IEEE Computer Society, Washington, DC (2010)CrossRefGoogle Scholar
  13. 13.
    Ibarguren, A., Maurtua, I., Sierra, B.: Layered architecture for real time sign recognition: Hand gesture and movement. Eng. Appl. Artif. Intell. 23(7), 1216–1228 (2010)CrossRefGoogle Scholar
  14. 14.
    Ibrhaim, M.H., Holley, K., Josuttis, N.M., Michelson, B., Thomas, D., de Vadoss, J.: The future of SOA: what worked, what didn’t, and where is it going from here? In: Companion to the 22nd ACM SIGPLAN Conference on Object-Oriented Programming Systems and Applications Companion (OOPSLA 2007), pp. 1034–1038. ACM, New York (2007)Google Scholar
  15. 15.
    Jung, Y., Li, K.J., Janissa, N.S., Gladys, W.L.C., Lee, K.M.: Games for a better life: effects of playing Wii games on the well-being of seniors in a long-term care facility. In: Proceedings of the Sixth Australasian Conference on Interactive Entertainment (IE 2009). ACM, New York (2009)Google Scholar
  16. 16.
    Kühnel, C., Westermann, T., Hemmert, F., Kratz, S., Müller, A., Möller, S.: I’m home: Defining and evaluating a gesture set for smart-home control. International Journal of Human-Computer Studies 69(11), 693–704 (2011)CrossRefGoogle Scholar
  17. 17.
    Lee, J.C.: Hacking the Nintendo Wii Remote. IEEE Pervasive Computing 7(3), 39–45 (2008)CrossRefGoogle Scholar
  18. 18.
    Li, W., Lee, Y.-H., Tsai, W.-T., Xu, J., Son, Y.-S., Park, J.-H., Moon, K.-D.: Service-oriented smart home applications: composition, code generation, deployment, and execution. Serv. Oriented Comput. Appl. 6(1), 65–79 (2012), doi:10.1007/s11761-011-0086-7CrossRefGoogle Scholar
  19. 19.
    Liu, J., Zhong, L., Wickramasuriya, J., Vasudevan, V.: uWave: Accelerometer-based personalized gesture recognition and its applications. Pervasive Mob. Comput. 5(6), 657–675 (2009)CrossRefGoogle Scholar
  20. 20.
    Luckham, D.: The Power of Events: An Introduction to Complex Event Processing in Distributed Enterprise Systems. Addison-Wesley (2002)Google Scholar
  21. 21.
    Maurer, U., Rowe, A., Smailagic, A., Siewiorek, D.P.: eWatch: A Wearable Sensor and Notification Platform. In: Proceedings of the International Workshop on Wearable and Implantable Body Sensor Networks (BSN 2006), pp. 142–145. IEEE Computer Society, Washington, DC (2006)CrossRefGoogle Scholar
  22. 22.
    McArthur, V., Castellucci, S.J., MacKenzie, I.S.: An empirical comparison of ”wiimote” gun attachments for pointing tasks. In: Proceedings of the 1st ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS 2009), pp. 203–208. ACM, New York (2009)CrossRefGoogle Scholar
  23. 23.
    McNeill, D.: Hand and mind: What gestures reveal about thought. University of Chicago Press (1992)Google Scholar
  24. 24.
    Mingkhwan, A., Fergus, P., Abuelma’Atti, O., Merabti, M., Askwith, B., Hanneghan, M.B.: Dynamic service composition in home appliance networks. Multimedia Tools Appl. 29(3), 257–284 (2006), doi:10.1007/s11042-006-0018-2CrossRefGoogle Scholar
  25. 25.
    Mitra, S., Acharya, T.: Gesture Recognition: A Survey. Trans. Sys. Man Cyber Part C 37(3), 311–324 (2007), doi:10.1109/TSMCC.2007.893280CrossRefGoogle Scholar
  26. 26.
    Pan, G., Wu, J., Zhang, D., Wu, Z., Yang, Y., Li, S.: GeeAir: a universal multimodal remote control device for home appliances. Personal Ubiquitous Comput. 14(8), 723–735 (2010)CrossRefGoogle Scholar
  27. 27.
    Panger, G.: Kinect in the kitchen: testing depth camera interactions in practical home environments. In: Proceedings of the 2012 ACM Annual Conference Extended Abstracts on Human Factors in Computing Systems Extended Abstracts (CHI EA 2012), pp. 1985–1990. ACM, New York (2012), doi:10.1145/2212776.2223740CrossRefGoogle Scholar
  28. 28.
    Poppe, R.: Vision-based human motion analysis: An overview. Comput. Vis. Image Underst. 108(1-2), 4–18 (2007)CrossRefGoogle Scholar
  29. 29.
    Poppe, R.: A survey on vision-based human action recognition. Image Vision Comput. 28(6), 976–990 (2010), doi:10.1016/j.imavis.2009.11.014CrossRefGoogle Scholar
  30. 30.
    Mahfujur Rahman, A.S.M., Hossain, M.A., Parra, J., El Saddik, A.: Motion-path based gesture interaction with smart home services. In: Proceedings of the 17th ACM International Conference on Multimedia (MM 2009), pp. 761–764. ACM, New York (2009)CrossRefGoogle Scholar
  31. 31.
    Ruiz, J., Li, Y., Lank, E.: User-defined motion gestures for mobile interaction. In: Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems (CHI 2011), pp. 197–206. ACM, New York (2011)Google Scholar
  32. 32.
    Sadri, F.: Ambient intelligence: A survey. ACM Comput. Surv. 43(4), Article 36, 66 (2011), doi:10.1145/1978802.1978815CrossRefGoogle Scholar
  33. 33.
    Schlömer, T., Poppinga, B., Henze, N., Boll, S.: Gesture recognition with a Wii controller. In: Proceedings of the 2nd International Conference on Tangible and Embedded Interaction (TEI 2008), pp. 11–14. ACM, New York (2008)CrossRefGoogle Scholar
  34. 34.
    van Seghbroeck, G., Verstichel, S., de Turck, F., Dhoedt, B.: WS-Gesture, a gesture-based state-aware control framework. In: Proceedings of the IEEE International Conference on Service-Oriented Computing and Applications, SOCA 2010 (2010)Google Scholar
  35. 35.
    Sonntag, D., Engel, R., Herzog, G., Pfalzgraf, A., Pfleger, N., Romanelli, M., Reithinger, N.: SmartWeb Handheld — Multimodal Interaction with Ontological Knowledge Bases and Semantic Web Services. In: Huang, T.S., Nijholt, A., Pantic, M., Pentland, A. (eds.) ICMI/IJCAI Workshops 2007. LNCS (LNAI), vol. 4451, pp. 272–295. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  36. 36.
    Urbieta, A., Barrutieta, G., Parra, J., Uribarren, A.: A survey of dynamic service composition approaches for ambient systems. In: Proceedings of the 2008 Ambi-Sys Workshop on Software Organisation and MonIToring of Ambient Systems (SOMITAS 2008), Brussels, Belgium (2008)Google Scholar
  37. 37.
    Vatavu, R.D.: Point & Click Mediated Interactions for Large Home Entertainment Displays. Multimedia Tools and Applications 59(1), 113–128 (2012)CrossRefGoogle Scholar
  38. 38.
    Vatavu, R.D.: Nomadic Gestures: A Technique for Reusing Gesture Commands for Frequent Ambient Interactions. Journal of Ambient Intelligence and Smart Environments 4(2), 79–93 (2012)Google Scholar
  39. 39.
    Wang, D.-X., Xiong, Z.-H., Zhang, M.-J.: An application oriented and shape feature based multi-touch gesture description and recognition method. Multimedia Tools and Applications 58(3), 497–519 (2012)CrossRefGoogle Scholar
  40. 40.
    Weinland, D., Ronfard, R., Boyer, E.: A survey of vision-based methods for action representation, segmentation and recognition. Comput. Vis. Image Underst. 115(2), 224–241 (2011)CrossRefGoogle Scholar
  41. 41.
    Wobbrock, J.O., Wilson, A.D., Li, Y.: Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes. In: Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology (UIST 2007), pp. 159–168. ACM, New York (2007), doi:10.1145/1294211.1294238CrossRefGoogle Scholar
  42. 42.
    Wobbrock, J.O., Morris, M.R., Wilson, A.D.: User-defined gestures for surface computing. In: Proceedings of the 27th International Conference on Human Factors in Computing Systems (CHI 2009), pp. 1083–1092. ACM, New York (2009)Google Scholar
  43. 43.
    Xu, J., Lee, Y.-H., Tsai, W.-T., Li, W., Son, Y.-S., Park, J.-H., Moon, K.-D.: Ontology-Based Smart Home Solution and Service Composition. In: Proceedings of the 2009 International Conference on Embedded Software and Systems (ICESS 2009), pp. 297–304. IEEE Computer Society, Washington, DC (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Radu-Daniel Vatavu
    • 1
  • Cătălin-Marian Chera
    • 2
  • Wei-Tek Tsai
    • 3
    • 4
  1. 1.University Stefan cel Mare of SuceavaSuceavaRomania
  2. 2.Computer Science DepartmentPolitehnica University of BucharestRomania
  3. 3.School of Computing, Informatics, and Decision Systems EngineeringArizona State UniversityTempeUSA
  4. 4.Department of Computer Science and TechnologyTsinghua UniversityBeijingChina

Personalised recommendations