Designing Touchless Gestural Interactions for Public Displays In-the-Wild

  • Vito Gentile
  • Alessio MaliziaEmail author
  • Salvatore Sorce
  • Antonio Gentile
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9170)


Public displays, typically equipped with touchscreens, are used for interactions in public spaces, such as streets or fairs. Currently low-cost visual sensing technologies, such as Kinect-like devices and high quality cameras, allow to easily implement touchless interfaces. Nevertheless, the arising interactions have not yet been fully investigated for public displays in-the-wild (i.e. in appropriate social contexts where public displays are typically deployed). Different audiences, cultures and social settings strongly affect users and their interactions. Besides gestures for public displays must be guessable to be easy to use for a wide audience. Issues like these could be solved with user-centered design: gestures must be chosen by users in different social settings, and then selected to be resilient to cultural bias and provide a good level of guessability. Therefore the main challenge is to define touchless gestures in-the-wild by using novel UCD methods applied out of controlled environments, and evaluating their effectiveness.


Public display User-centered design In-the-wild experiment Gestural interaction Natural user interaction Kinect-like devices Ubiquitous computing 


  1. 1.
    Weiser, M.: The computer for the 21st century. Sci. Am. 265(3), 94–104 (1991)CrossRefGoogle Scholar
  2. 2.
    Gentile, V., Sorce, S., Gentile, A.: Continuous hand openness detection using a kinect-like device. In: 2014 Eighth International Conference on Complex, Intelligent and Software Intensive Systems, pp. 553–557. IEEE Press, Birmingham (2014)Google Scholar
  3. 3.
    Müller, J., Bailly, G., Bossuyt, T., Hillgreen, N.: MirrorTouch: combining touch and mid-air gestures for public displays. In: Proceedings of the 16th International Conference on Human-Computer Interaction with Mobile Devices and Services, pp. 319–328. ACM Press, New York (2014)Google Scholar
  4. 4.
    Zhai, Y., Zhao, G., Alatalo, T., Heikkilä, J., Ojala, T., Huang, X.: Gesture interaction for wall-sized touchscreen display. In: Proceedings of the 2013 ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication, pp. 175–178. ACM Press, New York (2013)Google Scholar
  5. 5.
    Hardy, J., Rukzio, E., Davies, N.: Real world responses to interactive gesture based public displays. In: Proceedings of the 10th International Conference on Mobile and Ubiquitous Multimedia, pp. 33–39. ACM Press, New York (2011)Google Scholar
  6. 6.
    Wobbrock, J. O., Morris, M. R., Wilson, A.D.: User-defined gestures for surface computing. In: Proceedings of the 27th International Conference on Human Factors in Computing Systems, pp. 1083–1092. ACM Press, New York (2009)Google Scholar
  7. 7.
    Morris, M.R., Danielescu, A., Drucker, S., Fisher, D., Lee, B., Schraefel, M.C., Wobbrock, J.O.: Reducing legacy bias in gesture elicitation studies. Interactions 21(3), 40–45 (2014)CrossRefGoogle Scholar
  8. 8.
    Rico, J., Brewster, S.: Usable gestures for mobile interfaces: evaluating social acceptability. In: Proceedings of the 28th International Conference on Human Factors in Computing Systems, pp. 887–896. ACM Press, New York (2010)Google Scholar
  9. 9.
    Sharp, R., Rehman, K.: The 2005 UbiApp workshop: what makes good application-led research? IEEE Pervasive Comput. 4(3), 80–82 (2005)CrossRefGoogle Scholar
  10. 10.
    Jurmu, M.: Towards engaging multipurpose public displays. Ph.D. Thesis, University of Oulu (2014).
  11. 11.
    Chamberlain, A., Crabtree, A., Rodden, T., Jones, M., Rogers, Y.: Research in the wild: understanding “in the wild” approaches to design and development. In: Proceedings of Conference on Designing Interactive Systems Conference - DIS 2012, pp. 795–796. ACM Press, New York (2012)Google Scholar
  12. 12.
    Ojala, T., Kostakos, V., Kukka, H., Heikkinen, T., Linden, T., Jurmu, M., Hosio, S., Kruger, F., Zanni, D.: Multipurpose interactive public displays in the wild: three years later. Computer 45(5), 42–49 (2012)CrossRefGoogle Scholar
  13. 13.
    Johnson, R., Rogers, Y., van der Linden, J., Bianchi-Berthouze, N.: Being in the thick of in-the-wild studies: the challenges and insights of researcher participation. In: Proceedings of the 2012 ACM Annual Conference on Human Factors in Computing Systems CHI 2012, pp. 1135–1144. ACM Press, New York (2012)Google Scholar
  14. 14.
    Keskinen, T., Hakulinen, J., Heimonen, T., Turunen, M., Sharma, S., Miettinen, T., Luhtala, M.: Evaluating the experiential user experience of public display applications in the wild. In: Proceedings of the 12th International Conference on Mobile and Ubiquitous Multimedia MUM 2013, pp. 1–10. ACM Press, New York (2013)Google Scholar
  15. 15.
    Messeter, J., Molenaar, D.: Evaluating ambient displays in the wild: highlighting social aspects of use in public settings. In: Proceedings of the Conference on Designing Interactive Systems DIS 2012, pp. 478–481. ACM Press, New York (2012)Google Scholar
  16. 16.
    Hinrichs, U., Carpendale, S.: Gestures in the wild: studying multi-touch gesture sequences on interactive tabletop exhibits. In: Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems - CHI 2011, pp. 3023–3032. ACM Press, New York (2011)Google Scholar
  17. 17.
    Müller, J., Wilmsmann, D., Exeler, J., Buzeck, M., Schmidt, A., Jay, T., Krüger, A.: Display blindness: the effect of expectations on attention towards digital signage. In: Tokuda, H., Beigl, M., Friday, A., Brush, A., Tobe, Y. (eds.) Pervasive 2009. LNCS, vol. 5538, pp. 1–8. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  18. 18.
    Ballendat, T., Marquardt, N., Greenberg, S.: Proxemic interaction: designing for a proximity and orientation-aware environment. In: ACM International Conference on Interactive Tabletops and Surfaces ITS 2010, pp. 121–130. ACM Press, New York (2010)Google Scholar
  19. 19.
    Vogel, D., Balakrishnan, R.: Interactive public ambient displays: transitioning from implicit to explicit, public to personal, interaction with multiple users. In: Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology - UIST 2004, pp. 137–146. ACM Press, New York (2004)Google Scholar
  20. 20.
    Müller, J., Alt, F., Michelis, D., Schmidt, A.: Requirements and design space for interactive public displays. In: Proceedings of the International Conference on Multimedia - MM 2010, pp. 1285–1294. ACM Press, New York (2010)Google Scholar
  21. 21.
    Schönböck, J., König, F., Kotsis, G., Gruber, D., Zaim, E., Schmidt, A.: MirrorBoard – an interactive billboard. In: Proceedings of Mensch und Computer 2008, pp. 207–216. Oldenbourg Verlag, Lübeck (2008)Google Scholar
  22. 22.
    Nielsen, M., Störring, M., Moeslund, T.B., Granum, E.: A procedure for developing intuitive and ergonomic gesture interfaces for HCI. In: Camurri, A., Volpe, G. (eds.) GW 2003. LNCS (LNAI), vol. 2915, pp. 409–420. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  23. 23.
    Good, M.D., Whiteside, J.A., Wixon, D.R., Jones, S.J.: Building a user-derived interface. Commun. ACM 27(10), 1032–1043 (1984)CrossRefGoogle Scholar
  24. 24.
    Van Dam, A.: Post-WIMP user interfaces. Commun. ACM 40(2), 63–67 (1997)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Malizia, A., Bellucci, A.: The artificiality of natural user interfaces. Commun. ACM 55(3), 36–38 (2012)CrossRefGoogle Scholar
  26. 26.
    Brignull, H., Rogers, Y.: Enticing people to interact with large public displays in public spaces. In: Rauterberg, M., Menozzi, M., Wesson, J. (eds.) Proceedings of the IFIP International Conference on Human-Computer Interaction - INTERACT 2003, pp. 17–24. IOS Press, Amsterdam (2003)Google Scholar
  27. 27.
    Fogg, B.J.: Persuasive Technology: Using Computers to Change What We Think and Do. Morgan Kauffman, San Francisco (2002)Google Scholar
  28. 28.
    Sorce, S., Augello, A., Santangelo, A., Gentile, A., Genco, A., Gaglio, S., Pilato, G.: Interacting with augmented environments. IEEE Pervasive Comput. 9(2), 57–58 (2010). doi: 10.1109/MPRV.2010.34. ISSN 1536-1268CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Vito Gentile
    • 1
  • Alessio Malizia
    • 2
    Email author
  • Salvatore Sorce
    • 1
  • Antonio Gentile
    • 1
  1. 1.Dipartimento di Ingegneria Chimica, Gestionale, Informatica e Meccanica (DICGIM)Università degli Studi di PalermoPalermoItaly
  2. 2.Computer Science DepartmentBrunel University LondonUxbridgeUK

Personalised recommendations