uCanvas: A Web Framework for Spontaneous Smartphone Interaction with Ubiquitous Displays

  • Tilman DinglerEmail author
  • Tobias Bagg
  • Yves Grau
  • Niels Henze
  • Albrecht Schmidt
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9298)


In recent years the presence of displays has become ubiquitous. They range from small-sized screens, such as smartphones or tablets to large screens as they are found in projection screens or public displays. Each display requires a unique modality of interaction, such as a dedicated input device, direct touch or does not provide any interaction at all. With the ubiquity of smartphones people carry with them a high-end interaction device that can connect to any web-connected screen. To allow quick access, we built uCanvas (“Ubiquitous Canvas”), a system to engage with interactive surfaces. In contrast to previous work no additional hardware is required, nor do users need to install any proprietary software. Our system runs on all current smartphones equipped with magnetometer and accelerometer, which is used to define a canvas and transmit cursor positions to a server connected to the display. To integrate interactive surfaces into applications, we created a lean Javascript library that allows publishers to specify interaction parameters (such as pointing, clicking, menu selection and text entry) by adding just a few lines of code. We built two example applications to evaluate the feasibility of the system and findings show that (1) interaction is intuitive and (2) easy to set up on the user side.


Mobile interaction Public display Ubiquitous computing Magnetometer Pointing Smart environments Toolkit 



The authors acknowledge the financial support of the Future and Emerging Technologies (FET) programme within the 7th Framework Programme for Research of the European Commission, under FET grant number 612933 (RECALL). We further thank our study participants.


  1. 1.
    Ballagas, R., Rohs, M., Sheridan, J.G.: Sweep and point and shoot: phonecam-based interactions for large public displays. In: CHI 2005 Extended Abstracts on Human factors in Computing Systems. ACM, pp. 1200–1203 (2005)Google Scholar
  2. 2.
    Bolt, R.A.: Put-that-there: voice and gesture at the graphics interface. In: Proceedings of the 7th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1980. ACM, New York, NY, USA, pp. 262–270 (1980)Google Scholar
  3. 3.
    Boring, S., Jurmu, M., Butz, A.: Scroll, tilt or move it: using mobile phones to continuously control pointers on large public displays. In: Proceedings of the 21st Annual Conference of the Australian Computer-Human Interaction Special Interest Group: Design: Open 24/7. ACM, pp. 161–168 (2009)Google Scholar
  4. 4.
    Derthick, K., Scott, J., Villar, N., Winkler, C.: Exploring smartphone-based web user interfaces for appliances. In: Proceedings of the 15th International Conference on Human-Computer Interaction with Mobile Devices and Services. ACM, pp. 227–236 (2013)Google Scholar
  5. 5.
    Hardy, R., Rukzio, E.: Touch & interact: touch-based interaction of mobile phones with displays. In: Proceedings of the 10th International Conference on Human Computer Interaction with Mobile Devices and Services. ACM, pp. 245–254 (2008)Google Scholar
  6. 6.
    Magnusson, C., Molina, M., Rassmus-Grohn, K., Szymczak, D.: Pointing for non-visual orientation and navigation. In: Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries. ACM, pp. 735–738 (2010)Google Scholar
  7. 7.
    Seifert, J., Bayer, A., Rukzio, E.: PointerPhone: using mobile phones for direct pointing interactions with remote displays. In: Kotzé, P., Marsden, G., Lindgaard, G., Wesson, J., Winckler, M. (eds.) INTERACT 2013, Part III. LNCS, vol. 8119, pp. 18–35. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  8. 8.
    Weiser, M.: The computer for the 21st century. Sci. Am. 265(3), 94–104 (1991)CrossRefGoogle Scholar

Copyright information

© IFIP International Federation for Information Processing 2015

Authors and Affiliations

  • Tilman Dingler
    • 1
    Email author
  • Tobias Bagg
    • 1
  • Yves Grau
    • 1
  • Niels Henze
    • 1
  • Albrecht Schmidt
    • 1
  1. 1.VISUniversity of StuttgartStuttgartGermany

Personalised recommendations