Interacting with a Tabletop Display Using a Camera Equipped Mobile Phone

  • Seokhee Jeon
  • Gerard J. Kim
  • Mark Billinghurst
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4551)


Today’s mobile phones have not only become the most representative device in the new ubiquitous computing era but also dramatically improved in terms of their multi-modal sensing and display capabilities. This advance makes the mobile phone an ideal candidate for a more natural interaction device in ubiquitous computing environment. This paper proposes techniques which use camera-equipped mobile phones for interacting with 2D and 3D applications on a tabletop display environment. The camera acts as the main sensor for a gesture-based interaction. Using the mobile phone with an interactive touch screen allows the use of techniques that move beyond single hand/finger input to improve task performance. The interaction performances of the proposed techniques and design guidelines are also described in this paper.


Tabletop Motion flow Interaction techniques Cell/Mobile phones 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Fitzmaurice, G.W., Zhai, S., Chignell, M.H.: Virtual Reality for Palmtop Computers. ACM Transactions on Information Systems 11, 197–218 (1993)CrossRefGoogle Scholar
  2. 2.
    Watsen, K., Darken, R.P., Capps, M.: A Handheld Computer as an Interaction Device to a Virtual Environment. In: Proceedings of Intern. Projection Technologies Workshop (1999)Google Scholar
  3. 3.
    Kukimoto, N., Furusho, Y., Nonaka, J., Koyamada, K., Kanazawa, M.: PDA-based Visualization Control and Annotation Interface for Virtual Environment. In: Proceeding of 3rd IASTED International Conference Visualization, Image and Image Processing (2003)Google Scholar
  4. 4.
    Bayon, V., Griffiths, G.: Co-located interaction in virtual environments via de-coupled interfaces. In: HCI International 2003 (2003)Google Scholar
  5. 5.
    Madhavapeddy, A., Scott, D., Sharp, R., Upton, E.: Using camera-phones to enhance human-computer interaction. In: 6th International Conference on Ubiquitous Computing (2004)Google Scholar
  6. 6.
    Lourakis, M.I.A., Argyros, A.A.: Efficient, causal camera tracking in unprepared environments. Computer Vision and Image Understanding 99, 259–290 (2005)CrossRefGoogle Scholar
  7. 7.
    Wagner, D., Schmalstieg, D.: First Steps Towards Handheld Augmented Reality. In: Proceedings of the 7th International Conference on Wearable Computers (2003)Google Scholar
  8. 8.
    Paelke, V., Reimann, C., Stichling, D.: Foot-based Mobile Interaction with Games. In: ACM SIGCHI International Conference on Advances in Computer Entertainment Technology (ACE) (2004)Google Scholar
  9. 9.
    Hachet, M., Kitamura, Y.: 3D Interaction With and From Handheld Computers. In: IEEE VR 2005 Workshop: New Directions in 3D User Interfaces (2005)Google Scholar
  10. 10.
    Hachet, M., Pouderoux, J., Guitton, P.: A camera-based interface for interaction with mobile handheld computers. In: Proceedings of the 2005 symposium on Interactive 3D graphics and games Washington, District of Columbia (2005)Google Scholar
  11. 11.
    Rohs, M., Zweifel, P.: A Conceptual Framework for Camera Phone-based Interaction Techniques. In: Gellersen, H.-W., Want, R., Schmidt, A. (eds.) PERVASIVE 2005. LNCS, vol. 3468, pp. 171–189. Springer, Heidelberg (2005)Google Scholar
  12. 12.
  13. 13.
    Bouguet, J.Y.: Pyramidal Implementation of the Lucas Kanade Feature Tracker Description of the Algorithm (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Seokhee Jeon
    • 1
  • Gerard J. Kim
    • 2
  • Mark Billinghurst
    • 3
  1. 1.VRPM Lab., Dept. Of CSE, POSTECHKorea
  2. 2.DXP Lab., Dept. Of CSE, Korea UniversityKorea
  3. 3.HIT Lab NZ , University of CanterburyNew Zealand

Personalised recommendations