Cooperative Augmentation of Smart Objects with Projector-Camera Systems

  • David Molyneaux
  • Hans Gellersen
  • Gerd Kortuem
  • Bernt Schiele
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4717)


In this paper we present a new approach for cooperation between mobile smart objects and projector-camera systems to enable augmentation of the surface of objects with interactive projected displays. We investigate how a smart object’s capability for self description and sensing can be used in cooperation with the vision capability of projector-camera systems to help locate, track and display information onto object surfaces in an unconstrained environment. Finally, we develop a framework that can be applied to distributed projector-camera systems, cope with varying levels of description knowledge and different sensors embedded in an object.


Cooperative Augmentation Smart Objects Projector-Camera Systems 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bandyopadhyay, D., Raskar, R., Fuchs, H.: Dynamic Shader Lamps: Painting on Movable Objects. In: Proc. IEEE and ACM Int. Symposium on Augmented Reality, New York (2001)Google Scholar
  2. 2.
    Ehnes, J., Hirota, K., Hirose, M.: Projected Augmentation – Augmented Reality using Rotatable Video Projectors. In: Third IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2004), September-October 2004, Arlington, VA, USA (2004)Google Scholar
  3. 3.
    Kato, H., Billinghurst, M.: Marker Tracking and HMD Calibration for a video-based Augmented Reality Conferencing System. In: Proceedings of the 2nd International Workshop on Augmented Reality (IWAR 1999), October 1999, San Francisco, USA (1999)Google Scholar
  4. 4.
    Borkowski, S., Riff, O., Crowley, J.: Projecting Rectified Images In an Augmented Environment. In: IEEE International Workshop on Projector-Camera Systems (PROCAMS-2003), October 12, 2003, Nice, France (2003)Google Scholar
  5. 5.
    Lee, J.C., Hudson, S.E., Summet, J.W., Dietz, P.H.: Moveable Interactive Projected Displays Using Projector Based Tracking. In: Proceedings of the ACM Symposium on User Interface Software and Technology (UIST), October 23-26, 2005, Seattle, WA, pp. 63–72 (2005)Google Scholar
  6. 6.
    Summet, J., Sukthankar, R.: Tracking Locations of Moving Hand-held Displays Using Projected Light. In: Gellersen, H.-W., Want, R., Schmidt, A. (eds.) PERVASIVE 2005. LNCS, vol. 3468, Springer, Heidelberg (2005)Google Scholar
  7. 7.
    Raskar, R., Beardsley, P., van Baar, J., Wang, Y., Dietz, P., Lee, J., Leigh, D., Willwatcher, T.: RFIG Lamps: Interacting with a Self-Describing World via Photosensing Wireless Tags and Projectors. In: Proceedings of SIGGRAPH 2004, Los Angeles, USA (2004)Google Scholar
  8. 8.
    Levas, A., Pinhanez, C., Pingali, G., Kjeldsen, R., Podlaseck, M., Sukaviriya, N.: An Architecture and Framework for Steerable Interface Systems. In: Dey, A.K., Schmidt, A., McCarthy, J.F. (eds.) UbiComp 2003. LNCS, vol. 2864, Springer, Heidelberg (2003)Google Scholar
  9. 9.
    Gellersen, H., Beigl, M., Krull, H.: The MediaCup: Awareness Technology embedded in an Everyday Object. In: Gellersen, H.-W. (ed.) HUC 1999. LNCS, vol. 1707, pp. 308–310. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  10. 10.
    Strohbach, M., Gellersen, H.-W., Kortuem, G., Kray, C.: Cooperative Artefacts: Assessing Real World Situations with Embedded Technology. In: Davies, N., Mynatt, E.D., Siio, I. (eds.) UbiComp 2004. LNCS, vol. 3205, Springer, Heidelberg (2004)Google Scholar
  11. 11.
    Schmidt, A., Strohbach, M., Van Laerhoven, K., Friday, A., Gellersen, H.-W.: Context Acquisition based on Load Sensing. In: Boriello, G., Holmquist, L.E. (eds.) Proceedings of Ubicomp 2002, September 2002. LNCS, vol. 2498, pp. 333–351. Springer, Gothenburg, Sweden (2002)Google Scholar
  12. 12.
    Schmidt, A., Strohbach, M., Van Laerhoven, K., Gellersen, H.W.: Ubiquitous Interaction - Using Surfaces in Everyday Environments as Pointing Devices. In: 7th ERCIM Workshop User Interfaces For All (23 - 25 October, 2002)Google Scholar
  13. 13.
    Strohbach, M., Gellersen, H.-W., Kortuem, G., Kray, C.: Cooperative Artefacts: Assessing Real World Situations with Embedded Technology. In: Davies, N., Mynatt, E.D., Siio, I. (eds.) UbiComp 2004. LNCS, vol. 3205, Springer, Heidelberg (2004)Google Scholar
  14. 14.
    Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. Computer Vision Pattern Recognition, 246–252 (1999)Google Scholar
  15. 15.
    Karitsuka, T., Sato, K.: A Wearable Mixed Reality with On-board Projector. In: Second IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2003), 7-10 October 2003, Tokyo, Japan (2003)Google Scholar
  16. 16.
    Pinhanez, C.: The Everywhere Displays Projector: A Device to Create Ubiquitous Graphical Interfaces. In: Abowd, G.D., Brumitt, B., Shafer, S. (eds.) Ubicomp 2001: Ubiquitous Computing. LNCS, vol. 2201, Springer, Heidelberg (2001)CrossRefGoogle Scholar
  17. 17.
    Butz, A., Schneider, M., Spassova, M.: Searchlight: A Lightweight Search Function for Pervasive Environments. In: Ferscha, A., Mattern, F. (eds.) PERVASIVE 2004. LNCS, vol. 3001, pp. 21–23. Springer, Heidelberg (2004)Google Scholar
  18. 18.
    Bimber, O., Raskar, R.: Spatial Augmented Reality Merging Real and Virtual Worlds. A K Peters LTD (publisher), ISBN: 1-56881-230-2Google Scholar
  19. 19.
    Fujii, K., Grossberg, M.D., Nayar, S.K.: A Projector-Camera System with Real-Time Photometric Adaptation for Dynamic Environments. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 814–821 (2005)Google Scholar
  20. 20.
    Decker, C., Beigle, M., Krohn, A., Robinson, P., Kubach, U.: eSeal – a system for enhanced electronic assertion of authenticity and integrity. In: Ferscha, A., Mattern, F. (eds.) PERVASIVE 2004. LNCS, vol. 3001, Springer, Heidelberg (2004)Google Scholar
  21. 21.
    Mikolajczyk, K., Schmid, C.: Scale and Affine invariant interest point detectors. International Journal of Computer Vision (IJCV) 60(1), 63–86 (2004)CrossRefGoogle Scholar
  22. 22.
    Lowe, D.: SIFT: Distinctive image features from scale invariant keypoints. International Journal of Computer Vision (IJCV) 60(2), 91–110 (2004)CrossRefGoogle Scholar
  23. 23.
    Decker, C., Krohn, A., Beigl, M., Zimmer, T.: The Particle Computer System, IPSN Track on Sensor Platform, Tools and Design Methods for Networked Embedded Systems (SPOTS). In: Proceedings of the ACM/IEEE 4th International Conference on Information Processing in Sensor Networks (IPSN 2005), April 2005, Los Angeles, pp. 443–448 (2005)Google Scholar
  24. 24.
    Brooks Jr., F.P.: What’s Real About Virtual Reality? IEEE Computer Graphics and Applications 19(6), 16–27 (1999)CrossRefGoogle Scholar
  25. 25.
    Isard, M., Blake, A.: CONDENSATION – conditional density propagation for visual tracking. International Journal of Computer Vision (IJCV) 29(1), 5–28 (1998)CrossRefGoogle Scholar
  26. 26.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fittingwith applications to image analysis and automated cartography. Communications of the ACM 24, 6, 381–395 (1981)CrossRefGoogle Scholar
  27. 27.
    Swain, M.J., Ballard, D.H.: Color indexing. International Journal of Computer Vision (IJCV) 7(1) (1991)Google Scholar
  28. 28.
    Recognition without Correspondence using Multidimensional Receptive Field Histograms. International Journal of Computer Vision (IJCV) 36(1), 31–50 (2000)Google Scholar
  29. 29.
    Belongie, S., Malik, J., Puzicha, J.: Matching Shapes. In: Proceedings of International Conference on Computer Vision (ICCV 2001) (2001)Google Scholar
  30. 30.
    Kjeldsen, R., Pinhanez, C., Pingali, G., Hartman, J., Levas, T., Podlaseck, M.: Interacting with Steerable Projected Displays. In: Proc. of the 5th International Conference on Automatic Face and Gesture Recognition (FG 2002), May 20-21 2002, Washington (DC) (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • David Molyneaux
    • 1
  • Hans Gellersen
    • 1
  • Gerd Kortuem
    • 1
  • Bernt Schiele
    • 2
  1. 1.Computing Department, Lancaster UniversityEngland
  2. 2.Computer Science Department, Darmstadt University of Technology 

Personalised recommendations