Vision-Based Detection of Mobile Smart Objects

  • David Molyneaux
  • Hans Gellersen
  • Bernt Schiele
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5279)


We evaluate an approach for mobile smart objects to cooperate with projector-camera systems to achieve interactive projected displays on their surfaces without changing their appearance or function. Smart objects describe their appearance directly to the projector-camera system, enabling vision-based detection based on their natural appearance. This detection is a significant challenge, as objects differ in appearance and appear at varying distances and orientations with respect to a tracking camera. We investigate four detection approaches representing different appearance cues and contribute three experimental studies analysing the impact on detection performance, firstly of scale and rotation, secondly the combination of multiple appearance cues and thirdly the use of context information from the smart object. We find that the training of appearance descriptions must coincide with the scale and orientations providing the best detection performance, that multiple cues provide a clear performance gain over a single cue and that context sensing masks distractions and clutter, further improving detection performance.


Cooperative Augmentation Smart Objects Vision-Based Detection Natural Appearance Multi-Cue Detection 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Strohbach, M., Gellersen, H.-W., Kortuem, G., Kray, C.: Cooperative Artefacts: Assessing Real World Situations with Embedded Technology. In: Davies, N., Mynatt, E.D., Siio, I. (eds.) UbiComp 2004. LNCS, vol. 3205, pp. 250–267. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  2. 2.
    Molyneaux, D., Gellersen, H., Kortuem, G., Schiele, B.: Cooperative Augmentation of Smart Objects with Projector-Camera Systems. In: Krumm, J., Abowd, G.D., Seneviratne, A., Strang, T. (eds.) UbiComp 2007. LNCS, vol. 4717, pp. 501–518. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  3. 3.
    Ehnes, J., Hirota, K., Hirose, M.: Projected Augmentation - Augmented Reality using Rotatable Video Projectors. In: Proceedings of the Third IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2004), Arlington, VA, USA, September-October (2004)Google Scholar
  4. 4.
    Bandyopadhyay, D., Raskar, R., Fuchs, H.: Dynamic Shader Lamps: Painting on Movable Objects. In: Proc. of the IEEE and ACM International Symposium on Augmented Reality (ISAR 2001), New York (2001)Google Scholar
  5. 5.
    Borkowski, S., Riff, O., Crowley, J.L.: Projecting rectified images in an augmented environment. In: IEEE International Workshop on Projector-Camera Systems (PROCAMS 2003), Nice, France, October 12 (2003)Google Scholar
  6. 6.
    Pinhanez, C.S.: The Everywhere Displays Projector: A Device to Create Ubiquitous Graphical Interfaces. In: Abowd, G.D., Brumitt, B., Shafer, S. (eds.) UbiComp 2001. LNCS, vol. 2201. Springer, Heidelberg (2001)Google Scholar
  7. 7.
    Swain, M.J., Ballard, D.H.: Color indexing. International Journal of Computer Vision 7(1), 11–32 (1991)CrossRefGoogle Scholar
  8. 8.
    Jurie, F., Dhome, M.: Hyperplane Approximation for Template Matching. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(7), 996–1000 (2002)CrossRefGoogle Scholar
  9. 9.
    Schiele, B., Crowley, J.L.: Recognition without Correspondence using Multidimensional Receptive Field Histograms. International Journal of Computer Vision (IJCV) 36(1), 31–50 (2000)CrossRefGoogle Scholar
  10. 10.
    Murase, H., Nayar, S.K.: Visual Learning and Recognition of 3D Objects from Appearance. International Journal on Computer Vision 14(1), 5–24 (1995)CrossRefGoogle Scholar
  11. 11.
    Turk, M., Pentland, A.: Eigenfaces for Recognition. Journal of Cognitive Neuroscience 3(1), 71–86 (1991)CrossRefGoogle Scholar
  12. 12.
    Belongie, S., Malik, J., Puzicha, J.: Shape Matching and Object Recognition Using Shape Contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(4), 509–522 (2002)CrossRefGoogle Scholar
  13. 13.
    Mokhtarian, F.: Silhoutte based Isolated Object Recognition through Curvature Scale. IEEE Transactions on Pattern Analysis and Machine Intelligence 17(5), 539–544 (1995)CrossRefGoogle Scholar
  14. 14.
    Lowe, D.G.: Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  15. 15.
    Mikolajczyk, K., Schmid, C.: A Performance Evaluation of Local Descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(10), 1615–1630 (2005)CrossRefGoogle Scholar
  16. 16.
    Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., Gool, L.V.: A Comparison of Affine Region Detectors. International Journal on Computer Vision 65(1-2), 43–72 (2005)CrossRefGoogle Scholar
  17. 17.
    Spengler, M., Schiele, B.: Towards Robust Multi-cue Integration for Visual Tracking. In: Proceedings of the Second International Workshop on Computer Vision Systems. Springer, Heidelberg (2001)Google Scholar
  18. 18.
    Li, P., Chaumette, F.: Image Cues Fusion for Object Tracking Based on Particle Filter. In: Perales, F.J., Draper, B.A. (eds.) AMDO 2004. LNCS, vol. 3179, pp. 99–107. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  19. 19.
    Brasnett, P., Mihaylova, L., Canagarajah, N., Bull, D.: Particle Filtering with Multiple Cues for Object Tracking in Video Sequences. In: SPIE’s 17th Annual Symposium on Electronic Imaging, Science and Technology, San Jose California, USA, pp. 430–441 (2005)Google Scholar
  20. 20.
    Giebel, J., Gavrila, D.M., Schnörr, C.: A Bayesian Framework for Multi-cue 3D Object Tracking. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 241–252. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  21. 21.
    Lindeberg, T.: Scale-Space for Discrete Signals. IEEE Transactions on Pattern Analysis and Machine Intelligence 12(3), 234–254 (1990)CrossRefGoogle Scholar
  22. 22.
    Harris, C., Stephens, M.: A combined corner and edge detector. In: Proceedings of the 4th Alvey Vision Conference, pp. 147–151 (1988)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • David Molyneaux
    • 1
  • Hans Gellersen
    • 1
  • Bernt Schiele
    • 2
  1. 1.Computing DepartmentLancaster UniversityEngland
  2. 2.Computer Science DepartmentDarmstadt University of TechnologyGermany

Personalised recommendations