Mobile Museum Guide Based on Fast SIFT Recognition

  • Boris Ruf
  • Effrosyni Kokiopoulou
  • Marcin Detyniecki
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5811)


This article explores the feasibility of a market-ready, mobile pattern recognition system based on the latest findings in the field of object recognition and currently available hardware and network technology. More precisely, an innovative, mobile museum guide system is presented, which enables camera phones to recognize paintings in art galleries.

After careful examination, the algorithms Scale-Invariant Feature Transform (SIFT) and Speeded Up Robust Features (SURF) were found most promising for this goal. Consequently, both have been integrated in a fully implemented prototype system and their performance has been thoroughly evaluated under realistic conditions.

In order to speed up the matching process for finding the corresponding sample in the feature database, an approximation to Nearest Neighbor Search was investigated. The k-means based clustering approach was found to significantly improve the computational time.


Object Recognition Mobile Client Camera Phone Interest Point Detector Fisher Linear Discriminant 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Group, G.: 2006 Press Releases. November 2 (2006),
  2. 2.
    Rohs, M., Gfeller, B.: Using camera-equipped mobile phones for interacting with real-world objects. In: Advances in Pervasive Computing, pp. 265–271. Austrian Computer Society (OCG), Austria (2004)Google Scholar
  3. 3.
    Yeh, T., Grauman, K., Tollmar, K., Darrell, T.: A picture is worth a thousand keywords: image-based object search on a mobile platform. In: CHI 2005: CHI 2005 extended abstracts on Human factors in computing systems, pp. 2025–2028. ACM, New York (2005)CrossRefGoogle Scholar
  4. 4.
    Robertsone, D., Cipolla, R.: An image-based system for urban navigation. In: The 15th British Machine Vision Conference (BMVC 2004), pp. 819–828 (2004)Google Scholar
  5. 5.
    Swain, M.J., Ballard, D.H.: Color indexing, vol. 7(1), pp. 11–32. Kluwer Academic Publishers, Hingham (1991)Google Scholar
  6. 6.
    Chellappa, R., Wilson, C.L., Sirohey, S.: Human and machine recognition of faces: A survey. Proceedings of the IEEE 83(5), 705–740 (1995)CrossRefGoogle Scholar
  7. 7.
    Samal, A., Iyengar, P.A.: Automatic recognition and analysis of human faces and facial expressions: a survey. Pattern Recogn. 25(1), 65–77 (1992)CrossRefGoogle Scholar
  8. 8.
    Turk, M., Pentland, A.: Eigenfaces for recognition. CogNeuro. 3(1), 71–96 (1991)Google Scholar
  9. 9.
    Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenfaces vs. Fisherfaces: recognition using class specific linear projection. Transactions on Pattern Analysis and Machine Intelligence 19(7), 711–720 (1997)CrossRefGoogle Scholar
  10. 10.
    Harris, C., Stephens, M.: A combined corner and edge detection. In: Proceedings of The Fourth Alvey Vision Conference, pp. 147–151 (1988),
  11. 11.
    Schmid, C., Mohr, R.: Local grayvalue invariants for image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 19(5), 530–535 (1997)CrossRefGoogle Scholar
  12. 12.
    Gool, L.J.V., Moons, T., Ungureanu, D.: Affine/ photometric invariants for planar intensity patterns. In: Buxton, B.F., Cipolla, R. (eds.) ECCV 1996. LNCS, vol. 1064, pp. 642–651. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  13. 13.
    Lowe, D.: Object recognition from local scale-invariant features. In: The Proceedings of the Seventh IEEE International Conference on Computer Vision, 1999, vol. 2, pp. 1150–1157 (1999)Google Scholar
  14. 14.
    Siggelkow, S.: Feature histograms for content-based image retrieval. Ph.D. dissertation, University of Freiburg, Institute for Computer Science (2002)Google Scholar
  15. 15.
    Schaffalitzky, F., Zisserman, A.: Multi-view matching for unordered image sets, or ’How do i organize my holiday snaps? In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 414–431. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  16. 16.
    Mikolajczyk, K., Schmid, C.: An affine invariant interest point detector. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 128–142. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  17. 17.
    Quack, T., Monich, U., Thiele, L., Manjunath, B.: Cortina: A system for large-scale, content-based web image retrieval. In: ACM Multimedia 2004 (October 2004)Google Scholar
  18. 18.
    Burgard, W., Cremers, A., Fox, D., Hähnel, D., Lakemeyer, G., Schulz, D., Steiner, W., Thrun, S.: The interactive museum tour-guide robot. In: Proc. of the Fifteenth National Conference on Artificial Intelligence, AAAI 1998 (1998)Google Scholar
  19. 19.
    Thrun, S., Beetz, M., Bennewitz, M., Burgard, W., Cremers, A., Dellaert, F., Fox, D., Hähnel, D., Rosenberg, C., Roy, N., Schulte, J., Schulz, D.: Probabilistic algorithms and the interactive museum tour-guide robot Minerva. International Journal of Robotics Research 19(11), 972–999 (2000)CrossRefGoogle Scholar
  20. 20.
    Albertini, A., Brunelli, R., Stock, O., Zancanaro, M.: Communicating user’s focus of attention by image processing as input for a mobile museum guide. In: IUI 2005: Proceedings of the 10th international conference on Intelligent user interfaces, pp. 299–301. ACM, New York (2005)CrossRefGoogle Scholar
  21. 21.
    Bay, H., Fasel, B., Gool, L.V.: Interactive museum guide: Fast and robust recognition of museum objects. In: Proceedings of the first international workshop on mobile vision (2006)Google Scholar
  22. 22.
    Bruns, E., Brombach, B., Zeidler, T., Bimber, O.: Enabling mobile phones to support large-scale museum guidance. IEEE MultiMedia 14(2), 16–25 (2007)CrossRefGoogle Scholar
  23. 23.
    Lim, J.-H., Li, Y., You, Y., Chevallet, J.-P.: Scene recognition with camera phones for tourist information access. In: IEEE International Conference on Multimedia and Expo., 2007, July 2-5, pp. 100–103 (2007)Google Scholar
  24. 24.
    Bay, H., Tuytelaars, T., Van Gool, L.: SURF: Speeded Up Robust Features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  25. 25.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: CVPR, vol. 1, pp. I–511 – I–518 (2001)Google Scholar
  26. 26.
    Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(10), 1615–1630 (2005)CrossRefGoogle Scholar
  27. 27.
    Baumberg, A.: Reliable feature matching across widely separated views. In: Proceedings of Computer Vision and Pattern Recognition, 2000, vol. 1, pp. 774–781 (2000)Google Scholar
  28. 28.
    Beis, J., Lowe, D.: Shape indexing using approximate nearest-neighbour search in high-dimensional spaces. In: 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1997. Proceedings, June 17-19, pp. 1000–1006 (1997)Google Scholar
  29. 29.
    Jost, P., Vandergheynst, P., Frossard, P.: Tree-Based Pursuit: Algorithm and Properties. IEEE Transactions on Signal Processing 54(12), 4685–4697 (2006)CrossRefGoogle Scholar
  30. 30.
    Macqueen, J.B.: Some methods of classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, pp. 281–297 (1967)Google Scholar
  31. 31.
    Kren, E., Marx, D.: Web Gallery of Art,

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Boris Ruf
    • 1
  • Effrosyni Kokiopoulou
    • 1
  • Marcin Detyniecki
    • 2
  1. 1.Ecole Polytechnique Fédérale de Lausanne (EPFL) 
  2. 2.Laboratoire d’Informatique de Paris 6 (LIP6) 

Personalised recommendations