Multimedia Systems

, Volume 16, Issue 2, pp 105–125 | Cite as

Relevance ranking in georeferenced video search

  • Sakire Arslan Ay
  • Roger Zimmermann
  • Seon Ho Kim
Original Research Paper


The rapid adoption and deployment of ubiquitous video cameras has led to the collection of voluminous amounts of media data. However, indexing and searching of large video databases remain a very challenging task. Recently, some recorded video data are automatically annotated with meta-data collected from various sensors such as Global Positioning System (GPS) and compass devices. In our earlier work, we proposed the notion of a viewable scene model derived from the fusion of location and direction sensor information with a video stream. Such georeferenced media streams are useful in many applications and, very importantly, they can effectively be searched via their meta-data on a large scale. Consequently, search by geo-properties complements traditional content-based retrieval methods. The result of a georeferenced video query will in general consist of a number of video segments that satisfy the query conditions, but with more or less relevance. For example, a building of interest may appear in a video segment, but may only be visible in a corner. Therefore, an essential and integral part of a video query is the ranking of the result set according to the relevance of each clip. An effective result ranking is even more important for video than it is for text search, since the browsing of results can only be achieved by viewing each clip, which is very time consuming. In this study, we investigate and present three ranking algorithms that use spatial and temporal properties of georeferenced videos to effectively rank search results. To allow our techniques to scale to large video databases, we further introduce a histogram-based approach that allows fast online computations. An experimental evaluation demonstrates the utility of the proposed methods.


Global Position System Range Query Mean Average Precision Video Segment Rank List 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Zhang, H.J.: Multimedia content analysis and search: new perspectives and approaches. In: Proceedings of the ACM International Conference on Multimedia, pp. 1–2 (2009)Google Scholar
  2. 2.
    Ay, S.A., Zimmermann, R., Kim, S.H.: Viewable scene modeling for geospatial video search. In: Proceedings of the ACM International Conference on Multimedia, pp 309–318 (2008)Google Scholar
  3. 3.
    Rodden, K., Wood, K.R.: How do people manage their digital photographs? In: SIGCHI Conference on Human Factors in Computing Systems, pp. 409–416 (2003)Google Scholar
  4. 4.
    Toyama, K., Logan, R., Roseway, A.: Geographic location tags on digital images. In: Proceedings of the ACM International Conference on Multimedia, pp. 156–166 (2003)Google Scholar
  5. 5.
    Naaman, M., Song, Y.J., Paepcke, A., Garcia-Molina, H.: Automatic organization for digital photographs with geographic coordinates. In: 4th ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 53–62 (2004)Google Scholar
  6. 6.
    Pigeau, A., Gelgon, M.: Building and tracking hierarchical geographical & temporal partitions for image collection management on mobile devices. In: Proceedings of the ACM International Conference on Multimedia (2005)Google Scholar
  7. 7.
  8. 8.
  9. 9.
  10. 10.
    Epshtein, B., Ofek, E., Wexler, Y., Zhang, P.: Hierarchical photo organization using geo-relevance. In: 15th ACM International Symposium on Advances in Geographic Information Systems (GIS), pp 1–7 (2007)Google Scholar
  11. 11.
    Simon, I., Seitz, S.M.: Scene segmentation using the wisdom of crowds. In: ECCV ’08: Proceedings of the 10th European Conference on Computer Vision, pp. 541–553 (2008)Google Scholar
  12. 12.
    Kadobayashi, R., Tanaka, K.: 3D viewpoint-based photo search and information browsing. In: 28th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 621–622 (2005)Google Scholar
  13. 13.
    Gautam, S., Sarkis, G., Tjandranegara, E., Zelkowitz, E., Lu, Y.-H., Delp, E.J.: Multimedia for mobile environment: image enhanced navigation. In: Multimedia Content Analysis, Management, and Retrieval (2006)Google Scholar
  14. 14.
    Torniai, C., Battle, S., Cayzer, S.: Sharing, discovering and browsing geotagged pictures on the web. Technical report, Digital Media Systems Laboratory, HP Laboratories Bristol (2007)Google Scholar
  15. 15.
    Kennedy, L.S., Naaman, M.: Generating diverse and representative image search results for landmarks. In: WWW ’08: Proceeding of the 17th International Conference on the World Wide Web, pp 297–306 (2008)Google Scholar
  16. 16.
    Hwang, T.-H., Choi, K.-H., Joo, I.-H., Lee, J.-H.: MPEG-7 metadata for video-based GIS applications. In: Geoscience and Remote Sensing Symposium, vol. 6, pp. 3641–3643 (2003)Google Scholar
  17. 17.
    Kim, K.-H., Kim, S.-S., Lee, S.-H., Park, J.-H., Lee, J.-H.: The interactive geographic video. In: Geoscience and Remote Sensing Symposium, vol. 1, pp. 59–61 (2003)Google Scholar
  18. 18.
    Liu, X., Corner, M., Shenoy, P.: SEVA: sensor-enhanced video annotation. In: Proceedings of the ACM International Conference on Multimedia, pp. 618–627 (2005)Google Scholar
  19. 19.
    Arslan Ay, S., Kim, S.H., Zimmermann, R.: Relevance ranking in georeferenced video search. Technical Report 09-911, University of Southern California, Computer Science Department (2009)Google Scholar
  20. 20.
    Smeaton, A.F., Over, P., Kraaij, W.: Evaluation campaigns and TRECVid. In: MIR ’06: Proceedings of the 8th ACM International Workshop on Multimedia Information Retrieval, pp. 321–330 (2006)Google Scholar
  21. 21.
    Smeaton, A.F., Over, P., Kraaij, W.: High-level feature detection from video in TRECVid: a 5-year retrospective of achievements. In: Multimedia Content Analysis, Theory and Applications, pp. 151–174 (2009)Google Scholar
  22. 22.
    Beard, K., Sharma, V.: Multidimensional ranking for data in digital spatial libraries. Int. J. Digit. Libr. 1(2), pp. 153–160 (1997)CrossRefGoogle Scholar
  23. 23.
    Larson, R.R., Frontiera, P.: Geographic information retrieval (GIR) ranking methods for digital libraries. In: JCDL ’04: Proceedings of the 4th ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 415–415 (2004)Google Scholar
  24. 24.
    Gobel, S., Klein, P.: Ranking mechanisms in meta-data information systems for geo-spatial data. In: Earth Observation and Geo-Spatial Data (EOGEO) Technical Workshop (2002)Google Scholar
  25. 25.
    Ioannidis, Y.: The history of histograms (abridged). In: Proceedings of VLDB Conference (2003)Google Scholar
  26. 26.
    Graham, C.H., Bartlett, N.R., Brown, J.L., Hsia, Y., Mueller, C.C., Riggs, L.A.: Vision and Visual Perception. Wiley, New York (1965)Google Scholar
  27. 27.
    Hecht, E.: Optics, 4th edn. Addison-Wesley, Reading (2001)Google Scholar
  28. 28.
    Camera Calibration Toolbox for Matlab.
  29. 29.
    Lew, M.S., Sebe, N., Djeraba, C., Jain, R.: Content-based multimedia information retrieval: state of the art and challenges. ACM Trans Multimedia Comput Commun Appl TOMCCAP 2(1), 1–19 (2006)CrossRefGoogle Scholar
  30. 30.
    Datta, R., Joshi, D., Li, J., Wang, J.Z.: Image retrieval: ideas, influences, and trends of the new age. ACM Comput. Surv. 40(2), pp. 1–60 (2008)CrossRefGoogle Scholar
  31. 31.
    Snoek, C.G.M., Worring, M.: Concept-based video retrieval. Found Trends Info Retr 2(4), 215–322 (2009)CrossRefGoogle Scholar
  32. 32.
    Yanagawa, A., Chang, S.-F., Kennedy, L., Hsu, W.: Distributed policy management and comprehension with classified advertisements. Technical Report 222-2006-8, Columbia University ADVENT (2007)Google Scholar
  33. 33.
    Järvelin, K., Kekäläinen, J.: Cumulated gain-based evaluation of IR techniques. ACM Trans Info Syst 20(4), 422–446 (2002)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2010

Authors and Affiliations

  • Sakire Arslan Ay
    • 1
  • Roger Zimmermann
    • 2
  • Seon Ho Kim
    • 3
  1. 1.Department of Computer ScienceUniversity of Southern CaliforniaLos AngelesUSA
  2. 2.School of ComputingNational University of SingaporeSingaporeRepublic of Singapore
  3. 3.Department of Computer Science and Information TechnologyUniversity of the District of ColumbiaWashingtonUSA

Personalised recommendations