Advertisement

Multimedia Tools and Applications

, Volume 51, Issue 1, pp 5–33 | Cite as

Semantic analysis and retrieval in personal and social photo collections

  • Philipp Sandhaus
  • Susanne Boll
Article

Abstract

Semantic understanding of images has been an important topic in the research community for a long time as it is an important prerequisite to build meaningful retrieval systems which are accessible by both users and automatic reasoning algorithms. Recently, especially with the growing trend to share photos online, the social aspect of image retrieval becomes more and more prevalent and image retrieval more and more focusses specifically on photos and their special characteristics, especially on information outside the image itself. Researchers are starting to explore how and why photos are shot, shared and used and try to incorporate this additional knowledge to aid image analysis and retrieval. Several survey papers have been written in the past reviewing works in the general field of image analysis and retrieval. However, the social aspect of image retrieval and the focus on digital photos has not sufficiently been addressed in these works. In this article we give an overview over the current research field of semantic photo understanding, annotation and retrieval. We review over 160 contributions in the field and identify trending topics and implications for future directions of research.

Keywords

Personal photos Social photo collections Semantic image retrieval Survey 

References

  1. 1.
    3rd special workshop on multimedia semantics (2005). Pisa, ItalyGoogle Scholar
  2. 2.
    ACM (2010) Multimedia Grand Challenge. http://comminfo.rutgers.edu/conferences/mmchallenge/
  3. 3.
    Adobe (2005) Xmp specification. Tech. rep., Adobe Systems Inc. http://www.adobe.com/products/xmp/
  4. 4.
    Aigrain P, Zhang H, Petkovic D (1996) Content-based representation and retrieval of visual media: a state-of-the-art review. In: Representation and retrieval of visual media in multimedia systems, pp 3–26Google Scholar
  5. 5.
    Ames M, Eckles D, Naaman M, Spasojevic M, Van House N (2009) Requirements for mobile photoware. Personal and ubiquitous computing. doi: 10.1007/s00779-009-0237-4
  6. 6.
    Anguera X, Xu J, Oliver N (2008) Multimodal photo annotation and retrieval on a mobile phone. In: MIR ’08: proceeding of the 1st ACM international conference on multimedia information retrieval. ACM, New York, NY, USA, pp 188–194. doi:10.1145/1460096.1460127 CrossRefGoogle Scholar
  7. 7.
    Ardizzone E, La Cascia M, Vella F (2008) Mean shift clustering for personal photo album organization. In: Proc. of ICIP 2008. 15th IEEE international conference on image processing, pp 85–88. doi: 10.1109/ICIP.2008.4711697
  8. 8.
    Arni T, Tang J, Sanderson M, Clough P (2008) Creating a test collection to evaluate diversity in image retrieval. Beyond binary relevance: preferences, diversity and set-level judgmentsGoogle Scholar
  9. 9.
    Aslandogan Y, Yu C (2000) Diogenes: a web search agent for content based indexing of personal images. In: Proceedings of ACM SIGIR, pp 481–482Google Scholar
  10. 10.
    Avcıbaş İ, Sankur B, Sayood K (2002) Statistical evaluation of image quality measures. J Electron Imaging 11:206CrossRefGoogle Scholar
  11. 11.
    Banerjee S, Evans B (2004) Unsupervised automation of photographic composition rules in digital still cameras. In: Proceedings of SPIE, vol 5301, pp 364–373Google Scholar
  12. 12.
    Becker H, Naaman M, Gravano L (2010) Learning similarity metrics for event identification in social media. In: WSDM ’10: proceedings of the third ACM international conference on web search and data mining. ACM, New York, NY, USA, pp 291–300. doi:10.1145/1718487.1718524 CrossRefGoogle Scholar
  13. 13.
    Bentley F, Metcalf C, Harboe G (2006) Personal vs. commercial content: the similarities between consumer use of photos and music. In: CHI ’06: proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, NY, USA, pp 667–676. doi:10.1145/1124772.1124871 CrossRefGoogle Scholar
  14. 14.
    Berners-Lee T, Hendler J, Lassila O et al (2001) The semantic web. Sci Am 284(5):28–37CrossRefGoogle Scholar
  15. 15.
    Bloehdorn S, Petridis K, Saathoff C, Simou N, Tzouvaras V, Avrithis Y, Handschuh S, Kompatsiaris Y, Staab S, Strintzis M (2005) Semantic annotation of images and videos for multimedia analysis. In: The semantic web: research and applications, pp 592–607Google Scholar
  16. 16.
    Boutell M, Luo J (2004) Bayesian fusion of camera metadata cues in semantic scene classification. In: CVPRGoogle Scholar
  17. 17.
    Boutell M, Luo J (2004) Photo classification by integrating image content and camera metadata. In: ICPR ’04: proceedings of the pattern recognition, 17th international conference on (ICPR’04), vol 4. IEEE Computer Society, Washington, DC, USA, pp 901–904. doi: 10.1109/ICPR.2004.693 CrossRefGoogle Scholar
  18. 18.
    Boutell MR, Luo J (2005) Beyond pixels: exploiting camera metadata for photo classification. Pattern Recogn 38(6):935–946CrossRefGoogle Scholar
  19. 19.
    Boutell M, Brown C, Luo J (2006) Exploiting context for semantic scene classification. Univ. Rochester, Rochester, NY, Tech. Rep 894Google Scholar
  20. 20.
    Bovik A, Sheikh H (2005) No-reference quality assessment using natural scene statistics: JPEG2000. IEEE Trans Image Process 14(2005):1918–1927Google Scholar
  21. 21.
    Boyd D, Ellison NB (2007) Social network sites: definition, history, and scholarship. J Comput-Mediat Commun 13(1):210–230CrossRefGoogle Scholar
  22. 22.
    Bryant R (2007) Data-intensive supercomputing: the case for DISC. School of Computer Science, Carnegie Mellon University, Tech. Rep. Technical Report CMU-CS-07-128Google Scholar
  23. 23.
    Burnett IS, Pereira F, Walle RVd, Koenen R (2006) The MPEG-21 book. Wiley, New YorkCrossRefGoogle Scholar
  24. 24.
    Cao L, Luo J, Huang TS (2008) Annotating photo collections by label propagation according to multiple similarity cues. In: MM ’08: proceeding of the 16th ACM international conference on multimedia. ACM, New York, NY, USA, pp 121–130. doi: 10.1145/1459359.1459376 CrossRefGoogle Scholar
  25. 25.
    Chalfen R (1987) Snapshot versions of life. Bowling Green University Popular PressGoogle Scholar
  26. 26.
    Chandramouli K, Izquierdo E (2010) Semantic structuring and retrieval of event chapters in social photo collections. In: MIR ’10: proceedings of the international conference on multimedia information retrieval. ACM, New York, NY, USA, pp 507–516. doi:10.1145/1743384.1743472 CrossRefGoogle Scholar
  27. 27.
    Chang NS, Fu KS (1980) Query-by-pictorial-example. IEEE Trans Softw Eng 6(6):519–524. doi: 10.1109/TSE.1980.230801 CrossRefGoogle Scholar
  28. 28.
    Chen C, Oakes M, Tait J (2006) Browsing personal images using episodic memory (time + location). Adv in IR 3936:362–372. doi: 10.1007/11735106_32 Google Scholar
  29. 29.
    Chianese A, Picariello A, Sansone L, Sapino ML (2004) Managing uncertainties in image databases: a fuzzy approach. Multimed Tools Appl 23(3):237–252. doi: 10.1023/B:MTAP.0000031759.22145.5d CrossRefGoogle Scholar
  30. 30.
    Chu C, Kim S, Lin Y, Yu Y, Bradski G, Ng A, Olukotun K (2007) Map-reduce for machine learning on multicore. In: Advances in neural information processing systems 19: proceedings of the 2006 conference. MIT Press, Cambridge, p 281Google Scholar
  31. 31.
    Comité International des Télécommunications de Presse (1999) IPTC—NAA information interchange model version 4. Tech. rep. http://www.iptc.org/IIM/
  32. 32.
    Cooper M, Foote J, Girgensohn A, Wilcox L (2005) Temporal event clustering for digital photo collections. ACM Trans Multimed Comput Commun Appl 1(3):269–288. doi:10.1145/1083314.1083317 CrossRefGoogle Scholar
  33. 33.
    Crabtree A, Rodden T, Mariani J (2004) Collaborating around collections: informing the continued development of photoware. In: CSCW ’04: proceedings of the 2004 ACM conference on computer supported cooperative work. ACM, New York, NY, USA, pp 396–405. doi:10.1145/1031607.1031673 CrossRefGoogle Scholar
  34. 34.
    Cusano C, Ciocca G, Schettini R (2003) Image annotation using SVM. In: Santini S, Schettini R (eds) Society of photo-optical instrumentation engineers (SPIE) conference series, pp 330–338Google Scholar
  35. 35.
    Damera-Venkata N, Kite TD, Geisler WS, Evans BL, Bovik AC (2004) Image quality assessment based on a degradation model. IEEE Trans Image Process 9(4):636–650CrossRefGoogle Scholar
  36. 36.
    Datta R, Joshi D, Li J, Wang JZ (2006) Studying aesthetics in photographic images using a computational approach. In: Leonardis A, Bischof H, Pinz A (eds) ECCV (3). Lecture notes in computer science, vol 3953. Springer, pp 288–301. http://infolab.stanford.edu/~wangz/project/imsearch/Aesthetics/ECCV06/
  37. 37.
    Datta R, Ge W, Liu J, Wang JZ (2007) Image retrieval: ideas, influences, and trends of the new age. In: ACM computing surveys, 2007. http://wang.ist.psu.edu/survey/analysis/
  38. 38.
    Datta R, Li J, Wang JZ (2007) Learning the consensus on visual quality for next-generation image management. In: Proceedings of the ACM multimedia conference. http://infolab.stanford.edu/~wangz/project/imsearch/ALIP/ACMM M07A/
  39. 39.
    Datta R, Li J, Wang J (2008) Algorithmic inferencing of aesthetics and emotion in natural images: an exposition. In: 15th IEEE international conference on image processing, pp 105–108Google Scholar
  40. 40.
    Dean J, Ghemawat S (2008) Mapreduce: simplified data processing on large clusters. Commun ACM 51(1):107–113. doi:10.1145/1327452.1327492 CrossRefGoogle Scholar
  41. 41.
    Digital Imaging Group (2001) DIG35 specification—metadata for digital images—version 1.1. Tech. rep. http://www.digitalimaging.org
  42. 42.
    Duygulu P, Barnard K, De Freitas J, Forsyth D (2006) Object recognition as machine translation: learning a lexicon for a fixed image vocabulary. In: Computer vision—ECCV 2002, pp 349–354Google Scholar
  43. 43.
    Eisenthal Y, Dror G, Ruppin E (2006) Facial attractiveness: beauty and the machine. Neural Comput 18(1):119–142. doi: 10.1162/089976606774841602 CrossRefGoogle Scholar
  44. 44.
    Eskicioglu A (2000) Quality measurement for monochrome compressed images in the past 25 years. In: IEEE international conference on acoustics speech and signal processing, vol 4Google Scholar
  45. 45.
    Eskicioglu A, Fisher P (1995) Image quality measures and their performance. IEEE Trans Commun 43(12):2959–2965. doi: 10.1109/26.477498 CrossRefGoogle Scholar
  46. 46.
    Fan X, Xie X, Li Z, Li M, Ma WY (2005) Photo-to-search: using multimodal queries to search the web from mobile devices. In: MIR ’05: proceedings of the 7th ACM SIGMM international workshop on multimedia information retrieval. ACM, New York, NY, USA, pp 143–150. doi:10.1145/1101826.1101851 CrossRefGoogle Scholar
  47. 47.
    Flickner M, Sawhney H, Niblack W, Ashley J, Huang Q, Dom B, Gorkani M, Hafner J, Lee D, Petkovic D, Steele D, Yanker P (1995) Query by image and video content: the qbic system. Computer 28(9):23–32. doi: 10.1109/2.410146 CrossRefGoogle Scholar
  48. 48.
    Frankel C, Swain MJ, Athitsos V (1996) WebSeer: an image search engine for the world wide web. Tech. rep., University of Chicago, Chicago, IL, USAGoogle Scholar
  49. 49.
    Frohlich D, Kuchinsky A, Pering C, Don A, Ariss S (2002) Requirements for photoware. In: CSCW ’02: proceedings of the 2002 ACM conference on computer supported cooperative work. ACM, New York, NY, USA, pp 166–175. doi: 10.1145/587078.587102. http://portal.acm.org/citation.cfm?id=587102 CrossRefGoogle Scholar
  50. 50.
    Frohlich D, Wall S, Kiddle G (2008) Collaborative photowork: challenging the boundaries between photowork and phototalk. In: Proc. of CHI workshop on collocated social practices surrounding photosGoogle Scholar
  51. 51.
    Gargi U, Deng Y, Tretter DR (2002) Managing and searching personal photo collections. Tech. rep., HP Laboratories, Palo AltoGoogle Scholar
  52. 52.
    Girgensohn A, Adcock J, Cooper MD, Foote J, Wilcox L (2003) Simplifying the management of large photo collections. In: INTERACTGoogle Scholar
  53. 53.
    Girgensohn A, Adcock J, Wilcox L (2004) Leveraging face recognition technology to find and organize photos. In: MIR ’04: proceedings of the 6th ACM SIGMM international workshop on multimedia information retrieval. ACM, New York, NY, USA, pp 99–106. doi:10.1145/1026711.1026728 CrossRefGoogle Scholar
  54. 54.
    Gong Z, U LH, Cheang CW (2005) Web image semantic clustering. In: Proceedings of the 4th international conference on ontologies, database and applications of semantics (ODBASE 2005). Lecture notes in computer science. Springer, Agia, Napa, CyprusGoogle Scholar
  55. 55.
    Graham A, Garcia-Molina H, Paepcke A, Winograd T (2002) Time as essence for photo browsing through personal digital libraries. In: JCDL ’02: proceedings of the 2nd ACM/IEEE-CS joint conference on digital libraries. ACM, New York, NY, USA, pp 326–335. doi:10.1145/544220.544301 CrossRefGoogle Scholar
  56. 56.
    Gurrin C, Jones GJF, Lee H, O’Hare N, Smeaton AF, Murphy N (2005) Mobile access to personal digital photograph archives. In: MobileHCI ’05: proceedings of the 7th international conference on human computer interaction with mobile devices & services. ACM, New York, NY, USA, pp 311–314. doi:10.1145/1085777.1085842 CrossRefGoogle Scholar
  57. 57.
    Halaschek-wiener C, Schain A, Golbeck J, Parsia B, Hendler J (2005) A flexible approach for managing digital images on the semantic web. In: th international workshop on knowledge markup and semantic annotationGoogle Scholar
  58. 58.
    Hardman L, Nack F, Obrenovic Z, Kerherve B, Piersol K (2005) Canonical processes of media production. In: ACM workshop on multimedia for human communication—from capture to convey. http://homepages.cwi.nl/~media/projects/canonical/papers/model.pdf
  59. 59.
    Hare J, Sinclair P, Lewis P, Martinez K, Enser P, Sandom C (2006) Bridging the semantic gap in multimedia information retrieval: top-down and bottom-up approaches. In: 3rd European semantic web conferenceGoogle Scholar
  60. 60.
    Hare JS, Lewis PH, Enser PGB, Sandom CJ (2007) Semantic facets: an in-depth analysis of a semantic image retrieval system. In: ACM CIVR 2007: the 6th international conference on image and video retrieval, ACM. http://eprints.ecs.soton.ac.uk/14322/
  61. 61.
    Hare JS, Samangooei S, Lewis PH, Nixon MS (2008) Semantic spaces revisited: investigating the performance of auto-annotation and semantic retrieval using semantic spaces. In: CIVR ’08: proceedings of the 2008 international conference on content-based image and video retrieval. ACM, New York, NY, USA, pp 359–368. doi:10.1145/1386352.1386399 CrossRefGoogle Scholar
  62. 62.
    He X, Caia D, Wen JR, Ma WY, Zhang HJ (2007) Clustering and searching www images using link and page layout analysis. ACM Trans Multimed Comput Commun Appl (TOMCAP) 3:10CrossRefGoogle Scholar
  63. 63.
    Hollink L, Schreiber A, Wielemaker J, Wielinga B (2003) Semantic annotation of image collections. In: Knowledge capture, pp 41–48Google Scholar
  64. 64.
    Hunter J (2001) Adding multimedia to the semantic web building an Mpeg-7 ontology. In: International semantic web working symposium (SWWS)Google Scholar
  65. 65.
    Inoue M (2009) Image retrieval: research and use in the information explosion. Prog Inf 6:3–14CrossRefGoogle Scholar
  66. 66.
    Jaffe A, Naaman M, Tassa T, Davis M (2006) Generating summaries for large collections of geo-referenced photographs, pp 853–854. doi:10.1145/1135777.1135911
  67. 67.
    Jeon J, Lavrenko V, Manmatha R (2003) Automatic image annotation and retrieval using cross-media relevance models. In: SIGIR ’03: proceedings of the 26th annual international ACM SIGIR conference on research and development in informaion retrieval. ACM, New York, NY, USA, pp 119–126. doi:10.1145/860435.860459 CrossRefGoogle Scholar
  68. 68.
    Jia J, Yu N, Hua XS (2008) Annotating personal albums via web mining. In: MM ’08: proceeding of the 16th ACM international conference on multimedia. ACM, New York, NY, USA, pp 459–468. doi:10.1145/1459359.1459421 CrossRefGoogle Scholar
  69. 69.
    Ke Y, Tang X, Jing F (2006) The design of high-level features for photo quality assessment. In: CVPR ’06: proceedings of the 2006 IEEE computer society conference on computer vision and pattern recognition. IEEE Computer Society, Washington, DC, USA, pp 419–426. doi: 10.1109/CVPR.2006.303 Google Scholar
  70. 70.
    Kennedy LS, Naaman M (2008) Generating diverse and representative image search results for landmarks. In: WWW ’08: proceeding of the 17th international conference on world wide web. ACM, New York, NY, USA, pp 297–306. doi:10.1145/1367497.1367539 CrossRefGoogle Scholar
  71. 71.
    Kennedy L, Naaman M, Ahern S, Nair R, Rattenbury T (2007) How flickr helps us make sense of the world: context and content in community-contributed media collections. In: MULTIMEDIA ’07: proceedings of the 15th international conference on multimedia. ACM, New York, NY, USA, pp 631–640. doi:10.1145/1291233.1291384 CrossRefGoogle Scholar
  72. 72.
    Kherfi ML, Ziou D, Bernardi A (2004) Image retrieval from the world wide web: issues, techniques, and systems. ACM Comput Surv 36(1):35–67. doi:10.1145/1013208.1013210 CrossRefGoogle Scholar
  73. 73.
    Kirk D, Sellen A, Rother C, Wood K (2006) Understanding photowork. In: CHI ’06: proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, NY, USA, pp761–770. doi:10.1145/1124772.1124885 CrossRefGoogle Scholar
  74. 74.
    Ku W, Kankanhalli M, Lim J (2007) Using camera settings templates to classify photos. In: International workshop on advanced image technologyGoogle Scholar
  75. 75.
    Lacerda Y, de Figueiredo H, de Souza Baptista C, Sampaio M (2008) Photogeo: a self-organizing system for personal photo collections, pp 258–265. doi: 10.1109/ISM.2008.81
  76. 76.
    Lafon Y, Bos B (2002) Describing and retrieving photos using rdf and http. http://www.w3.org/TR/photo-rdf/
  77. 77.
    Lavrenko V, Manmatha R, Jeon J (2004) A model for learning the semantics of pictures. In: Advances in neural information processing systems, vol 16Google Scholar
  78. 78.
    Lew MS (2000) Next-generation web searches for visual content. IEEE Comput 33(11):46–53. doi: 10.1109/2.881694 Google Scholar
  79. 79.
    Lew MS, Sebe N, Djeraba C, Jain R (2006) Content-based multimedia information retrieval: state of the art and challenges. ACM Trans Multimed Comput Commun Appl 2(1):1–19. doi: 10.1145/1126004.1126005.CrossRefGoogle Scholar
  80. 80.
    Li J, Wang JZ (2008) Real-time computerized annotation of pictures. IEEE Trans Pattern Anal Mach Intell 30(6):985–1002CrossRefGoogle Scholar
  81. 81.
    Li X et al (2002) Blind image quality assessment. In: Proc. IEEE int. conf. image proc, vol 1, pp 449–452Google Scholar
  82. 82.
    Lindley SE, Durrant AC, Kirk DS, Taylor AS (2008) Collocated social practices surrounding photos. In: CHI ’08: CHI ’08 extended abstracts on human factors in computing systems. ACM, New York, NY, USA, pp 3921–3924. doi:10.1145/1358628.1358957 Google Scholar
  83. 83.
    Liu Y, Zhang D, Lu G, Ma WY (2007) A survey of content-based image retrieval with high-level semantics. Pattern Recogn 40(1):262–282. doi: 10.1016/j.patcog.2006.04.045 zbMATHCrossRefGoogle Scholar
  84. 84.
    Loui AC (2000) Automatic image event segmentation and quality screening for albuming applications. In: ICME, pp 1125–1128Google Scholar
  85. 85.
    Loui AC, Savakis AE (2003) Automated event clustering and quality screening of consumer pictures for digital albuming. IEEE Trans Multimedia 5(3):390–402CrossRefGoogle Scholar
  86. 86.
    Lu G, Williams B (1999) An integrated WWW image retrieval system. In: Proceedings of the AusWeb99, Lismore, AustraliaGoogle Scholar
  87. 87.
    Luo J, Boutell M, Brown C (2006) Pictures are not taken in a vacuum. IEEE Signal Process Mag 23(2):101– 114CrossRefGoogle Scholar
  88. 88.
    Lux M (2009) Caliph & Emir: Mpeg-7 photo annotation and retrieval. In: MM ’09: proceedings of the seventeen ACM international conference on multimedia. ACM, New York, NY, USA, pp 925–926. doi:10.1145/1631272.1631456 CrossRefGoogle Scholar
  89. 89.
    Lux M, Chatzichristofis SA (2008) Lire: lucene image retrieval: an extensible java cbir library. In: MM ’08: proceeding of the 16th ACM international conference on multimedia. ACM, New York, NY, USA, pp 1085–1088. doi:10.1145/1459359.1459577 CrossRefGoogle Scholar
  90. 90.
    Maekawa T, Hara T, Nishio S (2006) Image classification for mobile web browsing. In: WWW ’06: proc.of the 15th intl. conf. on world wide web. ACM, New York, NY, USA, pp 43–52. doi:10.1145/1135777.1135789 CrossRefGoogle Scholar
  91. 91.
    Manjunath BS, Salembier P, Sikora T (2002) Introduction to MPEG-7: multimedia content description interface. Wiley, New YorkGoogle Scholar
  92. 92.
    Mansoor AB, Haider M, Mian AS, Khan SA (2009) A hybrid image quality measure for automatic image quality assessment. In: SCIA ’09: proceedings of the 16th scandinavian conference on image analysis. Springer, Berlin, Heidelberg, pp 91–98. doi: 10.1007/978-3-642-02230-2_10 Google Scholar
  93. 93.
    Marchand-Maillet S, Beretta G (2005) The benchathlon network. http://www.benchathlon.net
  94. 94.
    Marchand-Maillet S, Worring M (2006) Benchmarking image and video retrieval: an overview. In: MIR ’06: proceedings of the 8th ACM international workshop on multimedia information retrieval. ACM, New York, NY, USA, pp 297–300. doi:10.1145/1178677.1178718 CrossRefGoogle Scholar
  95. 95.
    Mei T, Wang B, Hua X, Zhou H, Li S (2006) Probabilistic multimodality fusion for event based home photo clustering. In: IEEE international conference on multimedia and expo, pp 1757–1760Google Scholar
  96. 96.
    Metadata Working Group (2009) Guidelines for handling image metadata—version 1.0.1. Tech. rep., Metadata Working GroupGoogle Scholar
  97. 97.
    Miller G (1995) WordNet: a lexical database for english. Commun ACM 38(11):41CrossRefGoogle Scholar
  98. 98.
    Miller AD, Edwards WK (2007) Give and take: a study of consumer photo-sharing culture and practice. In: CHI ’07: proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, NY, USA, pp 347–356. doi:10.1145/1240624.1240682 CrossRefGoogle Scholar
  99. 99.
    Monaghan F, O’Sullivan D (2006) Automating photo annotation using services and ontologies. In: Proc IEEE international conference on mobile data management, p 79. doi:10.1109/MDM.2006.39
  100. 100.
    Monaghan F, O’Sullivan D (2007) Leveraging ontologies, context and social networks to automate photo annotation. In: SAMT, pp 252–255Google Scholar
  101. 101.
    Mori Y, Takahashi H, Oka R (1999) Image-to-word transformation based on dividing and vector quantizing images with words. In: Proceedings of the first international workshop on multimedia intelligent storage and retrieval management (MISRM’99)Google Scholar
  102. 102.
    Mulhem P, Lim JH (2003) Home photo retrieval: time matters. In: Proc. of the second international conference on image and video retrieval (CIVR). LNCS. Springer, Urbana-Champaign, IL, USA, pp 321–330Google Scholar
  103. 103.
    Müller H, Marchand-Maillet S, Pun T (2002) The truth about corel—evaluation in image retrieval. In: CIVR ’02: proceedings of the international conference on image and video retrieval. Springer, London, UK, pp 38–49Google Scholar
  104. 104.
    Müller H, Müller W, Marchand-Maillet S, Pun T, Squire D (2003) A framework for benchmarking in CBIR. Multimed Tools Appl 21(1):55–73CrossRefGoogle Scholar
  105. 105.
    Müller H, Müller W, Squire DM, Marchand-Maillet S, Pun T (2001) Performance evaluation in content-based image retrieval: overview and proposals. Pattern Recogn Lett 22(5):593–601. doi: 10.1016/S0167-8655(00)00118-5 zbMATHCrossRefGoogle Scholar
  106. 106.
    Munson EV, Tsymbalenko Y (2001) To search for images on the web, look at the text, then look at the images. In: Proceedings of the first international workshop on web document analysis (WDA2001)Google Scholar
  107. 107.
    Naaman M, Harada S, Wang Q, Paepcke A (2004) Adventures in space and time: browsing personal collections of geo-referenced digital photographs. Technical Report 2004-26, Stanford InfoLab. http://ilpubs.stanford.edu:8090/647/
  108. 108.
    Naaman M, Song YJ, Paepcke A, Garcia-Molina H (2004) Automatic organization for digital photographs with geographic coordinates. In: JCDL ’04: proceedings of the 4th ACM/IEEE-CS joint conference on digital libraries. ACM, New York, NY, USA, pp 53–62. doi:10.1145/996350.996366 Google Scholar
  109. 109.
    Nack F, Hardman L (2002) Towards a syntax for multimedia semantics. Tech. rep., Centrum voor Wiskunde en Informatica (CWI), AmsterdamGoogle Scholar
  110. 110.
    Nair R, Reid N, Davis M (2005) Photo loi: browsing multi-user photo collections. In: MULTIMEDIA ’05: proceedings of the 13th annual ACM international conference on multimedia. ACM, New York, NY, USA, pp 223–224. doi:10.1145/1101149.1101187 CrossRefGoogle Scholar
  111. 111.
    Negoescu RA, Gatica-Perez D (2008) Analyzing flickr groups. In: CIVR ’08: proceedings of the 2008 international conference on content-based image and video retrieval. ACM, New York, NY, USA, pp 417–426. doi:10.1145/1386352.1386406 CrossRefGoogle Scholar
  112. 112.
    Nov O, Naaman M, Ye C (2009) Motivational, structural and tenure factors that impact online community photo sharing. In: Third international conference on weblogs and social media (ICWSM), San Jose, CaliforniaGoogle Scholar
  113. 113.
    Obrador P, Anguera X, de Oliveira R, Oliver N (2009) The role of tags and image aesthetics in social image search. In: WSM ’09: proceedings of the first SIGMM workshop on Social media. ACM, New York, NY, USA, pp 65–72. doi:10.1145/1631144.1631158 CrossRefGoogle Scholar
  114. 114.
    O’Hare N, Gurrin C, Lee H, Murphy N, Smeaton AF, Jones GJ (2005) My digital photos: where and when? In: Proceedings of the 13th annual ACM international conference on multimedia (MM). ACM, New York, NY, USA, pp 261–262. doi:10.1145/1101149.1101196. http://portal.acm.org/citation.cfm?id=1101196# CrossRefGoogle Scholar
  115. 115.
    Oliva A, Torralba A (2002) Scene-centered description from spatial envelope properties. In: Biologically motivated computer vision, vol 2525/2010, pp 263–272Google Scholar
  116. 116.
    On AV an IT Storage Systems, TSC, Equipment (2002) Exchangeable image file format for digital still cameras: exif version 2.2. Tech. rep., Japan Electronics and Information Technology Industries AssociationGoogle Scholar
  117. 117.
    Ortega-Binderberger M, Mehrotra S, Chakrabarti K, Porkaew K (2000) WebMARS: a multimedia search engineGoogle Scholar
  118. 118.
    Peng Wu DT (2009) Close & closer: social cluster and closeness from photo collections. In: Proceedings of MM’09, p 709Google Scholar
  119. 119.
    Penta A, Picariello A, Tanca L (2007) Towards a definition of an image ontology. In: International workshop on database and expert systems applications, pp 74–78. doi:10.1109/DEXA.2007.83
  120. 120.
    Petersen T (1994) Art and architecture thesaurus. Oxford, New YorkGoogle Scholar
  121. 121.
    Pigeau A (2010) Myownlife: incremental and hierarchical classification of a personal image collection on mobile devices. Multimed Tools Appl 46(2–3):289–306. doi: 10.1007/s11042-009-0373-x CrossRefGoogle Scholar
  122. 122.
    Pigeau A, Gelgon M (2005) Building and tracking hierarchical geographical & temporal partitions for image collection management on mobile devices. In: MULTIMEDIA ’05: proceedings of the 13th annual ACM international conference on multimedia. ACM, New York, NY, USA, pp 141–150. doi:10.1145/1101149.1101170 CrossRefGoogle Scholar
  123. 123.
    Platt J (2000) Autoalbum: clustering digital photographs using probabalistic model merging. citeseer.ist.psu.edu/platt00autoalbum.html
  124. 124.
    Platt JC, Czerwinski M, Field BA (2002) Phototoc: automatic clustering for browsing personal photographs. Tech. Rep. MSR-TR-2002-17, Microsoft Research. http://research.microsoft.com/~jplatt/abstracts/phototoc.html
  125. 125.
    Ren K, Calic J (2009) FreeEye—interactive intuitive interface for large-scale image browsing. In: ACM multimedia, p 757Google Scholar
  126. 126.
    Rodden K, Wood KR (2003) How do people manage their digital photographs? In: Gilbert Cockton PK (ed) Proceedings of the conference on human factors and computing systems, ACM, pp 409–416Google Scholar
  127. 127.
    Sandhaus P, Boll S (2009) From usage to annotation: analysis of personal photo albums for semantic photo understanding. In: Proceedings of the first SIGMM workshop on social media co-located with acm multimedia conference, Bejing, ChinaGoogle Scholar
  128. 128.
    Sandhaus P, Boll S, Fageth R (2008) Employing a photo’s life cycle for multimedia retrieval. In: MS ’08: proceeding of the 2nd ACM workshop on multimedia semantics. ACM, New York, NY, USA, pp 56–59. doi:10.1145/1460676.1460688 CrossRefGoogle Scholar
  129. 129.
    Sandhaus P, Thieme S, Boll S (2008) Processes of photo book production. Multimedia Syst 14(6):351–357CrossRefGoogle Scholar
  130. 130.
    Santini S, Gupta A, Jain R (2001) Emergent semantics through interaction in image databases. IEEE Trans Knowl Data Eng 13(3):337–351. doi: 10.1109/69.929893 CrossRefGoogle Scholar
  131. 131.
    Scherp A, Jain R (2007) Towards an ecosystem for semantics. In: MS ’07: workshop on multimedia information retrieval on the many faces of multimedia semantics. ACM, New York, NY, USA, pp 3–12. doi:10.1145/1290067.1290069 CrossRefGoogle Scholar
  132. 132.
    Scherp A, Boll S, Cremer H (2006) Emergent semantics in personalized multimedia content. In: Fourth special workshop on multimedia semantics (WMS), Chania, GreeceGoogle Scholar
  133. 133.
    Sclaroff S, La Cascia M, Sethi S: Unifying textual and visual cues for content-based image retrieval on the world wide web. Comput Vis Image Underst 75(1–2):86–98 (1999). doi: 10.1006/cviu.1999.0765 CrossRefGoogle Scholar
  134. 134.
    Shadbolt N, Lee TB, Hall W (2006) The semantic web revisited. In: IEEE intelligent systems, vol 21, pp 96–101. doi:10.1109/MIS.2006.62
  135. 135.
    Shirahatti NV, Barnard K (2005) Evaluating image retrieval. In: CVPR ’05: proceedings of the 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), vol 1. IEEE Computer Society, Washington, DC, USA, pp 955–961. doi: 10.1109/CVPR.2005.147 Google Scholar
  136. 136.
    Sinha P, Jain R (2008) Classification and annotation of digital photos using optical context data. In: CIVR. ACM, New York, NY, USA, pp 309–318. doi:10.1145/1386352.1386394 CrossRefGoogle Scholar
  137. 137.
    Sinha P, Jain R (2008) Semantics in digital photos: a contenxtual analysis. In: ICSC ’08: proceedings of the 2008 IEEE international conference on semantic computing. IEEE Computer Society, Washington, DC, USA, pp 58–65. doi: 10.1109/ICSC.2008.87 CrossRefGoogle Scholar
  138. 138.
    Smeulders AW, Worring M, Santini S, Gupta A, Jain R (2000) Content-based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell 22(12):1349–1380. doi:10.1109/34.895972 CrossRefGoogle Scholar
  139. 139.
    Sun X, Yao H, Ji R, Liu S (2009) Photo assessment based on computational visual attention model. In: Proceedings of MM’09, p 541Google Scholar
  140. 140.
    Tan T, Chen J, Mulhem P, Kankanhalli M (2002) SmartAlbum: a multi-modal photo annotation system. In: MULTIMEDIA ’02: proceedings of the tenth ACM international conference on multimedia. ACM, New York, NY, USA, pp 87–88. doi:10.1145/641007.641025 CrossRefGoogle Scholar
  141. 141.
    Tong H, Li M, Zhang H, He J, Zhang C (2004) Classification of digital photos taken by photographers or home users. In: Aizawa K, Nakamura Y, Satoh S (eds) PCM (1). Lecture notes in computer science, vol 3331. Springer, pp 198–205Google Scholar
  142. 142.
    Troncy R, van Ossenbruggen J, Pan JZ, Stamou G (2007) Image annotation on the semantic web. W3c note, W3C. http://www.w3.org/2005/Incubator/mmsem/XGR-image-annotation/
  143. 143.
    Trust G (2000) Union list of artists names onlineGoogle Scholar
  144. 144.
    Tsymbalenko Y, Munson EV (2001) Using HTML metadata to find relevant images on the world wide web. In: Proceedings of internet computing 2001, vol 2. CSREA Press, Las Vegas, pp 842–848Google Scholar
  145. 145.
    Tuffield MM, Gibbins N, O’Hara K, Brewster C, Dupplaw DP, Wilks Y, Sleeman D, Shadbolt NR (2006) Image annotation with photocopain. In: International workshop on semantic web annotations for multimediaGoogle Scholar
  146. 146.
    Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3(1):71–86. doi: 10.1162/jocn.1991.3.1.71 CrossRefGoogle Scholar
  147. 147.
    Vailaya A, Figueiredo M, Jain AK, Zhang J (1999) Content-based hierarchical classification of vacation images. Tech. Rep. MSU-CPS-99-9, Department of Computer Science, Michigan State University, East Lansing, MichiganGoogle Scholar
  148. 148.
    Van House N, Davis M, Ames M, Finn M, Viswanathan V (2005) The uses of personal networked digital imaging: an empirical study of cameraphone photos and sharing. In: CHI ’05: CHI ’05 extended abstracts on human factors in computing systems. ACM, New York, NY, USA, pp 1853–1856. doi: 10.1145/1056808.1057039 CrossRefGoogle Scholar
  149. 149.
    Van House N, Davis M, Takhteyev Y, Ames M, Finn M (2004) The social uses of personal photography: methods for projecting future imaging applications. University of California, Berkeley, Working Papers 3, 2005Google Scholar
  150. 150.
    Viana W, Filho JB, Gensel J, Oliver MV, Martin H (2007) Photomap—automatic spatiotemporal annotation for mobile photos. In: W2GIS’07: proceedings of the 7th international conference on web and wireless geographical information systems. Springer, Berlin, Heidelberg, pp 187–201CrossRefGoogle Scholar
  151. 151.
    Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. citeseer.ist.psu.edu/article/viola01rapid.html
  152. 152.
    Waal H, Couprie L, Tholen E, Vellekoop G (1985) Iconclass: an iconographic classification system. North-Holland, Amsterdam, The NetherlandsGoogle Scholar
  153. 153.
    Wagenaar WA (1986) My memory: a study of autobiographical memory over six years. Cogn Psychol 18(2):225–252. doi: 10.1016/0010-0285(86)90013-7. http://www.sciencedirect.com/science/article/B6WCR-4D6YVW4-1M/2/0b3ea4936d6016c24e6e896cc48c15e3 CrossRefGoogle Scholar
  154. 154.
    Wang YM, Zhang H (2004) Detecting image orientation based on low-level visual content. Comput Vis Image Underst (CVIU) 3(3):328–346CrossRefGoogle Scholar
  155. 155.
    Wang Z, Bovik A, Lu L (2002) Why is image quality assessment so difficult? In: IEEE international conference on acoustics speech and signal processing, vol 4. IEEE, pp 3313–3316Google Scholar
  156. 156.
    Wenyin L, Sun Y, Zhang H (2000) MiAlbum—a system for home photo managemet using the semi-automatic image annotation approach. In: MULTIMEDIA ’00: proceedings of the eighth ACM international conference on multimedia. ACM, New York, NY, USA, pp 479–480. doi:10.1145/354384.379011 CrossRefGoogle Scholar
  157. 157.
    Yan R, Fleury MO, Merler M, Natsev A, Smith JR (2009) Large-scale multimedia semantic concept modeling using robust subspace bagging and mapreduce. In: LS-MMRM ’09: proceedings of the first ACM workshop on large-scale multimedia retrieval and mining. ACM, New York, NY, USA, pp 35–42. doi:10.1145/1631058.1631067 CrossRefGoogle Scholar
  158. 158.
    Yang MH, Kriegman DJ, Ahuja N (2002) Detecting faces in images: a survey. IEEE Trans Pattern Anal Mach Intell 24(1):34–58. doi: 10.1109/34.982883 CrossRefGoogle Scholar
  159. 159.
    Yavlinsky A, Schofield E, Rüger S (2005) Automated image annotation using global features and robust nonparametric density estimation. In: Image and video retrieval, pp 507–517Google Scholar
  160. 160.
    You J, Perkis A, Hannuksela MM, Gabbouj M (2009) Perceptual quality assessment based on visual attention analysis. In: Proceedings of MM’09, p 561Google Scholar
  161. 161.
    Zagoris K, Chatzichristofis SA, Papamarkos N, Boutalis YS (2009) img(anaktisi): a web content based image retrieval system. In: SISAP ’09: proceedings of the 2009 second international workshop on similarity search and applications. IEEE Computer Society, Washington, DC, USA, pp 154–155. doi: 10.1109/SISAP.2009.15 CrossRefGoogle Scholar
  162. 162.
    Zhang T, Chao H, Willis C, Tretter D (2010) Consumer image retrieval by estimating relation tree from family photo collections. Tech. rep., HP LaboratoriesGoogle Scholar
  163. 163.
    Zhao W, Chellappa R, Phillips PJ, Rosenfeld A (2003) Face recognition: a literature survey. ACM Comput Surv 35(4):399–458. doi:10.1145/954339.954342 CrossRefGoogle Scholar
  164. 164.
    Zhao M, Teo Y, Liu S, Chua TS, Jain R (2006) Automatic person annotation of family photo album. In: CIVR, pp 163–172Google Scholar
  165. 165.
    Zhao J, Klyne G, Shotton D (2008) Building a semantic web image repository for biological research images. In: The semantic web: research and applications, pp 154–169Google Scholar
  166. 166.
    Zhu C, Li K, Lv Q, Shang L, Dick RP (2009) Iscope: personalized multi-modality image search for mobile devices. In: MobiSys ’09: proceedings of the 7th international conference on mobile systems, applications, and services. ACM, New York, NY, USA, pp 277–290. doi:10.1145/1555816.1555845 CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.HealthOFFIS – Institute for Information TechnologyOldenburgGermany
  2. 2.Department of Computing Science - Multimedia and Internet TechnologiesUniversity of OldenburgOldenburgGermany

Personalised recommendations