Multimedia Tools and Applications

, Volume 55, Issue 1, pp 27–52

Reliability and effectiveness of clickthrough data for automatic image annotation

  • Theodora Tsikrika
  • Christos Diou
  • Arjen P. de Vries
  • Anastasios Delopoulos
Article

DOI: 10.1007/s11042-010-0584-1

Cite this article as:
Tsikrika, T., Diou, C., de Vries, A.P. et al. Multimed Tools Appl (2011) 55: 27. doi:10.1007/s11042-010-0584-1

Abstract

Automatic image annotation using supervised learning is performed by concept classifiers trained on labelled example images. This work proposes the use of clickthrough data collected from search logs as a source for the automatic generation of concept training data, thus avoiding the expensive manual annotation effort. We investigate and evaluate this approach using a collection of 97,628 photographic images. The results indicate that the contribution of search log based training data is positive despite their inherent noise; in particular, the combination of manual and automatically generated training data outperforms the use of manual data alone. It is therefore possible to use clickthrough data to perform large-scale image annotation with little manual annotation effort or, depending on performance, using only the automatically generated training data. An extensive presentation of the experimental results and the accompanying data can be accessed at http://olympus.ee.auth.gr/~diou/civr2009/.

Keywords

Image annotation Concepts Supervised learning Search logs Clickthrough data Collective knowledge Implicit feedback Reliability 

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • Theodora Tsikrika
    • 1
  • Christos Diou
    • 2
    • 3
  • Arjen P. de Vries
    • 1
    • 4
  • Anastasios Delopoulos
    • 2
    • 3
  1. 1.Centrum Wiskunde & InformaticaAmsterdamThe Netherlands
  2. 2.Multimedia Understanding Group, Electrical and Computer Engineering DepartmentAristotle University of ThessalonikiThessalonikiGreece
  3. 3.Informatics and Telematics InstituteCentre for Research and Technology HellasThessalonikiGreece
  4. 4.Delft University of TechnologyDelftThe Netherlands

Personalised recommendations