Improving the Usability of Hierarchical Representations for Interactively Labeling Large Image Data Sets

  • Julia Moehrmann
  • Stefan Bernstein
  • Thomas Schlegel
  • Günter Werner
  • Gunther Heidemann
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6761)


Image recognition systems require large image data sets for the training process. The annotation of such data sets through users requires a lot of time and effort, and thereby presents the bottleneck in the development of recognition systems. In order to simplify the creation of image recognition systems it is necessary to develop interaction concepts for optimizing the usability of labeling systems. Semi-automatic approaches are capable of solving the labeling task by clustering the image data unsupervised and presenting this ordered set to a user for manual labeling. A labeling interface based on self-organizing maps (SOM) was developed and its usability was investigated in an extensive user study with 24 participants. The evaluation showed that SOM-based visualizations are suitable for speeding up the labeling process and simplifying the task for users. Based on the results of the user study, further concepts were developed to improve the usability.


Self-organizing map SOM user study image labeling ground truth data 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    von Ahn, L., Dabbish, L.: Labeling Images with a Computer Game. In: SIGCHI Conference on Human Factors in Computing Systems, pp. 319–326 (2004)Google Scholar
  2. 2.
    von Ahn, L., Riu, R., Blum, M.: Peekaboom: a Game for Locating Objects in Images. In: SIGCHI Conference on Human Factors in Computing Systems (CHI 2006), pp. 55–64 (2006)Google Scholar
  3. 3.
    Ho, C.-J., Chang, T.-H., Lee, J.-C., Hsu, J.Y.-J., Chen, K.T.: KissKissBan: a Competitive Human Computation Game for Image Annotation. In: ACM SIGKDD Workshop on Human Computation, pp. 11–14 (2009)Google Scholar
  4. 4.
    Seneviratne, L., Izquierdo, E.: An Interactive Framework for Image Annotation through Gaming. In: International Conference on Multimedia Information Retrieval, pp. 517–526 (2010)Google Scholar
  5. 5.
    Russel, B.C., Torralba, A., Murphy, K.P., Freeman, W.T.: LabelMe: A Database and Web-based tool for image annotation. Int. J. on Computer Vision 77(1), 157–173 (2008)CrossRefGoogle Scholar
  6. 6.
    Yao, B., Yang, X., Zhu, S.-C.: Introduction to a Large-Scale general Purpose Ground Truth Database: Methodology, Annotation Tool and Benchmarks. In: International Conference on Energy Minimization Methods in Computer Vision and Pattern Recgnition, pp. 169–183 (2007)Google Scholar
  7. 7.
    Koskela, M., Laaksonen, J.: Semantic Annotation of Image Groups with Self-Organizing Maps. In: Leow, W.-K., Lew, M., Chua, T.-S., Ma, W.-Y., Chaisorn, L., Bakker, E.M. (eds.) CIVR 2005. LNCS, vol. 3568, pp. 518–527. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  8. 8.
    Laaksonen, J., Koskela, M., Laakso, S., Oja, E.: PicSOM – Content-based Image Retrieval with Self-Organizing Maps. Pattern Recognition LettersGoogle Scholar
  9. 9.
    Heidemann, G., Saalbach, A., Ritter, H.: Semi-Automatic Acquisition and Labelling of Image Data using SOMs. In: European Symposium on Artificial Neural Networks, pp. 503–508 (2003)Google Scholar
  10. 10.
    Bederson, B.: PhotoMesa: a Zoomable Image Browser using Quantum Treemaps and Bubblemaps. In: ACM Symposium on User Interface Software and Technology, pp. 71–80 (2001)Google Scholar
  11. 11.
    Platt, J., Czerwinski, M., Field, B.: PhotoTOC: Automatic Clustering for Browsing Personal Photographs. In: Joint Conference of the International Conference on Information, Communications and Signal Processing, and the Pacific Rim Conference on Multimedia, vol. 1, pp. 6–10 (2003)Google Scholar
  12. 12.
    Gomi, A., Miyazaki, R., Itoh, T., Li, J.: CAT: a Hierarchical Image Browser using a Rectangle Packing Technique. In: International Conference on Information Visualisation, pp. 82–87 (2008)Google Scholar
  13. 13.
    Kohonen, T.: The Self-Organizing Map. Proceedings of the IEEE 78(9), 1464–1480 (1990)CrossRefGoogle Scholar
  14. 14.
    Vesanto, J., Himberg, J., Alhoniemi, E., Parhankangas, J.: Self-Organizing Map in Matlab: the SOM toolbox. In: Matlab DSP Conference, pp. 35–40 (1999)Google Scholar
  15. 15.
    Ultsch, A., Sieman, H.P.: Kohonen’s Self Organizing Feature Maps for Exploratory Data Analysis. In: Proceedings of International Neural Networks Conference, pp. 305–308. Kluwer Academic Press, Dordrecht (1990)Google Scholar
  16. 16.
    Nene, S., Nayar, S., Murase, H.: Columbia Object Image Library (COIL-100). Technical report, Columbia University (1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Julia Moehrmann
    • 1
  • Stefan Bernstein
    • 2
  • Thomas Schlegel
    • 3
  • Günter Werner
    • 2
  • Gunther Heidemann
    • 1
  1. 1.Intelligent Systems GroupUniversity of StuttgartStuttgartGermany
  2. 2.University of Apllied Sciences MittweidaMittweidaGermany
  3. 3.Softwareentwicklung ubiquitärer SystemeTU DresdenDresdenGermany

Personalised recommendations