Multimedia Tools and Applications

, Volume 49, Issue 1, pp 119–144

Everyday concept detection in visual lifelogs: validation, relationships and trends

  • Daragh Byrne
  • Aiden R. Doherty
  • Cees G. M. Snoek
  • Gareth J. F. Jones
  • Alan F. Smeaton
Article

DOI: 10.1007/s11042-009-0403-8

Cite this article as:
Byrne, D., Doherty, A.R., Snoek, C.G.M. et al. Multimed Tools Appl (2010) 49: 119. doi:10.1007/s11042-009-0403-8

Abstract

The Microsoft SenseCam is a small lightweight wearable camera used to passively capture photos and other sensor readings from a user’s day-to-day activities. It captures on average 3,000 images in a typical day, equating to almost 1 million images per year. It can be used to aid memory by creating a personal multimedia lifelog, or visual recording of the wearer’s life. However the sheer volume of image data captured within a visual lifelog creates a number of challenges, particularly for locating relevant content. Within this work, we explore the applicability of semantic concept detection, a method often used within video retrieval, on the domain of visual lifelogs. Our concept detector models the correspondence between low-level visual features and high-level semantic concepts (such as indoors, outdoors, people, buildings, etc.) using supervised machine learning. By doing so it determines the probability of a concept’s presence. We apply detection of 27 everyday semantic concepts on a lifelog collection composed of 257,518 SenseCam images from 5 users. The results were evaluated on a subset of 95,907 images, to determine the accuracy for detection of each semantic concept. We conducted further analysis on the temporal consistency, co-occurance and relationships within the detected concepts to more extensively investigate the robustness of the detectors within this domain.

Keywords

Microsoft SenseCam Lifelog Passive photos Concept detection Supervised learning 

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • Daragh Byrne
    • 1
  • Aiden R. Doherty
    • 1
  • Cees G. M. Snoek
    • 2
  • Gareth J. F. Jones
    • 3
  • Alan F. Smeaton
    • 1
  1. 1.CLARITY: Centre for Sensor Web TechnologiesDublin City UniversityDublin 9Ireland
  2. 2.Intelligent Systems Lab AmsterdamUniversity of AmsterdamAmsterdamThe Netherlands
  3. 3.Centre for Digital Video ProcessingDublin City UniversityGlasnevinIreland