Journal on Multimodal User Interfaces

, Volume 4, Issue 2, pp 81–95

Design and implementation of an affect-responsive interactive photo frame

  • Hamdi Dibeklioğlu
  • Marcos Ortega Hortas
  • Ilkka Kosunen
  • Petr Zuzánek
  • Albert Ali Salah
  • Theo Gevers
Open AccessOriginal Paper

DOI: 10.1007/s12193-011-0057-5

Cite this article as:
Dibeklioğlu, H., Ortega Hortas, M., Kosunen, I. et al. J Multimodal User Interfaces (2011) 4: 81. doi:10.1007/s12193-011-0057-5

Abstract

This paper describes an affect-responsive interactive photo-frame application that offers its user a different experience with every use. It relies on visual analysis of activity levels and facial expressions of its users to select responses from a database of short video segments. This ever-growing database is automatically prepared by an offline analysis of user-uploaded videos. The resulting system matches its user’s affect along dimensions of valence and arousal, and gradually adapts its response to each specific user. In an extended mode, two such systems are coupled and feed each other with visual content. The strengths and weaknesses of the system are assessed through a usability study, where a Wizard-of-Oz response logic is contrasted with the fully automatic system that uses affective and activity-based features, either alone, or in tandem.

Keywords

Affective computingFacial expressionOptical flowInteractive photographHuman behavior understandingAutomatic video segmentation
Download to read the full article text

Copyright information

© The Author(s) 2011

Authors and Affiliations

  • Hamdi Dibeklioğlu
    • 1
  • Marcos Ortega Hortas
    • 2
  • Ilkka Kosunen
    • 3
  • Petr Zuzánek
    • 4
  • Albert Ali Salah
    • 1
  • Theo Gevers
    • 1
  1. 1.Intelligent Systems Lab Amsterdam, Informatics InstituteUniversity of AmsterdamAmsterdamThe Netherlands
  2. 2.University of A CorunaCorunnaSpain
  3. 3.Helsinki Institute for Information Technology HIITUniversity of HelsinkiHelsinkiFinland
  4. 4.Faculty of Information TechnologyCzech Technical University in PraguePrague 6Czech Republic