Journal on Multimodal User Interfaces

, Volume 4, Issue 2, pp 81–95

Design and implementation of an affect-responsive interactive photo frame

Authors

  • Hamdi Dibeklioğlu
    • Intelligent Systems Lab Amsterdam, Informatics InstituteUniversity of Amsterdam
  • Marcos Ortega Hortas
    • University of A Coruna
  • Ilkka Kosunen
    • Helsinki Institute for Information Technology HIITUniversity of Helsinki
  • Petr Zuzánek
    • Faculty of Information TechnologyCzech Technical University in Prague
    • Intelligent Systems Lab Amsterdam, Informatics InstituteUniversity of Amsterdam
  • Theo Gevers
    • Intelligent Systems Lab Amsterdam, Informatics InstituteUniversity of Amsterdam
Open AccessOriginal Paper

DOI: 10.1007/s12193-011-0057-5

Cite this article as:
Dibeklioğlu, H., Ortega Hortas, M., Kosunen, I. et al. J Multimodal User Interfaces (2011) 4: 81. doi:10.1007/s12193-011-0057-5

Abstract

This paper describes an affect-responsive interactive photo-frame application that offers its user a different experience with every use. It relies on visual analysis of activity levels and facial expressions of its users to select responses from a database of short video segments. This ever-growing database is automatically prepared by an offline analysis of user-uploaded videos. The resulting system matches its user’s affect along dimensions of valence and arousal, and gradually adapts its response to each specific user. In an extended mode, two such systems are coupled and feed each other with visual content. The strengths and weaknesses of the system are assessed through a usability study, where a Wizard-of-Oz response logic is contrasted with the fully automatic system that uses affective and activity-based features, either alone, or in tandem.

Keywords

Affective computingFacial expressionOptical flowInteractive photographHuman behavior understandingAutomatic video segmentation
Download to read the full article text

Copyright information

© The Author(s) 2011