Skip to main content

Part of the book series: Studies in Computational Intelligence ((SCI,volume 569))

  • 893 Accesses


An emotional scene detection method is proposed in order to retrieve impressive scenes from lifelog videos. The proposed method is based on facial expression recognition considering that a wide variety of facial expression could be observed in impressive scenes. Most of conventional facial expression techniques adopt supervised learning methods. This is a crucial problem because preparing sufficient training data requires considerable human effort due to the diversity of facial expressions observed in lifelog videos. We thus propose a more efficient emotional scene detection method using an unsupervised facial expression recognition on the basis of cluster ensembles. Our approach does not require any training data sets and is able to detect various emotional scenes. The detection performance of the proposed method is evaluated through an emotional scene detection experiment.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

USD 16.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Aizawa, K., Hori, T., Kawasaki, S., Ishikawa, T.: Capture and Efficient Retrieval of Life Log. In: Proc. of Pervasive 2004 Workshop on Memory and Sharing Experiences, pp. 15–20 (2004)

    Google Scholar 

  2. Gemmell, J., Bell, G., Luederand, R., Drucker, S., Wong, C.: MyLifeBits: Fulfilling the Memex Vision. In: Proc. of the 10th ACM International Conference on Multimedia, pp. 235–238 (2002)

    Google Scholar 

  3. Datchakorn, T., Yamasaki, T., Aizawa, K.: Practical Experience Recording and Indexing of Life Log Video. In: Proc. of the 2nd ACM Workshop on Continuous Archival and Retrieval of Personal Experiences, pp. 61–66 (2005)

    Google Scholar 

  4. Mehrabian, A.: Silent Messages. Wadsworth, Belmont (1971)

    Google Scholar 

  5. Datcu, D., Rothkrantz, L.: Facial Expression Recognition in Still Pictures and Videos Using Active Appearance Models: A Comparison Approach. In: Proc. of the 2007 International Conference on Computer Systems and Technologies, pp. 1–6 (2007)

    Google Scholar 

  6. Fanelli, G., Yao, A., Noel, P.-L., Gall, J., Van Gool, L.: Hough Forest-based Facial Expression Recognition from Video Sequences. In: Kutulakos, K.N. (ed.) ECCV 2010 Workshops, Part I. LNCS, vol. 6553, pp. 195–206. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  7. Littlewort, G., Bartlett, M.S., Fasel, I., Susskind, J., Movellan, J.: Dynamics of Facial Expression Extracted Automatically from Video. Journal of Image and Vision Computing 24(6), 615–625 (2004)

    Article  Google Scholar 

  8. Strehl, A., Ghosh, J.: Cluster Ensembles – a Knowledge Reuse Framework for Combining Multiple Partitions. Journal of Machine Learning Research 3, 583–617 (2003)

    MATH  MathSciNet  Google Scholar 

  9. Tian, Y., Kanade, T., Cohn, J.F.: Facial Expression Recognition. In: Li, S.Z., Jain, A.K. (eds.) Handbook of Face Recognition, pp. 487–519. Springer, London (2011)

    Chapter  Google Scholar 

  10. Lyons, M., Akamatsu, S.: Coding Facial Expressions with Gabor Wavelets. In: Proc. of the 3rd International Conference on Automatic Face and Gesture Recognition, pp. 200–205 (1998)

    Google Scholar 

  11. Feng, X., Pietikäinen, M.: Facial Expression Recognition with Local Binary Patterns and Linear Programming. Pattern Recognition and Image Analysis 15(2), 546–548 (2005)

    Google Scholar 

  12. Wang, J., Yin, L., Wei, X., Sun, Y.: 3D Facial Expression Recognition Based on Primitive Surface Feature Distribution. In: Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1399–1406 (2006)

    Google Scholar 

  13. Soyel, H., Demirel, H.O.: Facial Expression Recognition Using 3D Facial Feature Distances. In: Kamel, M.S., Campilho, A. (eds.) ICIAR 2007. LNCS, vol. 4633, pp. 831–838. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  14. Esau, N., Wetzel, E., Kleinjohann, L., Kleinjohann, B.: Real-time Facial Expression Recognition Using a Fuzzy Emotion Model. In: Proc. of International Conference on Fuzzy Systems, pp. 1–6 (2007)

    Google Scholar 

  15. Hupont, I., Cerezo, E., Baldassarri, S.: Sensing Facial Emotion in a Continuous 2D Affective Space. In: Proc. of International Conference on Systems, Man, and Cybernetics, pp. 2045–2051 (2010)

    Google Scholar 

  16. Nomiya, H., Morikuni, A., Hochin, T.: Emotional Video Scene Detection from Lifelog Videos using Facial Feature Selection. In: Proc. of 4th International Conference on Applied Human Factors and Ergonomics, pp. 8500–8509 (2012)

    Google Scholar 

  17. Hoey, J.: Hierarchical Unsupervised Learning of Facial Expression Categories. In: Proc. of IEEE Workshop on Detection and Recognition of Events in Video, pp. 99–106 (2001)

    Google Scholar 

  18. Gholami, B., Haddad, W.M., Tannenbaum, A.R.: An Unsupervised Learning Approach for Facial Expression Recognition Using Semi-Definite Programming and Generalized Principal Component Analysis. In: Proc. of Image Processing: Algorithms and Systems, pp. 1–10 (2010)

    Google Scholar 

  19. Nomiya, H., Morikuni, A., Hochin, T.: Impressive Scene Detection from Lifelog Videos by Unsupervised Facial Expression Recognition. In: Proc. of 14th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, pp. 444–449 (2013)

    Google Scholar 

  20. Luxand Inc., Luxand FaceSDK 4.0, (April 18, 2014)

  21. Ekman, P., Friesen, W.: Unmasking the Face: A Guide to Recognizing Emotions from Facial Clues. Prentice Hall, Englewood Cliffs (1975)

    Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Hiroki Nomiya .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Nomiya, H., Morikuni, A., Hochin, T. (2015). An Unsupervised Ensemble Approach for Emotional Scene Detection from Lifelog Videos. In: Lee, R. (eds) Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing. Studies in Computational Intelligence, vol 569. Springer, Cham.

Download citation

  • DOI:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-10388-4

  • Online ISBN: 978-3-319-10389-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics