Skip to main content

Addressing the Cold-Start Problem in Facial Expression Recognition

Part of the Lecture Notes in Computer Science book series (LNAI,volume 9343)

Abstract

In our previous research [5] we proposed a CBR approach to infer the emotional state of the user through the analysis of a picture taken from the front facing camera of her mobile device. We demonstrated that different people express emotions with different gestures and got the best accuracy using a personal case base with self pictures of the same user. However, in the cold start situation, where pictures of the querying user are not available, the CBR system uses a generic case base (GCB) made of pictures of anonymous people. Although the performance using the GCB was acceptable on average there were several users with a very low accuracy. In this paper we compare our GCB to other reference picture catalogues and evaluate our CBR approach with state-of-the-art Facial Expression Recognition (FER) algorithms. Results point out that our approach is only suitable for GCB including semantically similar users. We use an ontology to group together users with similar demographic and physiological information: sex, age and ethnic group. We evaluate our CBR approach with small and specialized case bases where pictures are semantically similar to the target population and demonstrate that it efficiently increases the accuracy in the cold start situation and minimizes the noise in the case base.

Keywords

  • Case Base
  • Recommender System
  • Cold Start
  • Facial Expression Recognition
  • Facial Expression Recognition System

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-24586-7_14
  • Chapter length: 15 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   59.99
Price excludes VAT (USA)
  • ISBN: 978-3-319-24586-7
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   79.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.

Notes

  1. 1.

    We used Google Image Search with the queries “Happy face”, “Unhappy faces” and “Surprise faces”.

  2. 2.

    We leave aside possible miss-classifications of pictures in the dataset.

  3. 3.

    The accuracy is taken from [9], whereas the DCS-S1 accuracy is obtained from [17].

  4. 4.

    We have used the"Geographical Races" taxonomy proposed by Garn [4].

References

  1. Adomavicius, G., Mobasher, B., Ricci, F., Tuzhilin, A.: Context-aware recommender systems. AI Mag. 32(3), 67–80 (2011)

    Google Scholar 

  2. Benou, P., Bitos, V.: Context-aware query processing in ad-hoc environments of peers. JECO 6(1), 88 (2008)

    Google Scholar 

  3. Braunhofer, M., Kaminskas, M., Ricci, F.: Location-aware music recommendation. IJMIR 2(1), 31–44 (2013)

    Google Scholar 

  4. Garn, S.M.: Human Races, 3rd edn. Thomas, Springfield (1971)

    Google Scholar 

  5. Jorro-Aragoneses, J.L., Díaz-Agudo, B., Recio-García, J.A.: Optimization of a CBR system for emotional tagging of facial expressions. In: UKCBR (2014)

    Google Scholar 

  6. Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 484–490 (2000)

    Google Scholar 

  7. Khanum, A., Mufti, M., Javed, M.Y., Shafiq, M.Z.: Fuzzy case-based reasoning for facial expression recognition. Fuzzy Sets Syst. 160(2), 231–250 (2009)

    MathSciNet  CrossRef  Google Scholar 

  8. Kotsia, I., Pitas, I.: Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Trans. Image Process. 16(1), 172–187 (2007)

    MathSciNet  CrossRef  Google Scholar 

  9. Lee, S.H., Plataniotis, K., Ro, Y.M.: Intra-class variation reduction using training expression images for sparse representation based facial expression recognition. In: IEEE Transactions on Affective Computing, p. 1 (2014)

    Google Scholar 

  10. Lika, B., Kolomvatsos, K., Hadjiefthymiades, S.: Facing the cold start problem in recommender systems. Expert Syst. Appl. 41(4, Part 2), 2065–2073 (2014). http://www.sciencedirect.com/science/article/pii/S0957417413007240

    CrossRef  Google Scholar 

  11. Lopez-de-Arenosa, P., Díaz-Agudo, B., Recio-García, J.A.: CBR tagging of emotions from facial expressions. In: Lamontagne, L., Plaza, E. (eds.) ICCBR 2014. LNCS, vol. 8765, pp. 245–259. Springer, Heidelberg (2014)

    Google Scholar 

  12. Lyons, M., Akamatsu, S.: Coding facial expressions with gabor wavelets. In: Coding Facial Expressions with Gabor Wavelets. pp. 200–205 (1998)

    Google Scholar 

  13. Novak, D., Nagle, A., Riener, R.: Linking recognition accuracy and user experience in an affective feedback loop. IEEE Trans. Affect. Comput. 5(2), 168–172 (2014)

    CrossRef  Google Scholar 

  14. Ojansivu, V., Heikkilä, J.: Blur insensitive texture classification using local phase quantization. In: Elmoataz, A., Lezoray, O., Nouboud, F., Mammass, D. (eds.) ICISP 2008. LNCS, vol. 5099, pp. 236–243. Springer, Heidelberg (2008)

    CrossRef  Google Scholar 

  15. Recio-García, J.A., Díaz-Agudo, B., González-Calero, P.A., Sánchez-Ruiz-Granados, A.: Ontology based CBR with jCOLIBRI. In: Proceedings of the 26th SGAI International Conference on Innovative Techniques and Applications of Artificial Intelligence, pp. 149–162. Springer, Cambridge (2006)

    Google Scholar 

  16. Son, L.H.: Dealing with the new user cold-start problem in recommender systems: A comparative review. Information Systems (0) (2014). http://www.sciencedirect.com/science/article/pii/S0306437914001525

  17. Taheri, S., Patel, V., Chellappa, R.: Component-based recognition of facesand facial expressions. IEEE Trans. Affect. Comput. 4(4), 360–371 (2013)

    CrossRef  Google Scholar 

  18. Tian, Y., Kanade, T., Cohn, J.F.: Facial expression recognition. Handbook of Face Recognition, pp. 487–519. Springer, New York (2011)

    CrossRef  Google Scholar 

  19. Tkalcic, M., de Gemmis, M., Semeraro, G.: Personality and emotions in decision making and recommender systems. In: First International Workshop on Decision Making and Recommender Systems, pp. 14–18. CEUR (2014)

    Google Scholar 

  20. Tkalcic, M., Kosir, A., Tasic, J.: Affective recommender systems: the role of emotions in recommender systems. In: Proceeding of the RecSys 2011 Workshop on Human Decision Making in Recommender Systems, pp. 9–13. Citeseer (2011)

    Google Scholar 

  21. Whitehill, J., Serpell, Z., Lin, Y.C., Foster, A., Movellan, J.: The faces of engagement: Automatic recognition of student engagementfrom facial expressions. IEEE Trans. Affect. Comput. 5(1), 86–98 (2014)

    CrossRef  Google Scholar 

  22. Zhen, W., Zilu, Y.: Facial expression recognition based on local phase quantization and sparse representation. In: 2012 Eighth International Conference on Natural Computation (ICNC), pp. 222–225 (2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jose L. Jorro-Aragoneses .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Jorro-Aragoneses, J.L., Díaz-Agudo, B., Recio-García, J.A. (2015). Addressing the Cold-Start Problem in Facial Expression Recognition. In: Hüllermeier, E., Minor, M. (eds) Case-Based Reasoning Research and Development. ICCBR 2015. Lecture Notes in Computer Science(), vol 9343. Springer, Cham. https://doi.org/10.1007/978-3-319-24586-7_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-24586-7_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-24585-0

  • Online ISBN: 978-3-319-24586-7

  • eBook Packages: Computer ScienceComputer Science (R0)