Advertisement

Image-based Analysis of Emotional Facial Expressions in Full Face Transplants

  • Merve Bedeloglu
  • Çagdas Topcu
  • Arzu Akgul
  • Ela Naz Döger
  • Refik Sever
  • Ozlenen Ozkan
  • Omer Ozkan
  • Hilmi Uysal
  • Ovunc Polat
  • Omer Halil ÇolakEmail author
Image & Signal Processing
Part of the following topical collections:
  1. Image & Signal Processing

Abstract

In this study, it is aimed to determine the degree of the development in emotional expression of full face transplant patients from photographs. Hence, a rehabilitation process can be planned according to the determination of degrees as a later work. As envisaged, in full face transplant cases, the determination of expressions can be confused or cannot be achieved as the healthy control group. In order to perform image-based analysis, a control group consist of 9 healthy males and 2 full-face transplant patients participated in the study. Appearance-based Gabor Wavelet Transform (GWT) and Local Binary Pattern (LBP) methods are adopted for recognizing neutral and 6 emotional expressions which consist of angry, scared, happy, hate, confused and sad. Feature extraction was carried out by using both methods and combination of these methods serially. In the performed expressions, the extracted features of the most distinct zones in the facial area where the eye and mouth region, have been used to classify the emotions. Also, the combination of these region features has been used to improve classifier performance. Control subjects and transplant patients’ ability to perform emotional expressions have been determined with K-nearest neighbor (KNN) classifier with region-specific and method-specific decision stages. The results have been compared with healthy group. It has been observed that transplant patients don’t reflect some emotional expressions. Also, there were confusions among expressions.

Keywords

Facial expression recognition Face transplantation Gabor wavelets Local binary pattern K-nearest neighbor 

Notes

Acknowledgements

We also acknowledge healthy group and transplant patients for their helping to make up the database.

Funding

This study was supported by TUBITAK (Project Number: 113E182).

Compliance with ethical standards

Conflict of interest

All authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. (Ethical Approval Date-Number: Akdeniz University, Clinical Research Ethics Committee: 04/03/2013–25).

Informed consent

Informed consent was obtained from all individual participants included in the study.

References

  1. 1.
    Bedeloglu, M., The Analysis of Emotional Expression Development of Face Transplant Patients by Using Image Processing Algorithms, MsC Thesis, Akdeniz University, 2016.Google Scholar
  2. 2.
    Bedeloglu, M., Topcu, C., Döger, E.N., Akgul, A., Sever, S., Ozkan, O., Ozkan, O., Uysal, H., Polat, O., Colak, O.H., Image based Analysis of Emotional Expression Developmetn in Facial Transplant Patient, TIPTEKNO2015, ss.1–4, 15–18 October 2015, Muğla, Turkey, 2015.Google Scholar
  3. 3.
    Shan C., Gong S., McOwan P.W., Robust facial expression recognition using local binary patterns. ICIP 2005. IEEE International Conference on (Vol. 2, pp. II-370). IEEE, 2005, September, 2005.Google Scholar
  4. 4.
    Tian, Y.L., Kanade, T., and Cohn, J.F., Facial expression analysis. Handbook of face recognition. Springer, New York, pp. 247–275, 2005.CrossRefGoogle Scholar
  5. 5.
    Tian, Y.L., Evaluation of face resolution for expression analysis. Computer Vision and Pattern Recognition Workshop. CVPRW'04. Conference on (pp. 82–82). IEEE, 2004, June, 2004.Google Scholar
  6. 6.
    Ekman, P., and Rosenberg, E.L., What the face reveals: Basic and applied studies of spontaneous expression using the facial action coding system (FACS). Oxford University Press, USA, 1997.Google Scholar
  7. 7.
    Nagarajan, R., Hariharan, M., and Satiyan, M., Luminance Sticker Based Facial Expression Recognition Using Discrete Wavelet Transform for Physically Disabled Persons. J. Med. Syst. 36(4):2225–2234, 2012.CrossRefPubMedGoogle Scholar
  8. 8.
    Haghighat, M., Zonouz, S., and Abdel-Mottaleb, M., Identification using encrypted biometrics. Computer Analysis of Images and Patterns. Springer, Berlin, Heidelberg, 2013.  https://doi.org/10.1007/978-3-642-40246-3_55.Google Scholar
  9. 9.
    Chao, W., Gabor wavelet transform and its application. Wei-lun Chao R989420 73.Google Scholar
  10. 10.
    Meshgini, S., Aghagolzadeh, A., and Seyedarabi, H., Face recognition using gabor filter bank, kernel principle component analysis and support vector machine. International Journal of Computer Theory and Engineering. 4(5):767–771, 2012.CrossRefGoogle Scholar
  11. 11.
    http://www.atasoyweb.net/Gabor-Filtresi, http://www.atasoyweb.net/Gabor-Filtresi. Accessed 13 February 2016.
  12. 12.
    Shen, L., Bai, L., and Fairhurst, M., Gabor wavelets and general discriminant analysis for face identification and verification. Image Vis. Comput. 25(5):553–563, 2007.CrossRefGoogle Scholar
  13. 13.
    Bartlett, M.S., Littlewort, G., Fasel, I., Movellan, J.R., Real time face detection and facial expression recognition: Development and applications to human computer interaction. Computer Vision and Pattern Recognition Workshop, 2003. CVPRW'03. Conference on (Vol. 5, pp. 53–53). IEEE, 2003, June, 2003.Google Scholar
  14. 14.
    Ojala, T., Pietikainen, M., and Harwood, D., A comparative study of texture measures with classification based on featured distributions. Pattern Recogn. 29(1):51–59, 1996.  https://doi.org/10.1016/0031-3203(95)00067-4.CrossRefGoogle Scholar
  15. 15.
    Ojala, T., Pietikainen, M., and Maenpaa, T., Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7):971–987, 2002.  https://doi.org/10.1109/TPAMI.2002.1017623.CrossRefGoogle Scholar
  16. 16.
    Feng, X., Hadid, A., and Pietikainen, M., A coarse-to-fine classification scheme for facial expression recognition. Image Analysis and Recognition., 2004.  https://doi.org/10.1007/978-3-540-30126-4_81.
  17. 17.
    Huang, D., Shan, C., Ardabilian, M., Wang, Y., and Chen, L., Local binary patterns and its application to facial image analysis: A survey. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 41(6):765–781, 2011.  https://doi.org/10.1109/TSMCC.2011.2118750.CrossRefGoogle Scholar
  18. 18.
    Singh, R., Vatsa, M., and Noore, A., Integrated multilevel image fusion and match score fusion of visible and infrared face images for robust face recognition. Pattern Recognition, Part Special issue: Feature Generation and Machine Learning for Robust Multimodal Biometrics. 41(3):880–893, 2008.  https://doi.org/10.1016/j.patcog.2007.06.022.CrossRefGoogle Scholar
  19. 19.
    Zhang, W., Shan, S., Gao, W., Chen, X., Zhang, H., Local gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition. Tenth IEEE International Conference on Computer Vision. ICCV 2005, 1:786–91 Vol. 1. doi:10.1109/ICCV.2005.147, 2005.Google Scholar
  20. 20.
    Neth, D., and Martinez, A.M., A computational shape-based model of anger and sadness justifies a configural representation of faces. Vis. Res. 50:1693–1711, 2010.CrossRefPubMedPubMedCentralGoogle Scholar
  21. 21.
    Ekman, P., Basic Emotions. In: Dalgleish, T., and Power, T. (Eds.), The Handbook of Cognition and Emotion. John Wiley & Sons, Ltd., Sussex, pp. 45–60, 1999.Google Scholar
  22. 22.
    Duchenne, B., Mécanisme de la physionomie humaine, ou analyse électrophysiologique de L'expression des passions. Bailliere, Paris, 1862.CrossRefGoogle Scholar
  23. 23.
    Ekman, P., and Friesen, W.V., Constants Accross Cultures in the Face and Emotion. J. Pers. Soc. Psychol. 17(2):124–129, 1971.CrossRefPubMedGoogle Scholar
  24. 24.
    Uysal, H., Topcu, C., Ozkan, O., Ozkan, O., Barcin, N.E., Akgul, A., Bedeloglu, M., Döger, E.N., Sever, R., Polat, O., and Colak, O.H., Electrophysiological evaluation of emotional expressions in the facial transplantation patients. Clin. Neurophysiol., 2016.  https://doi.org/10.1016/j.clinph.2015.11.431.

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Merve Bedeloglu
    • 1
  • Çagdas Topcu
    • 1
  • Arzu Akgul
    • 1
  • Ela Naz Döger
    • 1
  • Refik Sever
    • 1
  • Ozlenen Ozkan
    • 2
  • Omer Ozkan
    • 2
  • Hilmi Uysal
    • 3
  • Ovunc Polat
    • 1
  • Omer Halil Çolak
    • 1
    Email author
  1. 1.Faculty of Engineering, Department of Electrical and Electronics EngineeringAkdeniz UniversityAntalyaTurkey
  2. 2.Faculty of Medicine, Department of Plastic and Reconstructive SurgeryAkdeniz UniversityAntalyaTurkey
  3. 3.Faculty of Medicine, Department of NeurologyAkdeniz UniversityAntalyaTurkey

Personalised recommendations