Skip to main content
SpringerLink
Log in

Image super-resolution model using an improved deep learning-based facial expression analysis

  • Special Issue Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Image upsampling and noise removal are important tasks in digital image processing. Single-image upsampling and denoising influence the quality of the resulting images. Image upsampling is known as super-resolution (SR) and referred to as the restoration of a higher-resolution image from a given low-resolution image. In facial expression analysis, the resolution of the original image directly affects the reliability and validity of the emotional analysis. Hence, optimization of the resolution of the extracted original image during emotion analysis is important. In this study, a model is proposed, which applies an image super-resolution method to an algorithm that classifies emotions from facial expressions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Ekman, P., Keltner, D.: Facial expression of emotion. In: Lewis, M. and Haviland-Jones, J. (eds). Handbook of Emotions, pp. 236–249. New York, (2000)

  2. Navaz, A., Adel, S., Mathew, S.: Facial image pre-processing and emotion classification: a deep learning approach, IEEE/ACS 16th International Conference on Computer Systems and Applications (AICCSA). Abu Dhabi, United Arab Emirates, pp.1–8 (2019)

  3. Darwin, C.: The Expression of the Emotions in Man and Animals. John Murray, London (1872)

    Book  Google Scholar 

  4. Jack, R., Garrod, O., Yu, H., Caldara, R., Schyns, P.: Facial expressions of emotion are not culturally universal. PNAS 109(19), 7241–7244 (2012)

    Article  Google Scholar 

  5. Namburi, P., Al-Hasani, R., Calhoon, G., Bruchas, M., Tye, K.: Architectural representation of valence in the limbic system. Neuropsychopharmacology 41(7), 1697–1715 (2016)

    Article  Google Scholar 

  6. Fox, S.: Human Physiology. McGraw-Hill Higher Education, New York (2018)

    Google Scholar 

  7. Tomkins, S.: Affect, Imagery, and Consciousness. Springer, New York (1962)

    Google Scholar 

  8. Russell, J.: A circumplex model of affect. J. Pers. Soc. Psychol. 39(6), 1161–1178 (1980)

    Article  Google Scholar 

  9. Plutchik, R.: The nature of emotions: Human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice. Am. Sci. 89(4), 344–350 (2001)

    Article  Google Scholar 

  10. Dillard, J., Wilson, B.: Communication and effect. Commun Res 20, 637–646 (1993)

    Article  Google Scholar 

  11. van der Walma Molen, J., Klijn, M.: Recall of television versus print news: retesting the semantic overlap hypothesis. J. Broadcast Electron. Media 48, 89–107 (2010)

    Article  Google Scholar 

  12. Lang, A., Dhillon, K., Dong, Q.: Effects of emotional arousal and valence on television viewers’ cognitive capacity and memory. J. Broadcast Electron. Media 39(3), 313–327 (1995)

    Article  Google Scholar 

  13. Figner, B., Murphy, R.: Using skin conductance in judgment and decision making research. In: Schulte-Mecklenbeck, M., Kuehberger, A., Ranyard, R. (eds.) A Handbook of Process Tracing Methods for Decision Research, pp. 163–184. Psychology Press, New York (2011)

    Google Scholar 

  14. Stöckli, S., Schulte-Mecklenbeck, M., Borer, S.: Facial expression analysis with AFFDEX and FACET: a validation study. Behav. Res 50, 1446–1460 (2018)

    Article  Google Scholar 

  15. Noldus Information Technology: Reference Manual, Wageningen, 2015

  16. Viola, P., Jones, M.: Robust real-time face detection. Int. J. Comput. Vision 57(2), 137–154 (2004)

    Article  Google Scholar 

  17. Cootes T., Taylor, C.: Statistical models of appearance for computer vision. Technical report, University of Manchester, Wolfson Image Analysis Unit, Imaging Science and Biomedical Engineering (2000)

  18. Bishop, C.: Neural Networks for Pattern Recognition. Clarendon Press, Oxford (1995)

    MATH  Google Scholar 

  19. Hardie, R., Barnard, K., Armstrong, E.: Joint map registration and high-resolution image estimation using a sequence of undersampled images. IEEE Trans. Image Process 6(12), 1621–1633 (1997)

    Article  Google Scholar 

  20. Tipping, M., Bishop, C.: Bayesian image super-resolution. Conf. Proc. Adv. Neural Inf. Syst 16, 1303–1310 (2003)

    Google Scholar 

  21. Sun, J., Xu, Z., Shum, H.: Image super-resolution using gradient profile prior. Conf Proc IEEE Comput. Vis. Pattern Recognit. pp, 1–8 (2008)

  22. Chang, H., Yeung, D., Xiong, Y.: Super-resolution through neighbor embedding. Conf Proc IEEE Comput. Vis. Pattern Class. 1, pp. 275–282 (2004)

  23. Baker, S., Kanade, T.: Hallucinating faces, Conf Proc IEEE Int. Autom. Face Gesture Recognit. pp. 83–88 (2000)

  24. Wu, J., Anisetti, M., Wu, W., Damiani, E., Jeon, G.: Bayer demosaicking with polynomial interpolation. IEEE Trans. Image Process. 25(11), 5369–5382 (2016)

    Article  MathSciNet  Google Scholar 

  25. Paul, A.: Real-time power management for embedded M2M using intelligent learning methods. ACM Trans. Embedded Comput. Syst. 13(5), 148 (2014)

    Google Scholar 

  26. Paul, A., Rho, S., Bharanitharan, K.: Interactive scheduling for mobile multimedia service in M2M environment. Multimed. Tools Appl. 71(1), 235–246 (2014)

    Article  Google Scholar 

  27. McCloud, S.: Making Comics: Storytelling Secrets of Comics, Manga and Graphic Novels. HarperCollins, New York (2006)

    Google Scholar 

Download references

Acknowledgements

This work was supported by the Incheon National University Research Grant in 2020.

Author information

Authors and Affiliations

  1. Department of Korean Language Education, College of Education, Incheon National University, 12 Gaetbeol-ro, Yeonsu-gu, Incheon, Republic of Korea

    Pyoung Won Kim

Authors
  1. Pyoung Won Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pyoung Won Kim.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, P.W. Image super-resolution model using an improved deep learning-based facial expression analysis. Multimedia Systems 27, 615–625 (2021). https://doi.org/10.1007/s00530-020-00705-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-020-00705-1

Keywords

Search

Navigation