Skip to main content
SpringerLink
Log in

Pain detection from facial expressions using domain adaptation technique

  • Original Article
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Pain management is gaining the attention of clinical practitioners to relieve patients from pain in an effective manner. Pain management is primarily dependent on pain measurement. Researchers have proposed various techniques to measure pain from facial expressions improving the accuracy and efficiency of the traditional pain measurement such as self-reporting and visual analog scale. Developments in the field of deep learning have further enhanced the pain assessment technique. Despite of the state of the art performance of deep learning algorithms, adaptation to new subjects is still a problem due to availability of a few samples of the same. Authors have addressed this issue by employing a model agnostic meta-learning algorithm for pain detection and fast adaptation of the trained algorithm for new subjects using only a few labeled images. The model is pre-trained with labeled images of subjects with five pain levels to acquire meta-knowledge in the presented work. This meta-knowledge is then used to adapt the model to a new learning task in the form of a new subject. The proposed model is evaluated on a benchmark dataset, i.e., UNBC McMaster pain archive database. Experimental results show that the model can be very easily adapted to new subjects with the accuracy of \(96\%\) and \(98\%\) for 1-shot and 5-shot learning respectively, proving the potential of the proposed algorithm for clinical use.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Antoniou A, Edwards H, Storkey A (2018) How to train your maml. arXiv preprint arXiv:1810.09502

  2. Bargshady G, Soar J, Zhou X, Deo RC, Whittaker F, Wang H (2019) A joint deep neural network model for pain recognition from face. In: 2019 IEEE 4th International Conference on Computer and Communication Systems (ICCCS). pp. 52–56

  3. Bargshady G, Zhou X, Deo RC, Soar J, Whittaker F, Wang H (2020) Enhanced deep learning algorithm development to detect pain intensity from facial expression images. Expert Systems with Applications 149:113305

  4. Bargshady G, Zhou X, Deo RC, Soar J, Whittaker F, Wang H (2020) Ensemble neural network approach detecting pain intensity from facial expressions. Artificial Intelligence in Medicine 109:101954

  5. Brahnam S, Chuang CF, Shih FY, Slack MR (2006) Machine recognition and representation of neonatal facial displays of acute pain. Artificial Intelligence in Medicine 36(3):211–222

    Article  Google Scholar 

  6. Egede J, Valstar M, Martinez B (2017) Fusing deep learned and hand-crafted features of appearance, shape, and dynamics for automatic pain estimation. In: 2017 12th IEEE international conference on automatic face & gesture recognition (FG 2017). pp. 689–696. IEEE

  7. Ekman P, Friesen WV (1969) Nonverbal leakage and clues to deception. Psychiatry 32(1):88–106

    Article  Google Scholar 

  8. Ekman P, Friesen WV (1976) Measuring facial movement. Environmental psychology and nonverbal behavior 1(1):56–75

    Article  Google Scholar 

  9. Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: Proceedings of the 34th International Conference on Machine Learning-Volume 70. pp. 1126–1135. JMLR. org

  10. Hammal Z, Cohn JF (2012) Automatic detection of pain intensity. In: Proceedings of the 14th ACM international conference on Multimodal interaction. pp. 47–52. ACM

  11. Hammal Z, Cohn JF (2012) Automatic detection of pain intensity. In: Proceedings of the 14th ACM international conference on Multimodal interaction. pp. 47–52

  12. Hammal Z, Cohn JF (2018) Automatic, objective, and efficient measurement of pain using automated face analysis. In: Social and Interpersonal Dynamics in Pain, pp. 121–146. Springer

  13. Kaltwang S, Rudovic O, Pantic M (2012) Continuous pain intensity estimation from facial expressions. In: Advances in Visual Computing, pp. 368–377. Springer

  14. Lesage FX, Berjot S, Deschamps F (2012) Clinical stress assessment using a visual analogue scale. Occupational medicine 62(8):600–605

    Article  Google Scholar 

  15. Littlewort GC, Bartlett MS, Lee K (2009) Automatic coding of facial expressions displayed during posed and genuine pain. Image and Vision Computing 27(12):1797–1803

    Article  Google Scholar 

  16. Lopez-Martinez D, Picard R (2017) Multi-task neural networks for personalized pain recognition from physiological signals. In: 2017 Seventh International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW). pp. 181–184

  17. Lopez-Martinez D, Peng K, Lee A, Borsook D, Picard R (2019) Pain detection with fnirs-measured brain signals: a personalized machine learning approach using the wavelet transform and bayesian hierarchical modeling with dirichlet process priors. In: 2019 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW). pp. 304–309. IEEE

  18. Lopez-Martinez D, Peng K, Steele SC, Lee AJ, Borsook D, Picard R (2018) Multi-task multiple kernel machines for personalized pain recognition from functional near-infrared spectroscopy brain signals. In: 2018 24th International Conference on Pattern Recognition (ICPR). pp. 2320–2325. IEEE

  19. Lopez-Martinez D, Rudovic O, Picard R (2017) Physiological and behavioral profiling for nociceptive pain estimation using personalized multitask learning. Neural Information Processing Systems (NIPS) Workshop on Machine Learning for Health

  20. Lopez Martinez, Ognjen (Oggi) Rudovic, Rosalind Picard, D (2017) Personalized automatic estimation of self-reported pain intensity from facial expressions. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops

  21. Lucey P, Cohn JF, Prkachin KM, Solomon PE, Chew S, Matthews I (2012) Painful monitoring: Automatic pain monitoring using the unbc-mcmaster shoulder pain expression archive database. Image and Vision Computing 30(3):197–205

    Article  Google Scholar 

  22. Lucey S, Ashraf AB, Cohn JF (2007) Investigating Spontaneous Facial Action Recognition through AAM Representations of the Face pp. 275–286

  23. Moon JD (2015) Improving health management through clinical decision support systems. IGI Global

  24. Nelson R (2003) Decade of pain control and research gets into gear in usa. The Lancet 362(9390):1129

    Article  Google Scholar 

  25. Prkachin KM, Solomon PE (2008) The structure, reliability and validity of pain expression: Evidence from patients with shoulder pain. Pain 139(2):267–274

    Article  Google Scholar 

  26. Rathee N, Ganotra D (2015) A novel approach for pain intensity detection based on facial feature deformations. Journal of Visual Communication and Image Representation 33:247–254

    Article  Google Scholar 

  27. Rodriguez P, Cucurull G, Gonzàlez J, Gonfaus JM, Nasrollahi K, Moeslund TB, Roca FX (2017) Deep pain: Exploiting long short-term memory networks for facial expression classification. IEEE transactions on cybernetics

  28. Semwal A, Londhe ND (2018) Automated pain severity detection using convolutional neural network. In: 2018 International Conference on Computational Techniques, Electronics and Mechanical Systems (CTEMS). pp. 66–70

  29. Tavakolian M, Hadid A (2019) A spatiotemporal convolutional neural network for automatic pain intensity estimation from facial dynamics. International Journal of Computer Vision pp. 1–13

  30. Thevenot J, López MB, Hadid A (2017) A survey on computer vision for assistive medical diagnosis from faces. IEEE journal of biomedical and health informatics 22(5):1497–1511

    Article  Google Scholar 

  31. Vuorio R, Sun SH, Hu H, Lim JJ (2019) Multimodal model-agnostic meta-learning via task-aware modulation. arXiv preprint arXiv:1910.13616

  32. Walter S, Gruss S, Ehleiter H, Tan J, Traue HC, Werner P, Al-Hamadi A, Crawcour S, Andrade AO, da Silva GM (2013) The biovid heat pain database data for the advancement and systematic validation of an automated pain recognition system. In: 2013 IEEE international conference on cybernetics (CYBCO). pp. 128–131. IEEE

  33. Werner P, Al-Hamadi A, Limbrecht-Ecklundt K, Walter S, Gruss S, Traue HC (2016) Automatic pain assessment with facial activity descriptors. IEEE Transactions on Affective Computing 8(3):286–299

    Article  Google Scholar 

  34. Werner P, Lopez-Martinez D, Walter S, Al-Hamadi A, Gruss S, Picard R (2019) Automatic recognition methods supporting pain assessment: A survey. IEEE Transactions on Affective Computing

  35. Zhou J, Hong X, Su F, Zhao G (2016) Recurrent convolutional neural network regression for continuous pain intensity estimation in video. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. pp. 84–92

Download references

Author information

Authors and Affiliations

  1. Maharaja Surajmal Institute of Technology, affiliated to Guru Gobind Singh IndraPrastha University, C-4, Janakpuri, New Delhi, India

    Neeru Rathee & Sudesh Pahal

  2. DCRUST, Murthal, India

    Poonam Sheoran

Authors
  1. Neeru Rathee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sudesh Pahal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Poonam Sheoran
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sudesh Pahal.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rathee, N., Pahal, S. & Sheoran, P. Pain detection from facial expressions using domain adaptation technique. Pattern Anal Applic 25, 567–574 (2022). https://doi.org/10.1007/s10044-021-01025-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-021-01025-4

Keywords

Search

Navigation