Skip to main content
SpringerLink
Log in

Pain Classification and Intensity Estimation Through the Analysis of Facial Action Units

  • Conference paper
  • First Online:
Image Analysis and Processing - ICIAP 2023 Workshops (ICIAP 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14365))

Included in the following conference series:

  • 211 Accesses

Abstract

This study focuses on using facial expressions to evaluate acute pain levels. We analyse videos by relying on an extended set of 17 Action Units (AUs) and head pose components. Multiple models are trained and compared to detect the presence of pain and classify its intensity on a 5-point scale, ranging from no pain to high pain. Validation studies were conducted on two publicly available datasets, evaluating both in within- and cross-dataset conditions. The experimental results show better pain classification performance when using both the extended AU set, instead of the restricted AU set related to pain expressions, and head pose information.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Emotiva - Emotion AI Company, www.emotiva.it.

References

  1. Bartlett, M.S., Littlewort, G.C., Frank, M.G., Lee, K.: Automatic decoding of facial movements reveals deceptive pain expressions. Curr. Biol. 24(7), 738–743 (2014)

    Article  Google Scholar 

  2. Boccignone, G., Conte, D., Cuculo, V., D’Amelio, A., Grossi, G., Lanzarotti, R.: An open framework for remote-PPG methods and their assessment. IEEE Access 8, 216083–216103 (2020)

    Article  Google Scholar 

  3. Bursic, S., Boccignone, G., Ferrara, A., D’Amelio, A., Lanzarotti, R.: Improving the accuracy of automatic facial expression recognition in speaking subjects with deep learning. Appl. Sci. 10(11), 4002 (2020)

    Article  Google Scholar 

  4. Chen, Z.S.: Hierarchical predictive coding in distributed pain circuits. Front. Neural Circ. 17, 1073537 (2023)

    Google Scholar 

  5. Craig, K.D.: The facial expression of pain better than a thousand words? APS J. 1(3), 153–162 (1992)

    Article  Google Scholar 

  6. Craig, K.D., MacKenzie, N.E.: What is pain: are cognitive and social features core components? Paediatr. Neonatal Pain 3(3), 106–118 (2021)

    Article  Google Scholar 

  7. Craig, K.D., Prkachin, K.M., Grunau, R.V.: The Facial Expression of Pain. The Guilford Press (1992)

    Google Scholar 

  8. Das, P., Bhattacharyya, J., Sen, K., Pal, S.: Assessment of pain using optimized feature set from corrugator EMG. In: 2020 IEEE Applied Signal Processing Conference (ASPCON), pp. 349–353 (2020)

    Google Scholar 

  9. Ekman, P., Friesen, W.V.: Facial action coding system (1978)

    Google Scholar 

  10. Fernandes-Magalhaes, R., et al.: Pain E-motion Faces Database (PEMF): pain-related micro-clips for emotion research. Behav. Res. Methods, 1–14 (2022)

    Google Scholar 

  11. Gkikas, S.: Biovid holdouteval (2023). https://www.nit.ovgu.de/nit_media/Bilder/ Dokumente/BIOVID_Dokumente/BioVid_HoldOutEval_Proposal.pdf

  12. Hammal, Z., Cohn, J.F.: Automatic detection of pain intensity. In: Proceedings of the 14th ACM International Conference on Multimodal Interaction, pp. 47–52 (2012)

    Google Scholar 

  13. Kunz, M., Lautenbacher, S.: The faces of pain: a cluster analysis of individual differences in facial activity patterns of pain. Eur. J. Pain 18(6), 813–823 (2014)

    Article  Google Scholar 

  14. Lucey, P., et al.: Automatically detecting pain in video through facial action units. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 41(3), 664–674 (2010)

    Google Scholar 

  15. Lucey, P., Howlett, J., Cohn, J., Lucey, S., Sridharan, S., Ambadar, Z.: Improving pain recognition through better utilisation of temporal information (2008)

    Google Scholar 

  16. Othman, E., Werner, P., Saxen, F., Al-Hamadi, A., Walter, S.: Cross-database evaluation of pain recognition from facial video. In: 2019 11th International Symposium on Image and Signal Processing and Analysis (ISPA), pp. 181–186 (2019)

    Google Scholar 

  17. Prajod, P., Huber, T., André, E.: Using explainable AI to identify differences between clinical and experimental pain detection models based on facial expressions. In: Þór Jónsson, B., et al. (eds.) MMM 2022. LNCS, vol. 13141, pp. 311–322. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-98358-1_25

    Chapter  Google Scholar 

  18. Prkachin, K., Solomon, P.: The structure, reliability and validity of pain expression: Evidence from patients with shoulder pain (2008)

    Google Scholar 

  19. Prkachin, K.M.: The consistency of facial expressions of pain: a comparison across modalities, pp. 297–306 (1992)

    Google Scholar 

  20. Nelson, R.: Decade of pain control and research gets into gear in USA (2003)

    Google Scholar 

  21. Walter, S., et al.: The biovid heat pain database data for the advancement and systematic validation of an automated pain recognition system. In: 2013 IEEE International Conference on Cybernetics (CYBCO), pp. 128–131 (2013)

    Google Scholar 

  22. Werner, P., Al-Hamadi, A., Limbrecht-Ecklundt, K., Walter, S., Gruss, S., Traue, H.C.: Automatic pain assessment with facial activity descriptors. IEEE Trans. Affect. Comput. 8(3), 286–299 (2017)

    Article  Google Scholar 

  23. Werner, P., Al-Hamadi, A., Limbrecht-Ecklundt, K., Walter, S., Gruss, S., Traue, H.C.: Automatic pain assessment with facial activity descriptors. IEEE Trans. Affect. Comput. 8(3), 286–299 (2017)

    Article  Google Scholar 

  24. Werner, P., Al-Hamadi, A., Niese, R., Walter, S., Gruss, S., Traue, H.C.: Automatic pain recognition from video and biomedical signals. In: 2014 22nd International Conference on Pattern Recognition, pp. 4582–4587 (2014)

    Google Scholar 

  25. Werner, P., Al-Hamadi, A., Walter, S.: Analysis of facial expressiveness during experimentally induced heat pain. In: 2017 Seventh International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), pp. 176–180 (2017)

    Google Scholar 

  26. Werner, P., Lopez-Martinez, D., Walter, S., Al-Hamadi, A., Gruss, S., Picard, R.W.: Automatic recognition methods supporting pain assessment: a survey. IEEE Trans. Affect. Comput. 13(01), 530–552 (2022)

    Article  Google Scholar 

  27. Williams, A.C.C.: Facial expression of pain: an evolutionary account. Behav. Brain Sci. 25(4), 439–455 (2002)

    Google Scholar 

  28. Williams, A.C.C., Craig, K.D.: Updating the definition of pain. Pain 157(11), 2420–2423 (2016)

    Article  Google Scholar 

  29. Yang, R., et al.: On pain assessment from facial videos using spatio-temporal local descriptors. In: 2016 Sixth International Conference on Image Processing Theory, Tools and Applications (IPTA), pp. 1–6 (2016)

    Google Scholar 

  30. Zhi, R., Liu, M., Zhang, D.: A comprehensive survey on automatic facial action unit analysis. Vis. Comput. 36, 1067–1093 (2020)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Emotiva, Milan, Italy

    Federica Paolì, Andrea D’Eusanio & Federico Cozzi

  2. PHuSe Laboratory-Dipartimento di Informatica, Università degli Studi di Milano, Via Celoria 18, 20133, Milan, Italy

    Sabrina Patania & Giuseppe Boccignone

Authors
  1. Federica Paolì
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrea D’Eusanio
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Federico Cozzi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sabrina Patania
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Giuseppe Boccignone
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Federica Paolì .

Editor information

Editors and Affiliations

  1. University of Udine, Udine, Italy

    Gian Luca Foresti

  2. University of Udine, Udine, Italy

    Andrea Fusiello

  3. University of York, York, UK

    Edwin Hancock

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Paolì, F., D’Eusanio, A., Cozzi, F., Patania, S., Boccignone, G. (2024). Pain Classification and Intensity Estimation Through the Analysis of Facial Action Units. In: Foresti, G.L., Fusiello, A., Hancock, E. (eds) Image Analysis and Processing - ICIAP 2023 Workshops. ICIAP 2023. Lecture Notes in Computer Science, vol 14365. Springer, Cham. https://doi.org/10.1007/978-3-031-51023-6_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-51023-6_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-51022-9

  • Online ISBN: 978-3-031-51023-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation