Abstract
Pain experience, when intense or repetitive, may harm the development of newborns. Several clinical and non-clinical studies have been carried out to identify the presence of pain through behavioural analysis, mainly by facial mimicry. Advances in deep learning might show automatic, continuous and non-invasive solutions for neonatal pain assessment as well. In this context, this work investigates the following five state-of-the-art Convolutional Neural Networks (CNNs) for the classification of pain using two distinct face image datasets (UNIFESP and iCOPE): VGG-16, ResNet50, SENet50, and Inception-V3, all implemented with transfer learning, and the specific one called Neonatal CNN, which was trained end-to-end. Our experimental results, based on quantitative and qualitative analyses, indicate the superiority of models originally trained with face images, highlighting most relevant differences owing to the explainable information extracted by each model and the current issue of limited neonatal face images available.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
We used the pre-trained models available at [18].
References
IASP. “IASP Publication, Pain terms: a list with definitions and notes on usage”. In: Pain (1979).
Luda Diatchenko et al. “Genetic architecture of human pain perception”. In: TRENDS in Genetics 23.12 (2007), pp. 605–613.
Luda Diatchenko et al. “Idiopathic pain disorders-pathways of vulnerability”. In: Pain 123.3 (2006), pp. 226–230.
Kanwaljeet JS Anand, Paul R Hickey, et al. “Pain and its effects in the human neonate and fetus”. In: N Engl j Med 317.21 (1987), pp. 1321–1329.
Kanwaljeet JS Anand and David B Carr. “The neuroanatomy, neurophysiology, and neurochemistry of pain, stress, and analgesia in newborns and children”. In: Pediatric Clinics of North America 36.4 (1989), pp. 795–822.
Ruth VE Grunau and Kenneth D Craig. “Pain expression in neonates: facial action and cry”. In: Pain 28.3 (1987), pp. 395–410.
Ruth Guinsburg. “Avaliação e tratamento da dor no recém-nascido”. In: J Pediatr (Rio J) 75.3 (1999), pp. 149–60.
Fernanda G. Tamanaka et al. “Neonatal pain assessment: A Kendall analysis between clinical and visually perceived facial features”. In: Computer Methods in Biomechanics and Biomedical Engineering: Imaging and Visualization 0.0 (2022), pp. 1–10. https://doi.org/10.1080/21681163.2022.2044909.
Sheryl Brahnam et al. “Machine recognition and representation of neonatal facial displays of acute pain”. In: Artificial intelligence in medicine 36.3 (2006), pp. 211–222.
Lucas F. Buzuti et al. “Neonatal pain assessment from facial expression using Deep Neural Networks”. In: Anais do XVI Workshop de Visão Computacional (2020), pp. 87–92.
Lucas P. Carlini et al. “A Convolutional Neural Network-based Mobile Application to Bedside Neonatal Pain Assessment”. In: 2021 34th SIB-GRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). 2021, pp. 394–401. https://doi.org/10.1109/SIBGRAPI54419.2021.00060.
Tatiany Marcondes Heiderich, Ana Teresa Figueiredo Stochero Leslie, and Ruth Guinsburg. “Neonatal procedural pain can be assessed by computer software that has good sensitivity and specificity to detect facial movements”. In: Acta Paediatrica 104.2 (2015), e63–e69.
Ghada Zamzmi et al. “Neonatal pain expression recognition using transfer learning”. In: arXiv preprint arXiv:1807.01631 (2018).
Ghada Zamzmi et al. “Pain assessment from facial expression: Neonatal convolutional neural network (N-CNN)”. In: 2019 International Joint Conference on Neural Networks (IJCNN). IEEE. 2019, pp. 1–7.
Ramprasaath R Selvaraju et al. “Grad-cam: Visual explanations from deep networks via gradient-based localization”. In: Proceedings of the IEEE international conference on computer vision. 2017, pp. 618–626.
Omkar M. Parkhi, Andrea Vedaldi, and Andrew Zisserman. “Deep Face Recognition”. In: British Machine Vision Conference. 2015.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. “Deep inside convolutional networks: Visualising image classification models and saliency maps”. In: arXiv preprint arXiv:1312.6034 (2013).
Refik Can Malli. keras-vggface: VGGFace implementation with Keras Frame-work. [Online; accessed 20-March-2022]. 2016. https://github.com/rcmalli/keras-vggface
Qiong Cao et al. “Vggface2: A dataset for recognising faces across pose and age”. In: 2018 13th IEEE international conference on automatic face and gesture recognition (FG 2018). IEEE. 2018, pp. 67–74.
Kaiming He et al. “Deep residual learning for image recognition”. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016, pp. 770–778.
Jie Hu, Li Shen, and Gang Sun. “Squeeze-and-excitation networks”. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2018, pp. 7132–7141.
Christian Szegedy et al. “Rethinking the inception architecture for computer vision”. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016, pp. 2818–2826.
Geoffrey Hinton, Nitish Srivastava, and Kevin Swersky. Overview of minibatch gradient descent. [Online; accessed 23-July-2020]. 2012. http://www.cs.toronto.edu/?tijmen/csc321/slides/lectureslideslec6.pdf.
Bas HM van der Velden et al. “Explainable artificial intelligence (XAI) in deep learning-based medical image analysis”. In: arXiv preprint arXiv:2107.10912 (2021).
Lucas Pereira Carlini et al. “A Visual Perception Framework to Analyse Neonatal Pain in Face Images”. In: Image Analysis and Recognition. Proceedings of the 17th International Conference on Image Analysis and Recognition, ICIAR 2020. Ed. by Aurélio Campilho, Fakhri Karray, and Zhou Wang. Vol. 12131. Lecture Notes in Computer Science. Cham: Springer International Publishing, 2020, pp. 233–243. ISBN: 978-3-030-50347-5.
Acknowledgements
The authors would like to thank the financial support provided by the Brazilian funding agencies CNPq (401059/2019-7), FAPESP (2018/13076-9) and CAPES.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Coutrin, G.A.S. et al. (2023). Convolutional Neural Networks for Newborn Pain Assessment Using Face Images: A Quantitative and Qualitative Comparison. In: Su, R., Zhang, Y., Liu, H., F Frangi, A. (eds) Medical Imaging and Computer-Aided Diagnosis. MICAD 2022. Lecture Notes in Electrical Engineering, vol 810. Springer, Singapore. https://doi.org/10.1007/978-981-16-6775-6_41
Download citation
DOI: https://doi.org/10.1007/978-981-16-6775-6_41
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-6774-9
Online ISBN: 978-981-16-6775-6
eBook Packages: MedicineMedicine (R0)