Neonatal Facial Pain Assessment Combining Hand-Crafted and Deep Features

  • Luigi Celona
  • Luca Manoni
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10590)


In this paper we evaluate the combination of hand-crafted and deep learning-based features for neonatal pain assessment. To this end we consider two hand-crafted descriptors, i.e. Local Binary Patterns (LBP) and Histogram of Oriented Gradients (HOG), and features extracted from two pre-trained Convolutional Neural Networks (CNNs). Experimental results on the publicly available Infant Classification Of Pain Expressions (COPE) database show competitive results compared to previous methods.


Neonatal pain assessment Hand-crafted features Convolutional Neural Networks Transfer learning Features reduction Feature fusion 


  1. 1.
    Bianco, S., Celona, L., Schettini, R.: Robust smile detection using convolutional neural networks. J. Electron. Imaging 25(6), 063002 (2016)CrossRefGoogle Scholar
  2. 2.
    Bianco, S., Mazzini, D., Pau, D.P., Schettini, R.: Local detectors and compact descriptors for visual search: a quantitative comparison. Digital Sig. Process. 44, 1–13 (2015)CrossRefGoogle Scholar
  3. 3.
    Bianco, S., Schettini, R.: Adaptive color constancy using faces. IEEE Trans. Pattern Anal. Mach. Intell. 36(8), 1505–1518 (2014)CrossRefGoogle Scholar
  4. 4.
    Brahnam, S., Chuang, C.F., Sexton, R.S., Shih, F.Y.: Machine assessment of neonatal facial expressions of acute pain. Decis. Support Syst. 43(4), 1242–1254 (2007)CrossRefGoogle Scholar
  5. 5.
    Brahnam, S., Chuang, C.F., Shih, F.Y., Slack, M.R.: Machine recognition and representation of neonatal facial displays of acute pain. Artif. Intell. Med. 36(3), 211–222 (2006)CrossRefGoogle Scholar
  6. 6.
    Brahnam, S., Nanni, L., Sexton, R.: Introduction to neonatal facial pain detection using common and advanced face classification techniques. In: Yoshida, H., Jain, A., Ichalkaranje, A., Jain, L.C., Ichalkaranje, N. (eds.) Advanced Computational Intelligence Paradigms in Healthcare-1, pp. 225–253. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  7. 7.
    Brahnam, S., Nanni, L., Sexton, R.S.: Neonatal facial pain detection using NNSOA and LSVM. In: IPCV, pp. 352–357 (2008)Google Scholar
  8. 8.
    Cusano, C., Napoletano, P., Schettini, R.: Illuminant invariant descriptors for color texture classification. In: Tominaga, S., Schettini, R., Trémeau, A. (eds.) CCIW 2013. LNCS, vol. 7786, pp. 239–249. Springer, Heidelberg (2013). CrossRefGoogle Scholar
  9. 9.
    Cusano, C., Napoletano, P., Schettini, R.: Intensity and color descriptors for texture classification. Proc. SPIE 8661, 866113 (2013)CrossRefGoogle Scholar
  10. 10.
    Cusano, C., Napoletano, P., Schettini, R.: Combining local binary patterns and local color contrast for texture classification under varying illumination. JOSA A 31(7), 1453–1461 (2014)CrossRefGoogle Scholar
  11. 11.
    Cusano, C., Napoletano, P., Schettini, R.: Local angular patterns for color texture classification. In: Murino, V., Puppo, E., Sona, D., Cristani, M., Sansone, C. (eds.) ICIAP 2015. LNCS, vol. 9281, pp. 111–118. Springer, Cham (2015). CrossRefGoogle Scholar
  12. 12.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 886–893. IEEE (2005)Google Scholar
  13. 13.
    Florea, C., Florea, L., Butnaru, R., Bandrabur, A., Vertan, C.: Pain intensity estimation by a self-taught selection of histograms of topographical features. Image Vis. Comput. 56, 13–27 (2016)CrossRefGoogle Scholar
  14. 14.
    King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009)Google Scholar
  15. 15.
    Levi, G., Hassner, T.: Emotion recognition in the wild via convolutional neural networks and mapped binary patterns. In: Proceedings of ACM International Conference on Multimodal Interaction (ICMI), November 2015Google Scholar
  16. 16.
    Lindh, V., Wiklund, U., Håkansson, S.: Heel lancing in term new-born infants: an evaluation of pain by frequency domain analysis of heart rate variability. Pain 80(1), 143–148 (1999)CrossRefGoogle Scholar
  17. 17.
    Mansor, M.N., Rejab, M.N.: A computational model of the infant pain impressions with Gaussian and nearest mean classifier. In: 2013 IEEE International Conference on Control System, Computing and Engineering (ICCSCE), pp. 249–253. IEEE (2013)Google Scholar
  18. 18.
    Nanni, L., Brahnam, S., Lumini, A.: A local approach based on a local binary patterns variant texture descriptor for classifying pain states. Expert Syst. Appl. 37(12), 7888–7894 (2010)CrossRefGoogle Scholar
  19. 19.
    Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)CrossRefzbMATHGoogle Scholar
  20. 20.
    Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: British Machine Vision Conference (2015)Google Scholar
  21. 21.
    Petroni, M., Malowany, A.S., Johnston, C.C., Stevens, B.J.: Identification of pain from infant cry vocalizations using artificial neural networks (ANNs). In: SPIE’s 1995 Symposium on OE/Aerospace Sensing and Dual Use Photonics, pp. 729–738. International Society for Optics and Photonics (1995)Google Scholar
  22. 22.
    Razavian, A.S., Azizpour, H., Sullivan, J., Carlsson, S.: CNN features off-the-shelf: an astounding baseline for recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 806–813 (2014)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Department of Informatics, Systems and CommunicationUniversity of Milano-BicoccaMilanoItaly

Personalised recommendations