Automatic Pain Intensity Estimation with Heteroscedastic Conditional Ordinal Random Fields

  • Ognjen Rudovic
  • Vladimir Pavlovic
  • Maja Pantic
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8034)


Automatic pain intensity estimation from facial images is challenging mainly because of high variability in subject-specific pain expressiveness. This heterogeneity in the subjects causes their facial appearance to vary significantly when experiencing the same pain level. The standard classification methods (e.g., SVMs) do not provide a principled way of accounting for this heterogeneity. To this end, we propose the heteroscedastic Conditional Ordinal Random Field (CORF) model for automatic estimation of pain intensity. This model generalizes the CORF framework for modeling sequences of ordinal variables, by adapting it for heteroscedasticity. This is attained by allowing the variance in the ordinal probit model in the CORF to change depending on the input features, resulting in the model able to adapt to the pain expressiveness level specific to each subject. Our experimental results on the UNBC Shoulder Pain Database show that modeling heterogeneity in the subjects with the framework of CORFs improves the pain intensity estimation attained by the standard CORF model, and the other commonly used classification models.


Pain Intensity Local Binary Pattern Conditional Random Field Ordinal Regression Facial Appearance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lucey, P., Cohn, J., Prkachin, K., Solomon, P., Chew, S., Matthews, I.: Image and Vision Computing (42), 197–205Google Scholar
  2. 2.
    Lucey, P., Cohn, J., Prkachin, K., Solomon, P., Matthews, I.: Painful data: The UNBC-McMaster shoulder pain expression archive database. In: FG, pp. 57–64. IEEE (2011)Google Scholar
  3. 3.
    Prkachin, K., Solomon, P.: The structure, reliability and validity of pain expression: Evidence from patients with shoulder pain. Pain 139, 267–274 (2008)CrossRefGoogle Scholar
  4. 4.
    Kaltwang, S., Rudovic, O., Pantic, M.: Continuous pain intensity estimation from facial expressions. In: Bebis, G., et al. (eds.) ISVC 2012, Part II. LNCS, vol. 7432, pp. 368–377. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  5. 5.
    Hammal, Z., Cohn, J.F.: Automatic detection of pain intensity. In: Proceedings of the 14th ACM International Conference on Multimodal Interaction, ICMI 2012, pp. 47–52. ACM (2012)Google Scholar
  6. 6.
    Kim, M., Pavlovic, V.: Structured output ordinal regression for dynamic facial emotion intensity prediction. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part III. LNCS, vol. 6313, pp. 649–662. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  7. 7.
    Rudovic, O., Pavlovic, V., Pantic, M.: Kernel conditional ordinal random fields for temporal segmentation of facial action units. In: Fusiello, A., Murino, V., Cucchiara, R. (eds.) ECCV 2012 Ws/Demos, Part II. LNCS, vol. 7584, pp. 260–269. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  8. 8.
    Kanamori, T.: Statistical models and learning algorithms for ordinal regression problems. Information Fusion 14, 199–207 (2013)CrossRefGoogle Scholar
  9. 9.
    McCullagh, P.: Regression models for ordinal data. Journal of the Royal Statistical Society. Series B (42), 109–142Google Scholar
  10. 10.
    Lafferty, J., McCallum, A., Pereira, F.: Conditional Random Fields: Probabilistic models for segmenting and labeling sequence data. In: ICML, pp. 282–289 (2001)Google Scholar
  11. 11.
    Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24, 971–987 (2002)CrossRefGoogle Scholar
  12. 12.
    Rudovic, O., Pavlovic, V., Pantic, M.: Multi-output laplacian dynamic ordinal regression for facial expression recognition and intensity estimation. In: CVPR (2012) (in press)Google Scholar
  13. 13.
    Barla, A., Odone, F., Verri, A.: Histogram intersection kernel for image classification. In: ICIP 2003, vol. 3,2, pp. III-513–III-516 (2003)Google Scholar
  14. 14.
    Chang, C.C., Lin, C.J.: LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2, 27:1– 27:27 (2011)Google Scholar
  15. 15.
    Chu, W., Keerthi, S.S.: New approaches to support vector ordinal regression. In: ICML, pp. 145–152 (2005)Google Scholar
  16. 16.
    Chu, W., Ghahramani, Z.: Gaussian processes for ordinal regression. JMLR 6, 1019–1041 (2005)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Murphy, K.P.: The bayes net toolbox for matlab. Computing Science and Statistics 33, 2001 (2001)Google Scholar
  18. 18.
    Lafferty, J.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data, pp. 282–289. Morgan Kaufmann (2001)Google Scholar
  19. 19.
    He, X., Niyogi, P.: Locality Preserving Projections. In: NIPS (2004)Google Scholar
  20. 20.
    Shrout, P., Fleiss, J.: Intraclass correlations: uses in assessing rater reliability. Psychology Bulletin (1979)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Ognjen Rudovic
    • 1
  • Vladimir Pavlovic
    • 2
  • Maja Pantic
    • 1
    • 3
  1. 1.Comp. Dept.Imperial College LondonUK
  2. 2.Dept. of Computer ScienceRutgers UniversityUSA
  3. 3.EEMCSUniversity of TwenteThe Netherlands

Personalised recommendations