Constrained Maximum Likelihood Learning of Bayesian Networks for Facial Action Recognition

  • Cassio P. de Campos
  • Yan Tong
  • Qiang Ji
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5304)


Probabilistic graphical models such as Bayesian Networks have been increasingly applied to many computer vision problems. Accuracy of inferences in such models depends on the quality of network parameters. Learning reliable parameters of Bayesian networks often requires a large amount of training data, which may be hard to acquire and may contain missing values. On the other hand, qualitative knowledge is available in many computer vision applications, and incorporating such knowledge can improve the accuracy of parameter learning. This paper describes a general framework based on convex optimization to incorporate constraints on parameters with training data to perform Bayesian network parameter estimation. For complete data, a global optimum solution to maximum likelihood estimation is obtained in polynomial time, while for incomplete data, a modified expectation-maximization method is proposed. This framework is applied to real image data from a facial action unit recognition problem and produces results that are similar to those of state-of-the-art methods.


Bayesian Network Facial Expression Recognition Parameter Learning Probabilistic Graphical Model Computer Vision Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ji, Q., Luo, J., Metaxas, D., Torralba, A., Huang, T., Sudderth, E. (eds.): Special Issue on Probabilistic Graphical Models in Computer Vision, IEEE Transactions on Pattern Analysis and Machine Intelligence (2008),
  2. 2.
    Triggs, B., Williams, C. (eds.): Special Issue on Probabilistic Models for Image Understanding. International Journal of Computer Vision (2008),
  3. 3.
    Delage, E., Lee, H., Ng, A.: A dynamic bayesian network model for autonomous 3d reconstruction from a single indoor image. In: Proc. of the IEEE International Conference on Computer Vision and Pattern Recognition (2006)Google Scholar
  4. 4.
    Zhou, Y., Huang, T.S.: Weighted Bayesian network for visual tracking. In: Proc. of the International Conference on Pattern Recognition (2006)Google Scholar
  5. 5.
    Mortensen, E., Jia, J.: Real-time semi-automatic segmentation using a Bayesian network. In: Proc. of the IEEE International Conference on Computer Vision and Pattern Recognition (2006)Google Scholar
  6. 6.
    Zhang, Y., Ji, Q.: Active and dynamic information fusion for facial expression understanding from image sequence. IEEE Trans. on Pattern Analysis and Machine Intelligence 27(5), 699–714 (2005)CrossRefGoogle Scholar
  7. 7.
    Tong, Y., Liao, W., Ji, Q.: Facial action unit recognition by exploiting their dynamic and semantic relationships. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1683–1699 (2007)Google Scholar
  8. 8.
    Dempster, A., Laird, N., Rubin, D.: Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B 39(1), 1–38 (1977)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Niculescu, R.S.: Exploiting Parameter Domain Knowledge for Learning in Bayesian Networks. PhD thesis, Carnegie Mellon (2005) CMU-CS-05-147Google Scholar
  10. 10.
    Wellman, M.P.: Fundamental concepts of qualitative probabilistic networks. Artificial Intelligence 44, 257–303 (1990)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Wellman, M.P., Henrion, M.: Explaining explaining away. IEEE Transactions on Pattern Analysis and Machine Intelligence 15, 287–307 (1993)CrossRefzbMATHGoogle Scholar
  12. 12.
    van der Gaag, L.C., Bodlaender, H.L., Feelders, A.: Monotonicity in Bayesian networks. In: UAI, pp. 569–576. AUAI Press (2004)Google Scholar
  13. 13.
    Bolt, J.H., van der Gaag, L.C., Renooij, S.: Introducing situational influences in QPNs. In: Nielsen, T.D., Zhang, N.L. (eds.) ECSQARU 2003. LNCS (LNAI), vol. 2711, pp. 113–124. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  14. 14.
    Renooij, S., van der Gaag, L.C.: Enhancing QPNs for trade-off resolution. In: UAI, pp. 559–566 (1999)Google Scholar
  15. 15.
    Pantic, M., Bartlett, M.: Machine analysis of facial expressions, pp. 377–416. I-Tech Education and Publishing, Vienna, Austria (2007)Google Scholar
  16. 16.
    Ekman, P., Friesen, W.V.: Facial action coding system: A technique for the measurement of facial movement. Consulting Psychologists Press, Palo Alto (1978)Google Scholar
  17. 17.
    Bartlett, M.S., Littlewort, G.C., Frank, M.G., Lainscsek, C., Fasel, I., Movellan, J.R.: Automatic Recognition of Facial Actions in Spontaneous Expressions. Journal of Multimedia 1(6), 22–35 (2006)CrossRefGoogle Scholar
  18. 18.
    Pantic, M., Rothkrantz, L.J.M.: Automatic analysis of facial expressions: The state of the art. IEEE Trans. Pattern Analysis and Machine Intelligence 22, 1424–1445 (2000)CrossRefGoogle Scholar
  19. 19.
    Tian, Y., Kanade, T., Cohn, J.: Facial expression analysis. Springer, Heidelberg (2004)Google Scholar
  20. 20.
    Jordan, M. (ed.): Learning Graphical Models. The MIT Press, Cambridge (1998)Google Scholar
  21. 21.
    Wittig, F., Jameson, A.: Exploiting qualitative knowledge in the learning of conditional probabilities of Bayesian networks. In: UAI, pp. 644–652 (2000)Google Scholar
  22. 22.
    Altendorf, E., Restificar, A.C., Dietterich, T.G.: Learning from sparse data by exploiting monotonicity constraints. In: UAI, pp. 18–26 (2005)Google Scholar
  23. 23.
    Feelders, A., van der Gaag, L.C.: Learning Bayesian network parameters under order constraints. International Journal of Approximate Reasoning 42(1-2), 37–53 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Niculescu, R.S., Mitchell, T., Rao, B.: Bayesian network learning with parameter constraints. Journal of Machine Learning Research 7(Jul), 1357–1383 (2006)MathSciNetzbMATHGoogle Scholar
  25. 25.
    de Campos, C.P., Cozman, F.G.: Belief updating and learning in semi-qualitative probabilistic networks. In: UAI, pp. 153–160 (2005)Google Scholar
  26. 26.
    Renooij, S., van der Gaag, L.C.: Exploiting non-monotonic influences in qualitative belief networks. In: IPMU, Madrid, Spain, pp. 1285–1290 (2000)Google Scholar
  27. 27.
    Renooij, S., van der Gaag, L.C., Parsons, S.: Context-specific sign-propagation in qualitative probabilistic networks. Artificial Intelligence 140, 207–230 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. MPS/SIAM Series on Optimization. SIAM (2001)Google Scholar
  29. 29.
    Andersen, E.D., Jensen, B., Sandvik, R., Worsoe, U.: The improvements in mosek version 5. Technical report, Mosek Aps (2007)Google Scholar
  30. 30.
    Murtagh, B.A., Saunders, M.A.: Minos 5.4 user’s guide. Technical report, Systems Optimization Laboratory, Stanford University (1995)Google Scholar
  31. 31.
    Wu, C.F.J.: On the convergence properties of the EM algorithm. The Annals of Statistics 11(1), 95–103 (1983)MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Bartlett, M.S., Littlewort, G., Frank, M.G., Lainscsek, C., Fasel, I., Movellan, J.R.: Recognizing facial expression: Machine learning and application to spontaneous behavior. In: Proc. IEEE Int’l Conf. Computer Vision and Pattern Recognition, 2nd edn., pp. 568–573 (2005)Google Scholar
  33. 33.
    Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: IEEE International Conference on Automatic Face and Gesture Recognition, pp. 46–53 (2000)Google Scholar
  34. 34.
    Valstar, M.F., Patras, I., Pantic, M.: Facial action unit detection using probabilistic actively learned support vector machines on tracked facial point data. In: Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition, Workshop Vision for Human-Computer Interaction (2005)Google Scholar
  35. 35.
    Tian, Y., Kanade, T., Cohn, J.: Recognizing action units for facial expression analysis. IEEE Trans. Pattern Analysis and Machine Intelligence 23(2), 97–115 (2001)CrossRefGoogle Scholar
  36. 36.
    Douglas-Cowie, E., Cowie, R., Schroeder, M.: The description of naturally occurring emotional speech. In: Int’l Congress of Phonetic Sciences (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Cassio P. de Campos
    • 1
  • Yan Tong
    • 2
  • Qiang Ji
    • 1
  1. 1.Electrical, Computer and Systems Eng. Dept. Rensselaer Polytechnic InstituteTroy, NYUSA
  2. 2.Visualization and Computer Vision Lab, GE Global Research CenterNiskayuna, NYUSA

Personalised recommendations