Abstract
One of the richest avenues for nonverbal expression of emotion is emotional facial expression (EFE), which reflects inner psychic reality of an individual. It can be developed as a very important diagnostic index for psychiatric disorders. In this chapter, an attempt has been made to provide a systematic review of the following issues—the importance of facial expression as a diagnostic measure in psychiatric disorders, the effectiveness of computational models of facial action coding system (FACS) to aid in diagnosis, and, finally, the usefulness of computational approach on facial expression analysis as a measure of psychiatric diagnosis. The possibility of bringing objectivity in psychiatric diagnosis through computational model of EFEs will be discussed.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Anstadt, T., & Krause, R. (1989). The expression of primary affects in portraits drawn by schizophrenics. Psychiatry, 52, 13–24.
Arguedas, D., Stevenson, R. J., & Langdon, R. (2012). Source monitoring and olfactory hallucinations in schizophrenia. Journal of Abnormal Psychology, 121(4), 936–943.
Barlow, D. H., Chorpita, B., & Turovsky, J. (1996). Fear, panic, anxiety, and disorders of emotion. In D. Hope (Ed.), Perspectives on anxiety, panic and fear (pp. 251–328). 43rd Annual Nebraska Symposium on Motivation. Lincoln: University of Nebraska Press.
Bartlett, M. S., Littlewort, G. C., Frank, M. G., Lainscsek, C., Fasel, I. R., & Movellan, J. R. (2006). Automatic recognition of facial actions in spontaneous expressions. Journal of multimedia, 1(6), 22–35.
Bell, C. (Ed.) (1844) Anatomy and philosophy of expression as connected with the fine arts (7th Rev. ed.). London: G. Bell & Sons.
Belhumeur, P. N., Hespanha, J. P., & Kriegman, D. J. (1997). Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7), 711–720.
Benedetti, F., Anselmetti, S., Florita, M., Radaelli, D., Cavallaro, R., Colombo, C., et al. (2005). Reality monitoring and paranoia in remitted delusional depression. Clinical Neuropsychiatry, 2(3), 199–205.
Benson, P. J. (1999). A means of measuring facial expressions and a method for predicting emotion categories in clinical disorders of affect. Journal of Affective Disorders, 55(2–3), 179–185.
Black, M. J., & Yacoob, Y. (1995). A tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motions. In Proceedings of International Conference on Computer Vision (pp. 374–381).
Black, M. J., & Yacoob, Y. (1997). A recognizing facial expressions in image sequences using local parameterized models of image motion. International Journal of Computer Vision, 25(1), 23–48.
Borod, J. C., & Koff, E. (1984). Asymmetries in affective expression: Behaviour and anatomy. In N. Fox & R. Davidson (Eds.), The psychobiology of affective development (pp. 293–323). Hillsdale, NJ: Erlbaum.
Borod, J. C., & Koff, E. (1991). Facial asymmetry during posed and spontaneous expressions in stroke patients with unilateral lesions. Pharmacopsychoecologia, 4, 15–21.
Borod, J. C., Martin, C. C., Alpert, M., Brozgold, A., & Welkowitz, J. (1993). Perception of facial emotion in schizophrenics and right brain-damaged patients. Journal of Nervous and Mental Disorders, 181, 494–502.
Borod, J. C., Haywood, C. S., & Koff, E. (1997). Neuropsychological aspects of facial asymmetry during emotional expression: A review of the normal adult literature. Neuropsychology Review, 7, 41–60.
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992). A training algorithm for optimal margin classifiers. In COLT ‘92 Proceedings of the 5th annual Workshop on Computational learning theory at NY, USA (pp. 144–152).
Campbell, R. (1978). Asymmetries in interpreting and expressing a posed facial expression. Cortex, 14, 327–342.
Campbell, R. (1979). Left handers’ smiles: Asymmetries in the projection of a posed expression. Cortex, 14, 571–579.
Cohn, J. F., Zlochower, A. J., Lien, J. J., & Kanade, T. (1998). A feature-point tracking by optical flow discriminates subtle differences in facial expression. In Proceedings of International Conference Automatic Face and Gesture Recognition (pp. 396–401).
Cootes, T. F., Edwards, G. J., & Taylor, C. J. (1998). Active appearance models. In Proceedings of the European Conference on Computer Vision (Vol. 2, pp. 484–498).
Darwin, C. (1872/1998). The expression of the emotions in man and animals. New York: Oxford University Press.
Dimberg, U. (1982). Facial reactions to facial expressions. Psychophysiology, 19, 643–647.
Dimberg, U. (1997). Psychophysiological reactions to facial expressions. In U. Segerstrale & P. Molnar (Eds.), Nonverbal communication: Where nature meets culture (pp. 47–60). Mahwah, NJ: Erlbaum.
Domes, G., Czieschnek, D., Weidle, F., Berger, C., Fast, K., & Herpertz, S. C. (2008). Recognition of facial affect in borderline personality disorder. Journal of Personality Disorders, 22(2), 135–147.
Duchenne de Boulogne, G. B. (1862/1990). The mechanism of human facial expression. New York: Cambridge University Press.
Edwards, G. J., Cootes, T. F., & Taylor, C. J. (1998). A face recognition using active appearance models. In Proceedings of the European Conference on Computer Vision (Vol. 2, pp. 581–695).
Ekman, P. (1982). Emotion in the human face. Cambridge University Press.
Ekman, P. (1992a). Are there basic emotions? Psychological Review, 99, 550–553.
Ekman, P. (1992b). An argument for basic emotions. Cognition and Emotion, 6, 169–200.
Ekman, P., & Friesen, W. V. (1971). Constants across cultures in face and emotion. Journal of Personality and Social Psychology, 17, 124–129.
Ekman, P., & Friesen, W. V. (1975). Unmasking the face. Englewood Cliffs, N.J: Prentice-Hall.
Ekman, P., & Friesen, W. V. (1978). Facial Action Coding System (FACS): Manual. Palo Alto: Consulting Psychologists Press.
Ekman, P., Sorenson, E. R., & Friesen, W. V. (1969). Pancultural elements in facial displays of emotion. Science, 164, 86–88.
Essa, I., & Pentland, A. (1997). A coding, analysis interpretation, recognition of facial expressions. IEEE Trans. Pattern Analysis and Machine Intelligence, 19(7), 757–763.
Fasel, B., & Luettin, J. (2003). Automatic facial expression analysis: A survey. Pattern Recognition, 36(1), 259–275.
Gessler, S., Cutting, J., Frith, C. D., & Weinman, J. (1989). Schizophrenics’ inability to judge facial emotion: A controlled study. British Journal of Clinical Psychology, 28(1), 19–29.
Girard, J. M., Cohn, J. F., Mahoor, M. H., Mavadati, S., & Rosenwald, D. P. (2013). Social risk and depression: Evidence from manual and automatic facial expression analysis. In IEEE International Conference on Automatic Face & Gesture Recognition. Retrieved from http://www.pitt.edu/~jmg174/Girard%20et%20al%202013.pdf.
Gruber, J., Oveis, C., Keltner, D., & Johnson, S. L. (2008). Risk for mania and positive emotional responding: Too much of a good thing? Emotion, 8(1), 23–33.
Hall, J., Harris, J. M., Sprengelmeyer, R., Sprengelmeyer, A., Young, A. W., Santos, I. M., et al. (2004). Social cognition and face processing in schizophrenia. British Journal of Psychiatry, 185, 169–170.
Happy, S. L., George, A., & Routray, A. (2012). A real time facial expression classification system using local binary patterns. In IEEE Proceedings of 4th International Conference on Intelligent Human Computer Interaction, Kharagpur, India, December, 27–29.
Hermans, D., Martens, K., De Cort, K., Pieters, G., & Eelen, P. (2003). Reality monitoring and metacognitive beliefs related to cognitive confidence in obsessive-compulsive disorder. Behaviour Research and Therapy, 41(4), 383–401.
Hong, H., Neven, H., & von der Malsburg, C. (1998). An online facial expression recognition based on personalized galleries. In Proceedings of the. International Conference on Automatic Face and Gesture Recognition (pp. 354–359).
Hooker, C., & Park, S. (2002). Emotion processing and its relationship to social functioning in schizophrenia patients. Psychiatry Research, 112, 41–50.
Huang, C. L., & Huang, Y. M. (1997). A facial expression recognition using model-based feature extraction and action parameters classification. Journal of Visual Communication and Image Representation, 8(3), 278–290.
Izard, C. E. (1971). The face of emotion. New York: Appleton-Century-Crofts.
Izard, C. E. (1979). The maximally discriminative facial movement coding system (MAX). Newark, Del: University of Delaware, Instructional Resource Center.
Kanade, T. (1973). Picture processing system by computer complex and recognition of human faces. Unpublished Ph.D. dissertation, Tokyo University, Tokyo, Japan.
Kato, M., So, I., Hishinuma, Y., Nakamura, O., & Minami, T. (1992). Description and synthesis of facial expression based on isodensity maps. In L. Tosiyasu (Ed.), Visual computing (pp. 39–56). Tokyo: Springer.
Keltner, D., & Ekman, P. (2000). Facial expression of emotion. In M. Lewis & J. M. Haviland-Jones (Eds.), Handbook of emotions (pp. 236–249). New York, USA: Guilford Press.
Kimura, S., & Yachida, M. (1997). A facial expression recognition and its degree estimation. In Proceedings of Computer Vision and Pattern Recognition (pp. 295–300).
Kline, J. S., Smith, J. E., & Ellis, H. C. (1992). Paranoid and nonparanoid schizophrenic processing of facially displayed affect. Journal of Psychiatric Research, 26, 169–182.
Kobayashi, H., & Hara, F. (1992). Recognition of six basic facial expressions and their strength by neural network. Proc. Int'l Workshop Robo, H.t and Human Comm., 381–386.
Kobayashi, H., & Hara, F. (1997). Facial interaction between animated 3D face robot and human beings. In Proceedings of International Conference Systems, Man, Cybernetics (Vol. 3, pp. 732–3,737).
Kobayashi, H., Takahashi, H., Kimura, T., Kikuchi, K., & Tazaki, M. (2000). Study on automatic diagnosis of psychiatric disorder by facial expression. In 26th Annual Conference of the IEEE on Industrial Electronics Society at Nagoya, Japan (Vol. 1, pp. 487–492).
Kobayashi, H., Tange, K., & Hara, F. (1995). Realtime recognition of six basic facial expressions. In Proceedings of the IEEE International Workshop on Robot and Human Communication, Tokyo (pp. 179–186).
Kornreich, C., Philippot, P., Foisy, M. L., Blairy, S., Raynaud, E., Dan, B., et al. (2002). Impaired emotional facial expression recognition is associated with interpersonal problems in alcoholism. Alcohol and Alcoholism, 37(4), 394–400.
Ladouceur, C. D., Dahl, R. E., Birmaher, B., Axelson, D. A., & Ryan, N. D. (2006). Increased error-related negativity (ERN) in childhood anxiety disorders: ERP and source localization. Journal of Child Psychology and Psychiatry, 47(10), 1073–1082.
Lam, K. M., & Yan, H. (1998). An analytic-to-holistic approach for face recognition based on a single frontal view. IEEE Trans. Pattern Analysis and Machine Intelligence, 20(7), 673–686.
Lewis, S. F., & Garver, D. L. (1995). Treatment and diagnostic subtype in facial affect recognition in schizophrenia. Journal of Psychiatric Research, 29, 5–11.
Li, S., & Gu, L. (2001). Real-time multi-view face detection, tracking, pose estimation, alignment, and recognition. In IEEE Conference on Computer Vision and Pattern Recognition Demo Summary.
Lien, J. J., Kanade, T., Cohn, J. F., & Li, C. C. (1998). A automated facial expression recognition based on FACS action units. In Proceedings of International Conference Automatic Face and Gesture Recognition (pp. 390–395).
Liu, C. (2003). A Bayesian discriminating features method for face detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(6), 725–740.
Lucey, P., Cohn, J., Prkachin, K., Solomon, P., & Matthews, I. (2011). Painful data: The UNBC-McMaster shoulder pain expression archive database. In International Conference on Automatic Face and Gesture Recognition and Workshops, IEEE (pp. 57–64).
Lucey, P., Cohn, J. F., Prkachin, K. M., Solomon, P. E., Chew, S., & Matthews, I. (2012). Painful monitoring: Automatic pain monitoring using the UNBC-McMaster shoulder pain expression archive database. Image and Vision Computing, 30, 197–205.
Lyons, M. J., Budynek, J., & Akamatsu, S. (1999). Automatic classification of single facial images. IEEE Trans. Pattern Analysis and Machine Intelligence, 21(12), 1357–1362.
Mandal, M. K., & Ambady, N. (2004). Laterality of facial expressions of emotion: Universal and culture-specific influences. Behavioral Neurology, 15, 23–34.
Martin, C. C., Borod, J. C., Alpert, M., Brozgold, A., & Welkowitz, J. (1990). Spontaneous expression of facial emotion in schizophrenic and right brain damaged patients. Journal of Communication Disorder, 23, 287–301.
McIntosh, D. N., Reichmann-Decker, A., Winkielman, P., & Wilbarger, J. L. (2006). When the social mirror breaks: Deficits in automatic, but not voluntary, mimicry of emotional facial expressions in autism. Developmental Science, 9, 295–302.
Mineka, S., & Zinbarg, R. (1996). Conditioning and ethological models of anxiety disorders: Stress-in-dynamic context anxiety models. In D. Hope (Ed.), Perspectives on anxiety, panic and fear: Nebraska symposium on motivation. Lincoln: University of Nebraska Press.
Otsuka, T., & Ohya, J. (1996). A recognition of facial expressions using HMM with continuous output probabilities. In Proceedings of International Workshop Robot and Human Communication (pp. 323–328).
Padgett, C., & Cottrell, G. W. (1996). A representing face images for emotion classification. In Proceedings of Conference Advances in Neural Information Processing Systems (pp. 894–900).
Pantic, M., & Bartlett, M. S. (2007). Machine analysis of facial expressions. In K. Delac & M. Grgic (Eds.), Face Recognition (pp. 377–416). Vienna, Austria: I-Tech education and publishing.
Pantic, M., & Rothkrantz, L. J. M. (2000a). Expert system for automatic analysis of facial expression. Image and Vision Computing, 18(11), 881–905.
Pantic, M., & Rothkrantz, L. J. M. (2000b). Automatic analysis of facial expressions: the state of the art. IEEE Trans. of Pattern Analysis and Machine intelligence, 22(12), 1424–1445.
Pentland, A., Moghaddam, B., & Starner, T. (1994).View-based and modular eigenspaces for face recognition. In Proceedings of IEEE Conference Computer Vision and Pattern Recognition (pp. 84–91).
Prkachin, K. M., Hughes, E., Schultz, I., Joy, P., & Hunt, D. (2002). Real-time assessment of pain behavior during clinical assessment of low back pain patients. Pain, 95, 23–30.
Radaelli, D., Benedetti, F., Cavallaro, R., Colombo, C., & Smeraldi, E. (2013). The reality monitoring deficit as a common neuropsychological correlate of schizophrenic and affective psychosis. Behavioural Sciences, 3(2), 244–252.
Remillard, G. M., Andremann, F., Rhi-Suci, A., & Robbins, N. M. (1977). Facial asymmetry in patients with temporal lobe epilepsy. Neurology, 27, 109–114.
Rinn, W. E. (1984). The neuropsychology of facial expression: A review of the neurological and psychological mechanisms for producing facial expressions. Psychological Bulletin, 95, 52–77.
Rowley, H., Baluja, S., & Kanade, T. (1998). Neural network-based face detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(1), 23–38.
Sackeim, H. A., Gur, R. C., & Saucy, M. C. (1978). Emotions are expressed more intensely on the left side of the face. Science, 202, 434–436.
Sackett, G. P. (1966). Monkeys reared in isolation with pictures as visual input: Evidence for an innate releasing mechanism. Science, 154, 1468–1473.
Samal, A., & Iyengar, P. A. (1992). Automatic recognition and analysis of human faces and facial expressions: A survey. Pattern Recognition, 25(1), 65–77.
Schneider, F., Koch, J. D., Mattes, R., & Heimann, H. (1992). Recognition of emotions in facial expression of visual half-field images by schizophrenic and depressed patients. Nervenarzt, 63, 545–550.
Schneider, W., & Shiffrin, R. M. (1977). Controlled and automatic information processing: 1. Detection, search and attention. Psychological Review, 84, 1–66.
Schneiderman, H., & Kanade, T. (2000). A statistical model for 3d object detection applied to faces and cars. In IEEE Conference on Computer Vision and Pattern Recognition.
Shan, C., Gong, S., & McOwan, P. W. (2009). Facial expression recognition based on Local Binary Patterns: A comprehensive study. Image and Vision Computing, 27, 803–816.
Shenoy, A. (2009). Computational analysis of facial expressions. Unpublished Ph.D. Thesis, School of Computer Science, University of Hertfordshire.
Simoncelli, E. (1993). A distributed representation and analysis of visual motion. Unpublished PhD thesis, Massachusetts Institute of Technology.
Steffens, J., Elagin, E., & Neven, H., (1998). A person spotter fast and robust system for human detection, tracking, and recognition. In Proceedings of International Conference on Automatic Face and Gesture Recognition (pp. 516–521).
Steimer-Krause, E., Krause, R., & Wagner, G. (1990). Interaction regulation used by schizophrenic and psychosomatic patients: Studies on facial behavior in dyadic interactions. Psychiatry, 53, 209–228.
Stuhrmann, A., Suslow, T., & Dannlowski, U. (2011). Facial emotion processing in major depression: A systematic review of neuroimaging findings. Biology of Mood and Anxiety Disorders,. doi:10.1186/2045-5380-1-10.
Tassinary, L. G., Cacioppo, J. T., & Geen, T. R. (1989). A psychometric study of surface electrode placement for facial electromyography recording: 1. the brow & cheek muscle regions. Psychophysiology, 26, 1–16.
Tomkins, S. S. (1962). Affect, imagery and consciousness. The positive effects. New York: Springer.
Uljarevic, M., & Hamilton, A. (2013). Recognition of emotions in autism: A formal meta-analysis. Journal of Autism and Developmental Disorders, 43(7), 1517–1526.
Van Gelder, R. (1981). Neuroanatomy and facial asymmetry. Internal publication of the ILO, Department of Psychology, Free University of Amsterdam.
Viola, P. & Jones, M. (2001). Rapid object detection using boosted cascade of simple features. IEEE Conference on Computer Vision and Pattern Recognition.
Walker, E. F., Grimes, K. E., Davis, D. M., & Smith, A. J. (1993). Childhood precursors of schizophrenia: Facial expressions of emotion. American Journal of Psychiatry, 150, 1654–1660.
Wang, M., Iwai, Y., & Yachida, M. (1998). A expression recognition from time-sequential facial images by use of expression change model. In Proceedings of International Conference Automatic Face and Gesture Recognition (pp. 324–329).
Wang, P., Barrett, F., Martin, E., Milonova, M., Gurd, R., Gurb, R., et al. (2008). Automated video-based facial expression analysis of neuropsychiatric disorders. Journal of Neuroscience Methods, 168, 224–238.
Wehrle, T., & Scherer, K. R. (2001). Towards computational modeling of appraisal theories. In K. R. Scherer, A. Schorr, & T. Johnstone (Eds.), Appraisal processes in emotions: Theory, methods, research (pp. 350–365). New York: Oxford University Press.
Wiskott, L. (1995). Labelled graphs and dynamic link matching for face recognition and scene analysis. Verlag Harri Deutsch, Thun-Frankfurt am Main, Reihe Physik, p. 53.
Wolff, W. (1943). The expression of personality. New York, NY: Harper.
Yoneyama, M., Iwano, Y., Ohtake, A., & Shirai, K. (1997). A facial expressions recognition using discrete hopfield neural networks. In Proceedings of International Conference on Information Processing (Vol. 3, pp. 117–120).
Yuille, A. L., Cohen, D. S., & Hallinan, P. W. (1989). A feature extraction from faces using deformable templates. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 104–109).
Zajonc, R. B. (1980). Feeling and thinking: Preferences need no inferences. American Psychologist, 35, 151–175.
Zhang, Z., Lyons, M. J., Schuster, M., & Akamatsu, S. (1998). Comparison between geometry-based and Gabor-wavelets-based facial expression recognition using multi-layer perceptron. In IEEE International Conference on Automatic Face & Gesture Recognition (FG), Nara, Japan, pp. 454–459.
Zhao, J., & Kearney, G. (1996). A classifying facial emotions by backpropagation neural networks with fuzzy inputs. In Proceedings of the Conference on Neural Information Processing (Vol. 1, pp. 454–457).
Acknowledgment
The work is supported by the University Grants Commission (UGC) CPEPA, India (F. No. 8-2/2008 (NS/PE), dated December 14, 2011).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer India
About this chapter
Cite this chapter
Poria, S., Mondal, A., Mukhopadhyay, P. (2015). Evaluation of the Intricacies of Emotional Facial Expression of Psychiatric Patients Using Computational Models. In: Mandal, M., Awasthi, A. (eds) Understanding Facial Expressions in Communication. Springer, New Delhi. https://doi.org/10.1007/978-81-322-1934-7_10
Download citation
DOI: https://doi.org/10.1007/978-81-322-1934-7_10
Published:
Publisher Name: Springer, New Delhi
Print ISBN: 978-81-322-1933-0
Online ISBN: 978-81-322-1934-7
eBook Packages: Behavioral ScienceBehavioral Science and Psychology (R0)