More About Brows

A Cross-Linguistic Study via Analysis-by-Synthesis
  • Emiel Krahmer
  • Marc Swerts
Part of the Human-Computer Interaction Series book series (HCIS, volume 7)

Abstract

In a seminal paper, Ekman (1979) remarks that brows can play an accentuation role (e.g., to signal focus). However, the literature about eyebrows is inconclusive about their exact role and as a consequence there is no agreement among developers of embodied conversational agents about their precise timing and placement. In addition, it is unclear whether eyebrow movements perform the same role in different languages. In this chapter, an analysis-by-synthesis technique is used to find out what the role of eyebrow movements is for the perception of focus and to see whether this role is the same across different languages. Three experiments are performed, both for Dutch and Italian, investigating where subjects prefer eyebrow movements, whether brows influence the perceived prominence of words and whether they are used in a functional way when subjects interpret utterances. The results for Dutch and Italian are indeed different, but it is argued that these differences can be reduced to prosodic differences between the two languages. The advantages and potential limitations of studies via analysis-by-synthesis are discussed, and an approach to compensate for the limitations is offered.

Keywords

Audio-visual prosody eyebrow movements pitch accents focus prominence perception analysis-by-synthesis analysis-by-observation cross-linguistic comparisons 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bolinger, D. (1985). Intonation and its parts. Edward Arnold, London.Google Scholar
  2. Birdwhistell, R. (1970). Kinesics and context. University of Pennsylvania Press.Google Scholar
  3. Cassell, J., Sullivan, J., Prevost, S., and Churchill, E., editors (2000). Embodied Conversational Agents. The MIT Press, Cambridge, MA.Google Scholar
  4. Cassell, J., Vihjálmsson, H., and Bickmore, T. (2001). BEAT: The Behavior Expression Animation Toolkit. In Proceedings of SIG-GRAPH’01, pp. 477–486, Los Angeles.Google Scholar
  5. Cavé, C., Guaïtella, I., Bertrand, R., Santi, S., Harlay, F., and Espesser, R. (1996). About the relationship between eyebrow movements and F 0 variations. In Proceedings of the International Conference on Spoken Language Processing (ICSLP), pp. 2175–2179, Philadelphia.Google Scholar
  6. Chafe, W. (1974). Language and consciousness. Language 50: 111–133.CrossRefGoogle Scholar
  7. Condon, W. (1976). An analysis of behavioral organization. Sign Language Studies, 13: 285–318.CrossRefGoogle Scholar
  8. Cutler, A. (1984). Stress and accent in language production and understanding. In Gibbon, D. and Richter, H., editors, Intonation, accent and rhythm. Studies in Discourse Phonology. pp. 77–90, de Gruyter, Berlin.Google Scholar
  9. Cruttenden, A. (1997). Intonation, 2nd edition. Cambridge University Press, Cambridge.CrossRefGoogle Scholar
  10. Darwin, Ch. (1872). The Expression of the emotions in man and animals. Philosophical Library, New York.CrossRefGoogle Scholar
  11. Doherty-Sneddon, G., Bonner, L., and Bruce, V. (2001). Cognitive demands of face monitoring: Evidence for visuospatial overload. Memory and Cognition, 29(7): 909–919.CrossRefGoogle Scholar
  12. Efron, D. (1941). Gesture and environment. King’s Crown Press, New York.Google Scholar
  13. Eibl-Eibesfelt, I. (1972). Similarities and differences between cultures in expressive movements. In Hinde, R., editor, Non-verbal communication. Cambridge University Press, Cambridge.Google Scholar
  14. Ekman, P. (1979). About brows: Emotional and conversational signals. In von Cranach, M., Foppa, K., Lepenies, W. and Ploog, D., editors, Human ethology: Claims and limits of a new discipline. pp. 169–202, Cambridge University Press, Cambridge.Google Scholar
  15. Ekman, P. and Friesen, W. (1978). Facial Action Coding System. Consulting Psychologists Press, Inc, Palo Alto.Google Scholar
  16. Erickson, D., Fujimura, O., and Pardo, B. (1998). Articulatory correlates of prosodic control: Emotion and emphasis. Language and Speech, 41(3–4): 399–417.Google Scholar
  17. Granström, B., House, D., and Lundeberg, M. (1999). Prosodic cues to multimodal speech perception. In Proceedings 14th International Conference of the Phonetic Sciences (ICPhS), pp. 655658, San Francisco.Google Scholar
  18. Granström, B., House, D., and Swerts, M. (2002). Multimodal feedback cues in human-machine interactions. In Proceedings of Speech Prosody 2002. pp. 347–350, Aix en Provence, France.Google Scholar
  19. Gratch, J., Rickel, J., André, E., Badler, N., Cassell, J., and Petajan, E. (2002). Creating interactive virtual humans: Some assembly required. IEEE Intelligent Systems, 17(4): 54–63.CrossRefGoogle Scholar
  20. Hirschberg, J. (1993). Pitch accents in context: predicting intonational prominence from text. Artificial Intelligence, 63: 305–340.CrossRefGoogle Scholar
  21. Keating, P., Baroni, M., Mattys, S., Scarborough, R., Alwan, A., Auer, E., and Berstein, L. (2003). Optical phonetics and visual perception of lexical and phrasal stress in English. In Proceedings 16th International Conference of the Phonetic Sciences (ICPhS), pp. 2071–2074, Barcelona, Spain.Google Scholar
  22. Krahmer, E. and Swerts, M. (2001). On the alleged existence of contrastive accents, Speech Communication, 34: 391–405.MATHCrossRefGoogle Scholar
  23. Krahmer, E., Ruttkay, Zs., Swerts, M., and Wesselink, W. (2002a). Pitch, eyebrows and the perception of focus. In Proceedings of Speech Prosody 2002, pp. 443–446, Aix en Provence, France.Google Scholar
  24. Krahmer, E., Ruttkay, Zs., Swerts, M., and Wesselink, W. (2002b). Perceptual evaluation of audio-visual cues to prominence. In Proceedings of International Conference on Spoken Language Processing (IC-SLP’02), pp. 1933–1936, Denver, CO.Google Scholar
  25. Ladd, D. (1996). Intonational phonology. Cambridge University Press, Cambridge.Google Scholar
  26. van de Laar, L. (2003). Influence of eyes on the interpretation of utterances of embodied conversational agents: An experimental inquiry. MA thesis, Tilburg University.Google Scholar
  27. Morgan, B. (1953). Question melodies in American English. American Speech, 2: 181–191.CrossRefGoogle Scholar
  28. Nass, C., Isbister, K., and Lee, E. (2000). Truth is beauty: Researching embodied conversational agents. In Cassell, J., Sullivan, J., Prevost, S., and Churchill, E., editors, Embodied Conversational Agents. The MIT Press, Cambridge, MA.Google Scholar
  29. O’Sullivan, M. and Eyman, J. (1978). The signal value of eyebrow movements in conversation. J. Western Psychological Association Convention. San Francisco.Google Scholar
  30. Pelachaud, C., Badler, N., and Steedman, M. (1996). Generating facial expressions for speech. Cognitive Science, 20: 1–46.CrossRefGoogle Scholar
  31. Perlin, K. (1995). Real time responsive animation with personality. IEEE Transactions on Visualization and Computer Graphics, 1(1): 5–15.CrossRefGoogle Scholar
  32. Rimé, B. and Schiaratura, L. (1991). Gesture and speech. In Feldman, R. and Rimé, B., editors, Fundamentals of nonverbal behavior. pp. 239–281, Cambridge University Press, Cambridge.Google Scholar
  33. Rubin, P. and Vatikiotis-Bateson, E. (1998). Talking heads. In Burnham, D., Robert-Ribes, J. and Vatikiotis-Bateson, E., editors, International Conference on Auditory-Visual Speech Processing (AVSP’98), pp. 233238, Sydney, AustraliaGoogle Scholar
  34. Ruttkay, Zs. (2001). Constraint-based facial animation. Journal of Constraints, 6: 85–113MATHCrossRefGoogle Scholar
  35. Ruttkay, Zs. and Noot, H. (2000). Animated CharToon Faces. In Proceedings of NPAR 2000 — First International Symposium on Non Photorealistic Animation and Rendering, pp. 91–100, Annecy, France.Google Scholar
  36. Sanderman, A. and Collier, R. (1997). Prosodic phrasing and comprehension. Language and Speech, 40(4): 391–409.Google Scholar
  37. Swerts, M., Krahmer, E., and Avesani, C. (2002). Prosodic marking of information status in Dutch and Italian: A comparative analysis. Journal of Phonetics, 30(4): 629–654.CrossRefGoogle Scholar
  38. Terken, J. (1984). The distribution of pitch accents in instructions as a function of discourse structure. Language and Speech, 27: 269–289.Google Scholar
  39. Terken, J. and Nooteboom, S. (1987). Opposite effects of accentuation and deaccentuation on verification latencies for Given and New information. Language and Cognitive Processes, 2(3–4): 145–163.CrossRefGoogle Scholar
  40. Zappa, F. (1989). The Real Frank Zappa Book, Poseidon Press, New York.Google Scholar

Copyright information

© Kluwer Academic Publishers 2004

Authors and Affiliations

  • Emiel Krahmer
  • Marc Swerts

There are no affiliations available

Personalised recommendations