Cognitive Processing

, Volume 18, Issue 3, pp 285–306 | Cite as

Facial expressions and speech acts: experimental evidences on the role of the upper face as an illocutionary force indicating device in language comprehension

  • Filippo DomaneschiEmail author
  • Marcello Passarelli
  • Carlo Chiorri
Research Report


Language scientists have broadly addressed the problem of explaining how language users recognize the kind of speech act performed by a speaker uttering a sentence in a particular context. They have done so by investigating the role played by the illocutionary force indicating devices (IFIDs), i.e., all linguistic elements that indicate the illocutionary force of an utterance. The present work takes a first step in the direction of an experimental investigation of non-verbal IFIDs because it investigates the role played by facial expressions and, in particular, of upper-face action units (AUs) in the comprehension of three basic types of illocutionary force: assertions, questions, and orders. The results from a pilot experiment on production and two comprehension experiments showed that (1) certain upper-face AUs seem to constitute non-verbal signals that contribute to the understanding of the illocutionary force of questions and orders; (2) assertions are not expected to be marked by any upper-face AU; (3) some upper-face AUs can be associated, with different degrees of compatibility, with both questions and orders.


Psycholinguistics Experimental pragmatics Speech acts Illocutionary force indicating devices Facial expressions 



This research has been funded by the Italian Ministry of Education, University and Research within the three-year project SIR_2014 - EXPRESS directed by Filippo Domaneschi, project code RBSI147WM0.

Authors’ contribution

F.D. designed the research and prepared the experimental material, M.P. conducted the research and analyzed the data, C.C. helped with the statistics, F.D. wrote the paper, M.P. and C.C. revised the manuscript.


  1. Aikhenvald AY (2010) Imperatives and commands. Oxford University Press, OxfordGoogle Scholar
  2. Allwood J, Cerrato L, Jokinen K, Navarretta C, Paggio P (2005) The MUMIN annotation scheme for feedback, turn management and sequencing. In: Gothenburg papers in theoretical linguistics 92: Proceedings from the second Nordic conference on multimodal communication. Göteborg University, Göteborg, Sweden, pp 91–109Google Scholar
  3. Alston WP (2000) Illocutionary acts and sentence meaning. Cornell University Press, IthacaGoogle Scholar
  4. Argyle M, Cook M (1976) Gaze and mutual gaze. Cambridge University Press, CambridgeGoogle Scholar
  5. Austin J (1962) How to do things with words. Oxford University Press, OxfordGoogle Scholar
  6. Bach K (2006) Speech acts and pragmatics. In: Devitt M, Hanley R (eds) Blackwell guide to the philosophy of language. Blackwell, Oxford, pp 147–167Google Scholar
  7. Bartels C (1999) The intonation of English statements and questions. A compositional interpretation. Garland Publishing Inc., New YorkGoogle Scholar
  8. Braun D (2011) Implicating questions. Mind Lang 26(5):574–595. doi: 10.1111/j.1468-0017.2011.01431.x CrossRefGoogle Scholar
  9. Brown P, Levinson S (1987) Politeness. Some universals in language usage. Cambridge University Press, CambridgeGoogle Scholar
  10. Condoravdi C, Lauer S (2011) Performative verbs and performative acts. In: Reich I, Horch E, Pauly D (eds) Proceedings of Sinn & Bedeutung, vol. 15 (1–15). Universaar – Saarland University Press, SaarbrückenGoogle Scholar
  11. Davies E (1986) The English imperative. Croom Helm, LondonGoogle Scholar
  12. Dresner E, Herring SC (2010) Functions of the nonverbal in CMC: emoticons and illocutionary force. Commun Theory 20(3):249–268. doi: 10.1111/j.1468-2885.2010.01362.x CrossRefGoogle Scholar
  13. Ekman P (1979) About brows: emotional and conversational signals. In: von Cranach M, Foppa K, Lepenies W, Ploog D (eds) Human ethology. Cambridge University Press, Cambridge, pp 169–248Google Scholar
  14. Ekman P, Friesen WV (1978) Manual for the facial action coding system. Consulting Psychologists Press, Palo AltoGoogle Scholar
  15. Emerich S, Lupu E, Apatean A (2009) Emotions recognition by speechand facial expressions analysis. In: Signal processing conference, 2009 17th European. IEEE Computer Society, Washington, pp 1617–1621Google Scholar
  16. Esposito A, Esposito AM, Martone R, Müller V, Scarpetta G (eds) (2010) Towards autonomous, adaptive, and context-aware multimodal interfaces: theoretical and practical issues. Springer, NewYorkGoogle Scholar
  17. Goffman E (1967) Interaction ritual: essays in face to face behavior. Aldine Transaction, New BrunswickGoogle Scholar
  18. Granström B, House D, Beskow J (2002) Speech and gestures for talking faces in conversational dialogue systems. In: Granström B, House D, Karlsson I (eds) Multimodality in language and speech systems. Kluwer Academic Publishers, Dordrecht, pp 209–241CrossRefGoogle Scholar
  19. Green MS (2000) Illocutionary force and semantic content. Linguist Philos 23(5):435–473. doi: 10.1023/A:1005642421177 CrossRefGoogle Scholar
  20. Green M (2013) Assertions. In: Sbisà M, Turner K (eds) Handbook of pragmatics, vol II., Pragmatics of speech actionsDe Gruyter-Mouton, Berlin, pp 387–410Google Scholar
  21. Hanks PW (2007) The content–force distinction. Philos Stud 134(2):141–164. doi: 10.1007/s11098-007-9080-5 CrossRefGoogle Scholar
  22. Heritage J (2012) Epistemics in action: action formation and territories of knowledge. Res Lang Soc Interact 45:1–29CrossRefGoogle Scholar
  23. Jary M (2007) Are explicit performatives assertions? Linguist Philos 30(2):207–234. doi: 10.1007/s10988-007-9015-9 CrossRefGoogle Scholar
  24. Jokinen K, Nishida M, Yamamoto S (2010) On eye-gaze and turn-taking. In: Proceedings of the 2010 workshop on Eye gaze in intelligent human machine interaction. ACM, New York, pp 118–123Google Scholar
  25. Keltner D (1995) Signs of appeasement: evidence for the distinct displays of embarrassment, amusement, and shame. J Pers Soc Psychol 68(3):441–454. doi: 10.1037/0022-3514.68.3.441 CrossRefGoogle Scholar
  26. Kendon A (1995) Gestures as illocutionary and discourse structure markers in Southern Italian conversation. J Pragmat 23(3):247–279. doi: 10.1016/0378-2166(94)00037-F CrossRefGoogle Scholar
  27. Kendon A (2004) Gesture: visible action as utterance. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  28. Kissine M (2013) From utterances to speech acts. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  29. Kohler CG, Turner T, Stolar NM, Bilker WB, Brensinger CM, Gur RE, Gur RC (2004) Differences in facial expressions of four universal emotions. Psychiatry Res 128(3):235–244. doi: 10.1016/j.psychres.2004.07.003 CrossRefPubMedGoogle Scholar
  30. Konig E, Siemund P (2007) Speech act distinctions in grammar. In: Shopen T (ed) Language typology and syntactic description. Cambridge University Press, Cambridge, pp 276–324CrossRefGoogle Scholar
  31. Levinson S (1983) Pragmatics. Cambridge University Press, CambridgeGoogle Scholar
  32. McGurk H, MacDonald J (1976) Hearing lips and seeing voices. Nature 5588(264):746–748. doi: 10.1038/264746a0 CrossRefGoogle Scholar
  33. Oertel C, Wlodarczak M, Edlund J, Wagner P, Gustafson J (2012) Gaze patterns in turn-taking. In: Proceedings of Interspeech 2012. Portland, ORGoogle Scholar
  34. Pelachaud C, Prevost S (1994) Sight and sound: generating facial expressions and spoken intonation from context. In: Proceedings of the 2nd workshop on speech synthesis. New Paltz, New York, pp 216–219Google Scholar
  35. Pelachaud C, Carofiglio V, De Carolis B, de Rosis F, Poggi I (2002) Embodied contextual agent in information delivering application. In: Proceedings of the first international joint conference on autonomous agents and multiagent systems: Part 2. ACM, New York, pp 758–765Google Scholar
  36. Pierrehumbert J, Hirschberg J (1990) The meaning of intonational contours in the interpretation of discourse. In: Cohen P, Morgan J, Pollack M (eds) Intentions in communication. MIT Press, Cambridge, pp 271–311Google Scholar
  37. Poggi I (2007) Mind, hands, face and body. A goal and belief view of multimodal communication. Weidler Buchverlag, BerlinGoogle Scholar
  38. Poggi I, Pelachaud C (1998) Performative faces. Speech Commun 26(1):5–21. doi: 10.1016/S0167-6393(98)00047-8 CrossRefGoogle Scholar
  39. Poggi I, Pelachaud C (2001) The meanings of gaze in animated faces. In: McKevitt P, Nuàllain S, Mulvihill C (eds) Language, vision and music. John Benjamins, Amsterdam, p 2001Google Scholar
  40. Recanati F (2013) Content, mood, and force. Philos Compass 8(7):622–632. doi: 10.1111/phc3.12045 CrossRefGoogle Scholar
  41. Rizzi L (1997) On the position “int(errogative)” in the left periphery of the clause. In: Cinque G, Salvi G (eds) Current studies in Italian syntax. Elsevier, Amsterdam, pp 287–296Google Scholar
  42. Sadock JM (1974) Toward a linguistic theory of speech acts. Academic Press, New YorkGoogle Scholar
  43. Sbisà M (1987) Indicative mood, illocutionary force, and truth: some points for discussion. LAUD - Linguistic Agency University of Duisburg-Essen, DuisburgGoogle Scholar
  44. Sbisà M (2007) How to read Austin. Pragmatics 17(3):461. doi: 10.1075/prag.17.3.06sbi CrossRefGoogle Scholar
  45. Searle JR (1968) Austin on locutionary and illocutionary acts. Philos Rev 77(4):405–424. doi: 10.2307/2183008 CrossRefGoogle Scholar
  46. Searle J (1969) Speech acts. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  47. Searle JR (1975) Indirect speech acts. In: Cole P, Morgan JL (eds) Syntax and semantics, vol 3., Speech actsAcademic Press, New York, pp 59–82Google Scholar
  48. Searle JR, Vanderveken D (1985) The foundations of illocutionary logic. Cambridge University Press, CambridgeGoogle Scholar
  49. Shah M, Cooper DG, Cao H, Gur RC, Nenkova A, Verma R (2013) Action unit models of facial expression of emotion in the presence of speech. In: Affective computing and intelligent interaction (ACII), 2013 Humaine association conference. IEEE Computer Society, Washington, pp 49–54Google Scholar
  50. Stampe DW (1975) Meaning and truth in the theory of speech acts. In: Cole P, Morgan JL (eds) Speech acts. Academic Press, New York, pp 1–39Google Scholar

Copyright information

© Marta Olivetti Belardinelli and Springer-Verlag Berlin Heidelberg 2017

Authors and Affiliations

  • Filippo Domaneschi
    • 1
    Email author
  • Marcello Passarelli
    • 1
  • Carlo Chiorri
    • 1
  1. 1.Università degli Studi di GenovaGenoaItaly

Personalised recommendations