Gestural agreement

Abstract

We argue that a gesture replacing an English verb—a ‘gestural verb’—displays some properties of ‘agreement verbs’ in American Sign Language (ASL). Specifically, gestural verbs involving (among others) slapping and punching can be realized as targeting the addressee (SLAP-2, PUNCH-2) if the object is second person, or as targeting some other position (SLAP-a, PUNCH-a) if the object is third person. This property is shared with ASL verbs that display object agreement. Strikingly, in both cases the object agreement marker can be disregarded under ellipsis and under the focus-sensitive particle only, a behavior which is shared with phi-features in spoken language, and is not entirely reducible to the presuppositional nature of the marker. The main findings are based on introspective judgments, but crucial examples are validated by an experimental approach. In sum, we provide initial evidence that English gestural verbs have a grammar, and that it partly mirrors that of some sign language constructions.

This is a preview of subscription content, access via your institution.

Notes

  1. 1.

    Liddell (2003: 139) writes: “Each individual verb has specific gestural characteristics associated with it. Each verb’s directional characteristics are potentially distinct from those of any other ASL verb. Some verbs are unable to point, some point toward entities corresponding to their trajectors, some point toward entities corresponding to their landmarks, and some point to both. For those that do point, if they are directed at a person, they are directed at specific parts of the person (e.g. forehead, nose, chin, sternum). These are not general characteristics of gestural ‘accompaniments’ to signing. These are specific, semantically relevant, properties of individual verbs.”

  2. 2.

    Some of these gestural verbs make reference to objectionable actions. We use them because at this early stage it proved non-trivial to find many gestural verbs that are readily understood and have the right formal properties.

  3. 3.

    The term ‘pro-speech gesture’ is constructed with the prefix pro found in pronoun (‘replacing a noun’) and proconsul (‘replacing the consul’): a pro-speech gesture is a gesture that replaces a spoken expression. Existing typologies also include co-speech gestures, which co-occur with spoken expressions they modify; and post-speech gestures, which follow expressions they modify and have their own time slot. See Schlenker (to appear c) for a discussion of this typology.

  4. 4.

    See also Kuhn (2015) for a view in which loci establish an unlimited number of person distinctions, including among third persons.

  5. 5.

    We believe on the basis of ongoing investigations that a plurality of third person loci can be used with English co- and pro-speech gestures.

  6. 6.

    This point is worth emphasizing, for while it is obvious to competent linguists that sign languages are full-fledged—and extremely interesting—languages, and that they have a crucial role to play in the development of deaf children (e.g. Mellon et al. 2015), there are still attempts in some countries to assimilate them to mere gestural codes.

  7. 7.

    See also Merchant (2014) for a recent discussion of the behavior of gender (and plural) features in ellipsis contexts in Modern Greek.

  8. 8.

    Here and elsewhere, VERY is the sign which is sometimes glossed as WOW.

  9. 9.

    All loci were signed at a neutral height, except for IX-a in a. and c.

  10. 10.

    Two remarks should be added.

    1. Our consultant noted about (13)a: “this sentence seems to be structured to use space but then it isn’t used. As a result the grammar is slightly off” (he noted on another occasion that there wasn’t really an English influence but that the grammar was “slightly off”).

    2. Note that the verb GIVE can also target a neutral horizontal position, which we write without a locus suffix. But when this is the case, we do not know whether it can target a high position; the judgments we have in (ib) changed over time (we give judgments for the situation in which the brother is not present, which matters for (ib); our consultant notes that his preference for a translation in the indicative seems to be due to the high locus rather than to other aspects of the realization of GIVE). Note that the control sentence in (ic) is to some extent acceptable, but with an irrelevant reading, on which the object of GIVE is the addressee.

    1. (i)
      POSS-2 HEIGHT BROTHER MONEY IX-1
      a. 61-GIVE ‘Your tall brother, I would give money to.’
      b. 5.51-GIVE-high ‘Your tall brother, I will/have give(n) money to.’ (Judgments: 4, 6, 7, 5
      [the last one on the assumption that the tall brother is not present])
      c. 5.71-GIVE-2 ‘Concerning your tall brother, the money, I give you.’ (i.e. the money
      pertaining to your brother, I give you). (Judgments: 6, 6, 7, 4)
      (ASL, 34, 1574; 4 judgments)
  11. 11.

    An exception pertains to the object position of the expression be like, as in (i), from Davidson (2015). In this case, the object position is occupied by a gesture, not a word.

    (i) Bob was eating like [gobbling gesture].

    While the gesture in (i) could be replaced with the word this, for the sentence to be acceptable with a comparable meaning, this would have to refer to an imitation. By contrast, the pro-speech gestures studied in this piece need not be introduced by a special construction, and they can be replaced with verbs that refer to normal actions.

    Davidson (2015) also discusses a sign language construction, Action Role Shift, which has been analyzed as necessarily including an iconic component (Schlenker To appear a) or a gestural component (for Davidson, who takes some signs to make demonstrative reference to their own gestural form). In neither analysis are these pro-speech or pro-sign gestures: they are full-fledged words that are iconically or gesturally modulated.

  12. 12.

    This issue is also investigated in work in progress by Jon Gajewski.

  13. 13.

    The onomatopoeia might also help justify the absence of a spoken word. But this doesn’t seem to be the whole story, since ‘post-speech’ gestures, which come after the words they modify, also seem conducive to onomatopoeias, as in (i), where — should be taken to stand for a short pause:

    (i) This helicopter will soon take off – .

    See Schlenker (to appear c) for examples, involving a gesture of dozing off, which might not need to be accompanied by onomatopoeia.

  14. 14.

    Two remarks should be added:

    1. The projection of presuppositions should make itself felt in (16)b. First, TAKE-OFF-ROTATING triggers a presupposition that its argument is on the ground—which is unproblematic in the present context. Second, and more tentatively, in connection with (34) below we propose that TAKE-OFF-ROTATING triggers a presupposition that its argument is helicopter-like in involving a rotating motion. Now it has been argued (again tentatively) that a sentence of the form [only DP]F gives rise to the inference that alternatives to DP satisfy the presuppositions of F (e.g. Schlenker 2009). On this assumption, we predict for (17)b a presupposition that the plane is the kind of object that could move upwards by way of a rotating motion. Local accommodation of this presupposition would be needed to avoid infelicity in the case at hand. When local accommodation is applied, the result should be trivially true, since we will get a meaning akin to: Only this helicopter will take off by way of a rotating motion.

    2. An informant tells us that “if the gesture is faster and the circular motion is smaller”, he is more likely to obtain a reading on which it is denied (informatively) that the plane will take off; in other words, in this case the manner of motion seems to be ignored. By contrast, “if the gesture is slower and the circular motion is larger”, he gets an uninformative reading on which the plane won’t take off by way of a rotating movement. It would be interesting to test whether this correlates with the at-issue vs. not-at-issue nature of the contribution (see Potts et al. 2009 for related remarks pertaining to the ‘disappearance’ of expressive—and thus not-at-issue—material under ellipsis). The difficulty is twofold, however. First, we would have to explain why in this case only in (17)b seems to be more permissive than ellipsis resolution in (16)b. Second, it does not seem to be the case that all presuppositions can be ignored in the ‘focus dimension’ under only—for if this were the case, one would not get a presupposition that the plane is on the ground.

    Let us add that an anonymous referee mentions that s/he shares the general judgments in (16)–(17), but that the two paradigms don’t seem to differ in terms of whether the presupposition can be ignored. To the referee, (16)b becomes better “when the gesture is not that salient (and performed faster and smaller)”, which mirrors our informant’s observation about (17)b.

  15. 15.

    We believe SHOOT-1 can also be used with a non-reflexive meaning, as in John threatened to SHOOT-1. But we are not sure whether the most natural realization has the gun gesture starting from the side or from a more central position. An informant can obtain this non-reflexive meaning with the gestural ‘gun’ coming from the side, but he is even more inclined to a non-reflexive interpretation when the gestural ‘gun’ is pointed from the front instead of from the side. (Importantly, for him SHOOT-1 cannot be used as a neutral form appropriate for all objects irrespective of person.)

  16. 16.

    This point was independently noticed by Amir Anvari (p.c).

  17. 17.

    The behavior of English phi-features under ellipsis and only was discussed in Sect. 2. In addition, some or all of these phi-features are often given a presuppositional treatment, although this is the subject of current debates. See for instance Cooper (1983); Schlenker (2003); Heim (2008); Sauerland (2008) and Sudo (2013) for discussion.

  18. 18.

    An anonymous reviewer correctly notes that it would be interesting to test how ASL behaves with respect to presuppositional verbs that are comparable to TAKE-OFF-ROTATING, but this is a question we have to leave for future research.

  19. 19.

    As mentioned at the outset, our results do not tell us whether the distinction is a broader one among an arbitrary number of loci, as is the case in ASL, or solely pertains to second vs. third person per se.

  20. 20.

    A pilot experiment without gestural verbs did not find ellipsis to be degraded, but this was on the basis of very different stimuli, involving written sentences rather than videos, and acceptability judgments on entire sentences.

  21. 21.

    Besides object agreement, sign language verbs can display subject agreement. These could also be investigated with gestural verbs.

  22. 22.

    Two remarks should be added. First, we keep Kuhn’s transcription, but his ONLY-ONE corresponds to what would otherwise be transcribed as ONLY-CL_one. We treat the latter expression as pronominal when it is signed in a locus that was established earlier, and thus had a prior reference; this decision should be revisited in future research. Second, in Kuhn’s video ONLY-ONE is in fact localized, and thus a more correct transcription would be: ONLY-ONEb in Kuhn’s notation, and ONLY-CL_one_b in our preferred notation; this is the reason we have added(b) as a subscript to ONLY-ONE in (45). (Thanks to J. Lamberton for discussion of this point, and to J. Kuhn for sharing his video.)

  23. 23.

    The same issues arise in examples with ellipsis. But these arguably involve independent problems: in ellipsis resolution, it has been argued that a Logical Form with a bound variable representation can give rise to a strict reading in the elided clause (Fox 2000; Schlenker 2005). This is the reason the present discussion solely appeals to strict readings under only.

  24. 24.

    Extraction from the embedded object position is even less clear—a question that we leave for future research (we leave open whether null resumptive pronouns play a role in these facts). Still, for the present discussion what matters is whether the embedded subject could move covertly out of the embedded clause. If it could, one could derive from a version of (50) the Logical Form in (i), with the (covertly) moved subject outside of the scope of the lambda-operator —which would allow it to get a strict reading without impinging on the claim that loci are variables.

    (i) [the guy who PUNCH-b] x only [your brother]b I wonder whether [\(\mathbf{t}_{\mathbf{x}}\)

  25. 25.

    Q is the manual question marker.

  26. 26.

    Importantly, if turns out that a reading can be obtained on which the direction of the movement can be disregarded under ellipsis, one will have to ask (i) whether other properties of gestures, as in our helicopter examples, can also be so disregarded, and (ii) whether whatever accounts for the special behavior of directionality could also account for the behavior under ellipsis of agreement verbs in sign language, and possibly also of phi-features in spoken language.

References

  1. Anvari, Amir. 2016. Co-nominal pointing: Toward a formal semantic analysis, MA thesis, Université Pierre et Marie Curie, Paris.

  2. Chemla, Emmanuel. 2009. Presuppositions of quantified sentences: Experimental data. Natural Language Semantics 17(4): 299–340.

    Article  Google Scholar 

  3. Cooper, Robin. 1983. Quantification and syntactic theory. In Synthese language library 21. Reidel: Dordrecht

    Google Scholar 

  4. Davidson, Kathryn. 2015. Quotation, demonstration, and iconicity. Linguistics and Philosophy 38(6): 477–520.

    Article  Google Scholar 

  5. Ebert, Cornelia, and Christian Ebert. 2014. Gestures, Demonstratives, and the Attributive/Referential Distinction. In Handout of a talk given at Semantics and Philosophy in Europe (SPE 7). Berlin, June 28, 2014.

    Google Scholar 

  6. Fox, Danny. 2000. Economy and semantic interpretation. Cambridge: MIT Press.

    Google Scholar 

  7. Gibson, Edward, Steve Piantadosi, and Kristina Fedorenko. 2011. Using Mechanical Turk to obtain and analyze English acceptability judgments: Linguistic acceptability on Mechanical Turk. Language and Linguistics Compass 5(8): 509–524.

    Article  Google Scholar 

  8. Heim, Irene. 1991. The first person. Class handout, MIT.

  9. Heim, Irene. 2005. Features on bound pronouns: Semantics or syntax? Ms, MIT.

  10. Heim, Irene. 2008. Features on bound pronouns. In Phi-theory: Phi-features across modules and interfaces, eds. Daniel Harbour, David Adger, and Susana Bejar, 35–56. Oxford: Oxford University Press.

    Google Scholar 

  11. Heim, Irene, and Angelika Kratzer. 1998. Semantics in generative grammar. Oxford: Basil Blackwell.

    Google Scholar 

  12. Jacobson, Pauline. 2012. Direct compositionality and ‘uninterpretability’: The case of (sometimes) ‘uninterpretable’ features on pronouns. Journal of Semantics 29: 305–343.

    Article  Google Scholar 

  13. Kegl, Judy. 2004. ASL syntax: Research in progress and proposed research. Sign Language and Linguistics 7(2): 173–206. Reprint of 1977 ms., MIT.

    Article  Google Scholar 

  14. Kizach, Johannes. 2014. Analyzing Likert-scale data with mixed-effects linear models: A simulation study. Tübingen, Germany. Poster presented at Linguistic Evidence 2014.

  15. Kratzer, Angelika. 2009. Making a pronoun: Fake indexicals as windows into the properties of pronouns. Linguistic Inquiry 40(2): 187–237.

    Article  Google Scholar 

  16. Kuhn, Jeremy. 2015. ASL loci: Variables or features? Journal of Semantics 33(3): 449–491.

    Article  Google Scholar 

  17. Kuhn, Jeremy. To appear. Discourse anaphora—Theoretical perspectives. In Invited contribution to Theoretical and experimental Sign Language research, eds. Josep Quer, Roland Pfau, and Annika Herrmann. London: Routledge.

  18. Liddell, Scott K. 2000. Indicating verbs and pronouns: Pointing away from agreement. In The signs of language revisited: An anthology to honor Ursula Bellugi and Edward Klima, eds. Karen Emmorey and Harlan Lane, 303–320. Mahwah: Lawrence Erlbaum Associates.

    Google Scholar 

  19. Liddell, Scott K. 2003. Grammar, gesture, and meaning in American Sign Language. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  20. Lillo-Martin, Diane. 1991. Universal grammar and American Sign Language: Setting the null argument parameters. Dordrecht: Kluwer Academic Publishers.

    Book  Google Scholar 

  21. Lillo-Martin, Diane, and Edward S. Klima. 1990. Pointing out differences: ASL pronouns in syntactic theory. In Theoretical issues in Sign Language research, Vol.1: Linguistics, eds. Susan D. Fischer and Patricia Siple, 191–210. Chicago: University of Chicago Press.

    Google Scholar 

  22. Lillo-Martin, Diane, and Richard P. Meier. 2011. On the linguistic status of ‘agreement’ in sign languages. Theoretical Linguistics 37(3–4): 95–141.

    Google Scholar 

  23. Meier, Richard. 1990. Person deixis in American Sign Language. In Theoretical issues in Sign Language research, eds. Susan D. Fischer and Patricia Siple, 175–190. Chicago: University of Chicago Press.

    Google Scholar 

  24. Mellon, Nancy K., John K. Niparko, Christian Rathmann, Gaurav Mathur, Tom Humphries, Donna Jo Napoli, Theresa Handley, Sasha Scambler, and John D. Lantos. 2015. Should all deaf children learn Sign Language? Pediatrics 136(1): 170–176.

    Article  Google Scholar 

  25. Merchant, Jason. 2014. Gender mismatches under nominal ellipsis. Lingua 151: 9–32. http://dx.doi.org/10.1016/j.lingua.2014.01.008.

    Article  Google Scholar 

  26. Norman, G. 2010. Likert scales, levels of measurement and the “laws” of statistics. Advances in Health Sciences Education 15(5): 625–632.

    Article  Google Scholar 

  27. Potts, Christopher, Ash Asudeh, Seth Cable, and Yurie Hara. 2009. Expressives and identity conditions. Linguistic Inquiry 49(2): 356–366.

    Article  Google Scholar 

  28. R Core Team. 2016. R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. Available at https://www.R-project.org/. Accessed 6 August 2017.

    Google Scholar 

  29. Rooth, Mats. 1996. Focus. In Handbook of contemporary semantic theory, ed. Shalom Lappin, 271–297. Oxford: Blackwell.

    Google Scholar 

  30. Sag, Ivan. 1976. Deletion and logical form. PhD diss., MIT

  31. Sauerland, Uli. 2008. Implicated presuppositions. In Sentence and context: Language, context, and cognition, ed. Anita Steube, 581–600. Berlin: de Gruyter.

    Google Scholar 

  32. Schlenker, Philippe. 1999. Propositional attitudes and indexicality: A cross-categorial approach. PhD diss.

  33. Schlenker, Philippe. 2003. A plea for monsters. Linguistics and Philosophy 26: 29–120.

    Article  Google Scholar 

  34. Schlenker, Philippe. 2005. Non-redundancy: Towards a semantic reinterpretation of binding theory. Natural Language Semantics 13(1): 1–92.

    Article  Google Scholar 

  35. Schlenker, Philippe. 2009. Local contexts. Semantics and Pragmatics 2(3): 1–78. doi:10.3765/sp.2.3.

    Google Scholar 

  36. Schlenker, Philippe. 2014. Iconic features. Natural Language Semantics 22(4): 299–356.

    Article  Google Scholar 

  37. Schlenker, Philippe. 2015a. Gestural presuppositions (squib). Snippets 30: 9–11. doi:10.7358/snip-2015-030-schl.

    Article  Google Scholar 

  38. Schlenker, Philippe. 2016. Featural variables. Natural Language and Linguistic Theory 34(3): 1067–1088.

    Article  Google Scholar 

  39. Schlenker, Philippe. To appear c. Iconic pragmatics. Natural Language and Linguistic Theory. Available at http://ling.auf.net/lingbuzz/003215. Accessed 6 August 2017.

  40. Schlenker, Philippe. 2017. Sign Language and the foundations of anaphora. Annual Review of Linguistics.

  41. Schlenker, Philippe. To appear a. Super monsters II: Role shift, iconicity and quotation in Sign Language. To appear in Semantics and Pragmatics.

  42. Schlenker, Philippe. To appear b. Gesture projection and cosuppositions. Accepted with minor revisions in Linguistics and Philosophy.

  43. Schlenker, Philippe, and Gaurav Mathur. 2013. A strong crossover effect in ASL (squib). Snippets 27: 16–18. doi:10.7358/snip-2013-027-schl.

    Google Scholar 

  44. Schlenker, Philippe, Jonathan Lamberton, and Mirko Santoro. 2013. Iconic variables. Linguistics and Philosophy 36(2): 91–149.

    Article  Google Scholar 

  45. Spathas, Giorgos. 2007. Interpreting gender features on bound pronouns. In North East Linguistic Society (NELS) 38, eds. Anisa Schardl, Martin Walkow, and Muhammad Abdurrahman. Amherst: GLSA.

    Google Scholar 

  46. Sprouse, Jon, and Diego Almeida. 2012. Assessing the reliability of textbook data in syntax: Adger’s Core Syntax. Journal of Linguistics 48(3): 609–652.

    Article  Google Scholar 

  47. von Stechow, Arnim. 2004. Binding by verbs: Tense, person and mood under attitudes. In The syntax and semantics of the left periphery, eds. Horst Lohnstein and Susanne Trissler, 431–488. Berlin: Mouton de Gruyter.

    Google Scholar 

  48. Strickland, Brent, Carlo Geraci, Emmanuel Chemla, Philippe Schlenker, Meltem Kelepir, and Roland Pfau. 2015. Event representations constrain the structure of language: Sign language as a window into universally accessible linguistic biases. In National Academy of Sciences (PNAS), 112(19). Available at www.pnas.org/cgi/doi/10.1073/pnas.1423080112. Accessed 6 August 2017.

    Google Scholar 

  49. Sudo, Yasutada. 2013. On the semantics of Phi features on pronouns. PhD diss., MIT.

Download references

Acknowledgements

Many thanks to Amir Anvari, Masha Esipova, Gabe Greenberg, Rob Pasternak and Lyn Tieu for discussion. Rob Pasternak was immensely patient with discussions of English judgments — special thanks to him. References were prepared with Lucie Ravaux’s help; she also checked averages of ASL judgments. Thanks also to Brent Strickland, who provided initial practical help with the experiment that was run on Mechanical Turk.

We are particularly grateful to three anonymous reviewers for Natural Language and Linguistic Theory, who made very constructive remarks and criticisms.

ASL consultant: Jonathan Lamberton. Special thanks to Jonathan Lamberton, who has provided exceptionally fine-grained ASL data throughout the research reported here. He also checked transcriptions and translations.

English videos: special thanks to Jeremy Kuhn, who kindly re-recorded video stimuli prepared by one of the authors.

Grant acknowledgments:

Schlenker and Chemla: Research was conducted at Institut d’Etudes Cognitives, Ecole Normale Supérieure – PSL Research University. Institut d’Etudes Cognitives is supported by grants ANR-10-LABX-0087 IEC et ANR-10-IDEX-0001-02 PSL*.

Schlenker: The research leading to these results received funding from the European Research Council under the European Union’s Seventh Framework Programme (FP/2007-2013) / ERC Grant Agreement N°324115–FRONTSEM (PI: Schlenker).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Philippe Schlenker.

Electronic Supplementary Material

Appendix: Experimental investigation: Earlier pilots

Appendix: Experimental investigation: Earlier pilots

Earlier and failed or inconclusive pilots had the following general properties. Test sentences remained the same across pilots, but included the following differences:

Pilot 1 (created June 30, 2016): the acceptability question was of the form: “How natural is this sentence, as realized on the video right above? (0 = worst; 10 = best).” Instructions did not include videos varying the height of the gestures, as in the final experiment. The task included semantic questions about whether ‘you understand the speaker to be referring to actions performed in non-standard ways?’. This was intended to detect judgments on which, say, SLAP-a with a second person object argument is considered as acceptable, but to mean that the slapping was done from the side. The question was hard to understand, and turned out not to be necessary, as in the final experiment clear acceptability differences were found across the target sentences.

Pilot 2 (created July 10, 2016): related to Pilot 1, but asked subjects: “How well do the gestures fit with the rest of the sentence?”.

Pilot 3 (created July 15, 2016): this was a control experiment, replacing videos with written sentences without gestures, e.g. Your brother, I am going to slap, then you, I am going to punch, vs. Your brother, I am going to punch, then you, too. The goal was to determine whether ellipsis might on its own be degraded, which could have been combined with the results of Pilot 1 to obtain sharper judgments. Results did not seem promising (but as mentioned in the text, the independent contribution of ellipsis should be assessed in future research).

Pilot 4 (created August 5, 2016): related to Pilot 2, but with presentation of the sentences by pairs (overt person match vs. ellipsis, overt person mismatch vs. ellipsis) rather than by triples. Questions about “actions performed in non-standard ways” were eliminated.

Pilot 5 (created August 6, 2016): related to Pilot 2, but with modified instructions, related to those used in the final version of the experiment. Subjects were explicitly asked to “pay close attention to the shape of the gesture and to the direction of the movement”, and questions were of the form: “How well does the 1st gesture fit with the meaning of the sentence?”, “How well does the 2nd gesture with the meaning of the sentence?”, “How well does the gesture fit with the meaning of the sentence?”.

Pilot 6 (created August 8, 2016): related to Pilot 5, but presentation was by triples.

The final experiment (created August 22, 2016, and reported in the main text) was related to Pilot 6, except that each subject saw two versions of the experiment: they saw the ‘local’ condition used in Pilots 5 and 6, and they also saw the ‘global’ condition in which the main question pertained, in the plural, to whatever gestures appeared in the sentence (= “How well do the gestures fit with the meaning of the sentence?”).

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Schlenker, P., Chemla, E. Gestural agreement. Nat Lang Linguist Theory 36, 587–625 (2018). https://doi.org/10.1007/s11049-017-9378-8

Download citation

Keywords

  • Agreement
  • Gestures
  • Sign language
  • Agreement verbs
  • Gestural verbs
  • Pro-speech gestures
  • Ellipsis
  • Focus
  • Iconicity