Advertisement

Corpus Pragmatics

, Volume 2, Issue 1, pp 27–55 | Cite as

Tense, Grammatical Aspect and Subjectivity: An Experimental Study Using Inter-Annotator Agreement Rates and Corpus-Based Data

Original Paper

Abstract

In this paper, I investigate experimentally the question of subjectivity and its supposed triggering by the categories of tense and grammatical aspect. The study is carried out in the relevance theoretic pragmatic framework, which assumes that certain linguistic expressions encode procedural information constraining the determination of the explicit content of an utterance (that is, the explicature), and of the implicatures (that is, implicit premises and implicit conclusions). In the current state of the art, the notion of subjectivity, which roughly means the expression of a point of view or perspective, has been correlated to a series of linguistic expressions, such as deictic elements (personal, spatial, temporal), grammatical aspect and connectives. Here, I examine the relation between subjectivity and two parameters of temporal reference: verbal tenses and grammatical aspect. Annotation experiments were carried out on corpus data in English, French and Serbian in order to test whether native speakers are able to consciously identify and evaluate information about subjectivity in corpus data. Based on the results of these experiments and using the notion of (specific) procedural versus general (pragmatic) inference, I discuss the status of subjectivity as semantic (encoded procedural information) or pragmatic (general inferential) information, and give evidence in favour of the latter.

Keywords

Subjectivity Verbal tenses Grammatical aspect Corpus data Annotation experiments Cross-linguistic study 

Introduction

The notion of subjectivity is highly used in the literature to account for various linguistic phenomena, among which the pragmatic uses of verbal tenses and pragmatic interpretations of the perfective and imperfective grammatical aspect are of interest in this paper. Subjectivity has been defined as the type of construal relation built in the discourse (in Langacker’s Cognitive Grammar 1996, 2006), the expression of the self in literary criticism (e.g. Banfield 1982/1995), the expression of the speaker’s or a third party’s perspective in linguistic and pragmatic studies (e.g. Traugott 1989, 1995, 1999), and the speaker’s evidence for her claims in computational linguistics (e.g. Wiebe 1994) and psycholinguistics (e.g. Sanders et al. 1992, 1993; Sanders 2005). Despite this heterogeneity, scholars agree that subjectivity in language refers to the speaker’s involvement in the description of situations (also called empathy by Kuno and Kaburaki 1975; Kuno 2004). In this study, I investigate experimentally and from a cross-linguistic perspective two types of linguistic cues of subjectivity, which are verbal tenses and grammatical aspect. The research is carried out in naturally occurring data coming from parallel corpora. Using offline experiments with linguistic judgement task, I have tested whether subjectivity is identifiable with a reliable agreement rate by native speakers.

This paper is structured as follows.1 Section “Different approaches to subjectivity” critically discusses the various approaches to the issue and points out the necessity to have a definition of subjectivity that allows empirical testing. This definition and its operationalization are presented in “A working definition of subjectivity in this study” section. Section “Experimental work” is dedicated to the experimental work carried out. The results of the experiments and the integration of these results into a pragmatic model of verbal tenses are given in “Subjectivity: a pragmatic feature” section. Section “Conclusion” concludes this paper and offers a series of possible developments of the present research in future work. Additionally, the Appendix contains the English version of the annotation guidelines used in the experiments.

Defining Subjectivity

Different Approaches to Subjectivity

The notion of subjectivity has been conceived of in different ways in the literature. The array of definitions of subjectivity goes from a subjective vs. an objective construal relation in Langacker’s Cognitive Grammar (Langacker 1991, 2006; Trnavac 2006), to subjectivity as a cognitive principle in T. Sanders’ cognitive approach to connectives and discourse relations (Sanders et al. 1992, 1993; Pit 2003; Sanders et al. 2012; Stukker and Sanders 2012), to the expression of an experiencing vs. narrating self in literary criticism in general (Genette 1972; Fleischman 1990), linked to a certain discourse type, such as the Free Indirect Discourse (FID) (Banfield, 1982/1995; Reboul 1992; Schlenker 2004; Reboul et al. 2016; Eckardt 2014), or linked to types of modes of discourse, such as narrative, description, report, information and argument (Smith 2003). A series of studies adopting a linguistic or a pragmatic perspective define subjectivity as the expression of the speaker’s or of a third party’s perspective (Benveniste 1966; Lyons 1982, 1995; Traugott 1989, 1995, 1999; Sthioul 2000; Tahara 2000; Saussure 2013). Finally, subjectivity defined as the speaker’s evidence for her claims was identified as an important factor for discourse processing both for machines (Wiebe 1994, Wiebe et al. 1999; Chen 2008) and for humans (Sanders 2005; Canestrelli et al. 2013; Zufferey and Gygax 2016).

Benveniste (1966/1971) and Lyons (1982) pointed out that languages provide the speaker with the linguistic means to express her attitudes and beliefs. For them, subjectivity is expressed at the level of pronominal and temporal deixis. Benveniste predicted that verbal tenses are used according to the type of enunciation and personal pronouns: the simple past, the imperfect and the pluperfect with the 3rd personal pronoun are used in the historical enunciation (i.e., past time events), whereas the present, the past compound and the future are excluded. As for discursive enunciation, all verbal tenses may be used with the exception of the simple past. Later on, adopting a pragmatic approach, Moeschler (2014) showed with the help of the historical present that Benveniste’s view of subjectivity and its link to verbal tenses and personal pronouns is problematic. He suggested that subjectivity is a pragmatic component of language, and that verbal tenses have subjective and non-subjective usages.

Banfield (1982) was the first to reject Benveniste’s proposal. She showed that subjectivity can be identified for other verbal tenses (such as the imperfect) and other pronouns (such as the third person pronoun), specifically in their usage in the FID (containing represented speech and thought). Banfield based her distinction between objective and subjective sentences on discourse structure (reported speech that shares features with both direct and indirect speech). Subjective sentences integrate the consciousness of an experiencing character (using Banfield’s terminology) within the description of a series of situations, thus expressing an individual’s evaluations, emotions, judgments, uncertainties, beliefs and attitudes. Subjective sentences may express an individual’s thought or perception called represented thought or represented perception, and an individual’s private state (seeing, wanting, feeling).

For Banfield, the difference between French sentences in (1) containing a passé simple (simple past) and in (2) containing an imparfait (imperfect) (both sentences are translations into French of the same English sentence containing a simple past) lies in the absence or presence, respectively, of a “self at a moment corresponding to an act of consciousness” (Banfield 1982, 158; also Fleischman 1990, 1995). The utterance in (1) objectively describes the seeing of the moon by a narrating-self, while the utterance in (2) implies that the event of the seeing of the moon is experienced at a moment by an experiencing-self. In other words, the passé simple conveys the objective narration of the seeing of the moon event and the imparfait conveys the subjective experiencing interpretation.
  1. (1)

    Elle vit la lune.

    ‘She saw the moon.’

     
  2. (2)

    Elle voyait la lune (maintenant)

    ‘She saw the moon (was seeing the moon now).’

     
Hence, the passé simple is non-experiential and entirely detached from the speaker (Fleischman 1990, 31). Consequently, all other tense-aspect categories used in narratives involve an experiencing-self at some degree of detachment depending on each verbal tense. However, the deterministic relation passé simple—non-subjective and imparfait—subjective was contested by several scholars adopting a pragmatic approach, such as Sthioul (1998, 2000), Tahara (2000), Saussure (2003, 2013). Sthioul (2000) Moeschler et al. (1998) argued that the French passé simple points to the moment when an individual becomes aware of the situation under description as in (3) and (4). The reader understands by inference that the situation described existed before the moment of awareness. These examples show the subjective usage of the passé simple.
  1. (3)

    Paul sortit. Dehors, il fit bigrement froid.

    ‘Paul went out. Outside, it was fantastically cold.’

     
  2. (4)

    La route sortit de la forêt. (Vailland, 325,000 francs)

    ‘The road left the forest.’

     
Other scholars (Reboul 1992; Vuillaume 1990; Tahara 2000) extended subjectivity to include the second person pronoun and argued that subjectivity can occur in narratives that are not FID, such as in (5). In the following fragment, the italicized verbs are in the preterit form (the passé simple) and they express the advancement of time seen from Emma’s point of view (she was terrified and exhausted).
  1. (5)

    – Monsieur vous attend, madame, la soupe est servie. Et il fallut descendre! Il fallut se mettre à table! Elle essaya de manger. Les morceaux l’étouffaient. (Flaubert, Madame Bovary)

    ‘– Sir (Charles) is waiting for you, madam; the soup is served. And she had to go downstairs! She had to sit to the table! She tried to eat. The bites of food suffocated her.’

     
Moreover, Traugott’s work on subjectivity (1989, 1995, 1999) pointed to two central ideas defended in the linguistic and pragmatic approaches: (i) subjectivity arises when the speaker’s psychological or emotional perspective is incorporated into the description of a situation, and (ii) subjectivity is a context-dependent feature. For Traugott and Dasher (2002, 98), ‘most frequently, an expression is neither subjective nor objective in itself; rather the whole utterance and its context determine de degree of subjectivity’.
Langacker’s Cognitive Grammar (1991, 2006) adopts a conceptual semantics based on human experience and on our capacity to construct and to mentally represent situations differently at various occasions. Hence, linguistic meaning embodies a particular way of constructing a situation. For Langacker, a speaker establishes a construal relationship with two elements: the conceptualizer—who is part of the ground, including the speech participants and the speech act—and the object of conceptualization. Usually, the viewpoint of the conceptualizer is reflected in the linguistic expression. This can be done implicitly, as in the case of verbal tenses, which indirectly indicate the temporal relationship between the ground and the event, or explicitly as in (6), where the conceptualizer is ‘onstage’ and his relation with the object of conceptualization is explicit. In this case the construal is objective. The construal is subjective when the presence of the conceptualizer and his viewpoint are expressed implicitly, as in (7) (Langacker 2002, 19). However, for authors adopting the linguistic and pragmatic approach, these two examples are both objective.
  1. (6)

    Mulroney was sitting across the table from me.

     
  2. (7)

    Mulroney was sitting across the table.

     
For Langacker, deixis implying reference to the ground is a crucial element for determining subjectivity. Nevertheless, in this framework, first person reference is used in objective construals, whereas it is a central component for the expression of subjectivity in linguistic and pragmatic approaches (Narrog 2012).

In a theory of cognitive discourse representation, Sanders et al. (1992, 1993) treat subjectivity as a fundamental cognitive and discursive principle, which accounts not only for use of causal relations and their linguistic properties, but also for the cognitive complexity of discourse connections in language acquisition and discourse processing. Subjectivity is understood as the degree to which the conceptualizer—the person responsible for the causal relation, who may be the speaker or an agent—is present in the utterance (Spooren et al. 2010). Pit (2007) suggested that the crucial element determining whether a causal relation is subjective or not, is its locus of effect, which is the participant or entity around whom/which causality is centred. If the locus of effect coincides with the speaker or the author, the causal relation is subjective. Pit (2003) found that the Dutch objective connective omdat ‘because’ occurs more frequently with objective pluperfect or simple past verbal tenses, whereas the subjective want ‘for’ (causal) was more often combined with the subjective simple present and the future. Sanders and Redeker (1996) found that sequences of sentences in the simple past imply an objective, distanced view of the narrator, whereas the simple present tense is more lively and expressive, ‘as if the narrator or character gives an eyewitness report’ (Sanders 2005, 108).

Since Comrie’s seminal work on aspect, the imperfective aspect has been associated with subjective interpretations, while perfective aspect has been connected with objective interpretations (Comrie 1976; Caenepeel 1989). The imperfective aspect takes an “internal perspective”, focusing on a sub-interval of an ongoing event. This is interpreted as pointing to the speaker’s experiencing mind. The perfective aspect, on the other hand, presents the situation from an external perspective by offering an understanding of the situation as a whole (Comrie 1976). The main assumption is that the relation between grammatical aspect and modality involves subjectivity: the imperfective aspect expresses perspectivized and subjective information in narratives, where it can represent speech and thoughts of an individual other than the speaker (Caenepeel 1989; Fleischmann 1995; Smith 1997; Boogaart 1999, 2007; Trnavac 2006). Trnavac (2006) investigates this hypothesized correlation in a corpus-based study on Russian, Serbian, Dutch and German. Using Langacker’s notion of subjectivity, she found that modal interpretations may be expressed through both imperfective and perfective aspects in tensed and in aspect-prominent languages. However, subjective modal meanings most often correlate with the imperfective aspect.

This literature review shows two main problems regarding the notion of subjectivity. The first one is that subjectivity was studied from a theoretical point of view, with the exception of Trnavac’s (2006) corpus-based investigation. In her study, subjective and non-subjective usages of grammatical aspect are identified by the author herself by three tests in order to estimate degrees of subjectivity. This is problematic because naïve speakers might not have the same interpretations of the corpus data. The second one is the fact that the notion of subjectivity is understood and used differently from one study to another. Particularly, this can be seen when comparing linguistic, pragmatic and Sanders’ cognitive standpoints to Langacker’s, in which the concepts of subjectivity, perspective and viewpoint result in different evaluations of utterances (as shown in examples (6) and (7)). For this reason, in this research, I will not further consider Langacker’s approach to subjectivity. Despite this variety of assessments of subjectivity, a common idea which seems to emerge is that it refers to the speaker’s involvement. Linguistic and pragmatic approaches pointed to a series of linguistic expressions which may be more or less correlated with subjective and non-subjective interpretations. Hence, in this study, I will pursue this path and will experimentally test in a systematic manner the predictions made by scholars for French and English verbal tenses and for grammatical aspect.

A Working Definition of Subjectivity in This Study

As we have seen, both grammatical and lexical forms may give rise to subjective interpretations. Specifically, three main areas of linguistic sources of subjectivity were identified. The first is deixis and its linguistic expression, which mainly includes reference to the here and now in language. The second is modality and evidentiality, in which subjectivity is a characteristic of mental-state predicates, modal verbs, and connectives that express the speaker’s degree of commitment towards what is said (cf. Sanders and colleagues’ analysis of causal connectives in Dutch). The third category consists of forms expressing personal perspective, such as personal and reflexive pronouns, and grammatical aspect. For Smith (2003), these various types of linguistic cues of subjectivity should be categorized into four classes: communication (verbs such as say, ask, request, confess), contents of mind (verbs such as believe, assume), evaluation and evidentials (anyway, still, but, and respectively, frankly, clearly, surprising, seem) and perception and perspectival cues (direct and inferred perception, temporal and spatial deictics, reflexive and possessive pronouns, and the imperfective viewpoint). I think that her classification of sources of subjectivity is a useful tool for the type of analysis I am interested in. This research targets verbal tenses and grammatical aspect, which are listed in Smith’s fourth category. For this reason, in the present analysis I disregard the first three categories, which should be considered in further research.

In this research, the account of subjectivity is based on previous descriptions and proposals and on empirical validation through offline experiments with linguistic judgment task. According to Smith (2003), subjectivity should be used as a generic term that integrates the notions of perspective and viewpoint. Here, I follow Smith and use subjectivity as a generic term for all subjective manifestations, be it grammatical, lexical or discursive. I use the notion of viewpoint to refer to cases when subjectivity is closely related to grammatical aspect (perfective vs. imperfective) and the notion of perspective for cases when subjectivity is linked to the category of tense.

We define subjectivity as the speaker’s viewpoint, psychological perspective and perceptions, which might or might not be included in the description of a situation or series of situations. Hence, the speaker is the responsible source (Smith 2003), which can be the author (as in fiction) or the communicator/participant in the text situation, and the locus of effect (in Pit’s terminology) of subjectivity. In subjective sentences, the description of a situation or series of situations is centred on the speaker’s subjectivity. In non-subjective sentences, the speaker describes a situation or series of situations without including her viewpoint, psychological perspective and perceptions. This definition of subjectivity was operationalized in this study as the [±subjectivity] feature.

Experimental Work

The speaker’s ability to judge subjectivity was tested experimentally for French and English verbal tenses expressing past time, and for grammatical aspect in Serbian. In this study, I tested hypotheses formulated based on previous research regarding subjective and non-subjective usages of French and English verbal tenses and of grammatical aspect. Therefore, I tested these same languages. Explicitly, the first studies that proposed a link between subjectivity and verbal tenses focused on the French passé simple and imparfait and the English simple past (Banfield 1982; Fleischman 1990, 1995; Sthioul 1998, 2000). As for Serbian, it is one of the two Slavic languages investigated in the first empirical study investigating the relation between subjectivity and grammatical aspect expressed morphologically (Trnavac 2006).

In this study, three experiments were carried out, in which native speakers were asked to judge whether occurrences of target verbal tenses (as described below) express or not the speaker’s subjectivity:
  • Experiment 1 English simple past (SP) and the [±subjectivity] feature.

  • Experiment 2 French passé composé (PC), passé simple (PS) and imparfait (IMP) and the [±subjectivity] feature.

  • Experiment 3 Serbian grammatical aspect and the [±subjectivity] feature.

The main hypothesis is that in the interpretation process, the hearer makes inferences regarding the speaker’s subjective perspective on the eventualities described (states and events) based on the information provided by verbal tenses and grammatical aspect. If subjectivity is triggered by linguistic markers analysed, I expect to find statistically significant correlations between the French IMP and the imperfective aspect in Serbian and subjective interpretations, as well as between the French PC and PS, the English SP and the perfective aspect in Serbian and non-subjective interpretations. By contrast, if subjectivity is not triggered by linguistic cues, and therefore, it is a pragmatic feature, the results of the experiments will not show statistically significant correlations between verbal tenses, grammatical aspect and the two values of the [±subjectivity] feature.

Methodology and Procedure

The experimental work carried out follows the design of what is called sense-annotation experiments in the field of Computational Linguistics (CL). Roughly, this method consists in asking two or more native speakers to categorize sentences or excerpts into two or more theoretical categories according to certain annotation guidelines. This method is used in CL to produce human annotated resources, which are further considered as gold-standard data 2 for training, testing and evaluating the performances of automatic tools. In order to create this perfect human annotated data, it is required that annotators have very high agreement rates. Several solutions have been proposed by scholars (e.g. Bayerl and Paul 2011; Spooren and Degand 2010; Scholman et al. 2016) in order to increase the agreement rates, such as (i) intensifying the training when participants are inexperienced, also called naïve speakers (ii) use highly skilled and experienced annotators, (iii) do the annotation in several phases, or (iv) use several types of post hoc techniques such as the double coding (that is, the discussion of disagreements and explicitation of the judgments made) and one-coder-does-all (that is, one annotator is responsible for coding the entire corpus). For example, Wiebe et al. (1999) developed a gold-standard data set for subjectivity. In their study, subjectivity was defined with respect to Smith’s (2003) third class of sources, that is evaluation and evidentiality. In order to increase inter-annotator agreement rates, two of the annotators were the researchers themselves and two other highly trained speakers. The annotation procedure consisted of several steps, in which annotators returned to the data several times. Following this procedure, inter-annotator agreement rate was moderate.

However, my suggestion is that this kind of experiments may be used as well to investigate naïve speakers’ intuitions about a linguistic or pragmatic phenomenon, without necessarily aiming at producing gold-standard data. One advantage of this type of experiment is that it is ecological because it reduces the degree of artificiality that characterizes the more classical type of experiments originating in psychology both with respect to the procedure and to the experimental items. There are two drawbacks to sense-annotation experiments and the evaluation of their results. The first is agreement by chance and the second is annotator bias. I will discuss them below.

Agreement by chance refers to the amount of agreement we would expect to occur by chance, that is, if annotators made a decision without taking into account the annotation guidelines. This amount depends on the number and the frequency of categories to be assigned. For example, for two equally frequent categories there are 50% of chances that when one annotator makes a decision, the second one makes the same decision. In order to avoid this problem, chance-corrected statistical coefficients can be used to measure inter-annotator agreement rates. These coefficients calculate the agreement due to chance and the agreement which goes beyond chance level. This study provides the results of the experiments as evaluated with one the most frequent measures used in CL, which is the kappa coefficient (henceforth, Қ) (Carletta 1996). Its values range between 0 (lack of agreement other than due to chance) to 1 (perfect agreement).

The second drawback of sense-annotation experiments refers to the fact that the annotators may apply individual annotation strategies. For a study in which annotation experiments are used to investigate the speakers’ intuitions regarding the phenomenon of interest, annotator bias in itself is not necessarily problematic. As Spooren and Degand (2010, 254) point out when speaking about the one-coder-does-all strategy,

Of course the coding will be subject to individual strategies developed by the coder, but these strategies will presumably be systematic and there is no reason to assume that such strategies will be conflated with the phenomenon of interest. […] So if our research question is whether judgements3 occur more often with want than with omdat¸ an overcoding of judgments will not impede answer to the research question.

This means that the annotator’s strategy corresponds to his/her way of understanding the phenomenon of interest. The problem of annotator bias, which is a measure for variance in the data, comes from the fact that it influences the way in which chance-corrected agreement is calculated. In other words, the value of the coefficient slightly depends on how chance-corrected agreement is calculated [cf. Artstein and Poesio (2008) for a more detailed technical discussion]. When there are more than two annotators, the effect of the variance of their individual annotation strategies on the value of the chance-corrected coefficient becomes more similar to random noise.

All the three experiments discussed in this paper were carried out following similar procedures. Participants received annotation guidelines (cf. Appendix for the English version), which included the definition of subjectivity, the definition of subjective and non-subjective sentences, Smith’s (2003) fourth category of sources of subjectivity, and examples for each type of source. They passed through training on 7–10 occurrences of the target verbal tense, in which disagreements were discussed in a debriefing session. The debriefing session was an open discussion, in which the annotator(s) explained their decisions for each disagreement case and verified if they understood the annotation guidelines correctly. They annotated the data by working independently. The disagreements were not discussed after the annotation experiments. Since there were three or more participants for each experiment, pairwise agreement and mean Қ values were calculated (using the Қ values for each pair of annotators). Pairwise agreement for each item is the “proportion of agreeing judgement pairs out of the total number of judgement pairs for that item” (Artstein and Poesio 2008, 562). In other words, for each occurrences of the target verbal tense, the majority of decisions given by the annotators for that item was calculated. Experiments 1 and 2 were preceded by pilot experiments, which were carried out following the same procedure as in the experiments themselves.

The data used in the first two experiments is corpus data. All the data was randomly chosen from the two parallel corpora built by Grisot (2015). The first is the English-French parallel corpus, containing texts written in English and their translations into French. The second is the French–English parallel corpus, containing texts written in French and their translations into English. Except for the English-Serbian sub-corpus, translations were not made for research purposes. Translation corpora were used instead of comparable corpora for two reasons. The first is that they make it possible to control for the cotext and context across languages, which are relevant parameters for judging subjectivity. The second is that they give access to cross-linguistic semantic and pragmatic equivalences for specific linguistic expressions (Dyvik 1998; Noël 2003). Each occurrences of the target verbal tense consisted of a short excerpt, as in examples (8)–(9), (14)–(19) and (20)–(23), for which the context provided was considered sufficient for judging the [±subjectivity] feature and its triggering by the linguistic cues investigated.

Both parallel corpora contain texts of three stylistic registers in equivalent proportions: literature, journalistic and discussions from the European Parliament (the Europarl corpus4). The texts were randomly chosen from existing multilingual corpora or already existing translations of the original texts, as follows. The literature sub-corpus contains texts from The portrait of Mr. W. H. by O. Wilde, Sense and sensibility by J. Austen, Le petit prince by A. St. Exupéry, Cinq semaines en ballon and Vingt mille lieues sous les mers by J. Verne. The journalistic sub-corpus contains texts from the News Commentaries corpus, Time Magazine, VoxEurop and Le monde diplomatique. The Europarl sub-corpus contains texts from the Europarl corpus (Koehn 2005). Finally, the data used in Experiment 3 contains randomly selected examples from the SP sub-corpus (originating in the English-French parallel corpus), which was translated into Serbian by a Serbian native speaker, who was not aware of the way in which the translation was going to be used. The decision to produce an English-French-Serbian parallel corpus is justified by the lack of an already existing multilingual corpus, in which these three languages are represented.

The Қ coefficient was used to evaluate the results regarding the type of information encoded by the feature tested, more precisely, procedural (specific inference triggered by a linguistic marker) or purely pragmatic (general non-demonstrative inference). In Grisot (2014), I propose a scale for interpreting inter-annotator agreement rates measured with the Қ coefficient with respect to the type of linguistic or pragmatic information that is dealt with. More precisely, the scale and its interpretation is the following:
  • High agreement (indicative minimal threshold for Қ values 0.7) signal that the information judged is easily accessible to consciousness and available to conscious thought, hence it is conceptual information.

  • Medium agreement (indicative thresholds for Қ values 0.4–0.7) signal that the information judged is not easily accessible to consciousness and available to conscious thought, hence it is procedural information.

  • Low agreement (indicative maximal threshold for Қ values 0.4) signal that the information judged depends uniquely on contextual hypotheses that the addressee formulates, hence it is purely pragmatic information recovered through non-demonstrative inferences.

This scale is based on a series of annotation experiments targeting verbal tenses (Grisot 2014, 2015). Their results have indicated that systematically when participants deal with the conceptual meaning of a linguistic expression, they have no difficulties in consciously evaluating it, that it is available to conscious thought and it is easily expressed in conceptual terms. On the other hand, when they deal with the procedural meaning encoded by a linguistic expression, the procedure is automatically executed regardless of the contextual assumptions. This procedure leads to a specific pragmatic inference whose result depends on the contextual assumptions formulated. Consciously evaluating this type of meaning, which is not available to consciousness, is a rather difficult task for the annotators. This is shown by their systematic behaviour when judging procedural information: the inter-annotator agreement rates measured with Қ are between 0.4 and 0.7. For example, in Grisot (2015) I show that two experiments carried out using the same protocol as that presented in this paper confirmed theoretical proposals regarding the type of meaning encoded by grammatical and lexical aspect. More precisely, it is suggested that grammatical aspect encodes procedural information whereas lexical aspect referring to ontological types of situations encodes conceptual information (cf. Escandell-Vidal and Leonetti 2011). When grammatical and lexical aspects are judged in annotation experiments, annotators’ behaviour is completely different. For the former, they had difficulty in consciously thinking of what perfective and imperfective aspects mean and in identifying them for verbal tenses in a language where they are not morphologically expressed. For the latter, annotators were able to identify the type of situation verbal phrases express with very high agreement rates (Қ value of 0.084). In the debriefing session, all the disagreements were resolved (Қ value of 1).

Experiment 1

The purpose of this experiment was to test whether subjectivity, as defined in “A working definition of subjectivity in this study” section, can be identified in English in a consistent manner by native speakers when they are asked to judge corpus data. Scholars suggested that when a speaker uses a SP, she describes the situation in an objective manner, that is, without including her viewpoint, psychological perspective and perceptions. Assuming that speakers are able to reliably identify subjectivity when judging sentences containing the SP in English, and that the SP cannot be used to express subjectivity, then all SP occurrences from the data will be judged as non-subjective.

There were three participants (3 women, mean age 33, range 19–60), who did not have prior knowledge of linguistics. They are native speakers of English: annotator 1 comes from New Zealand, annotator 2 from Australia and annotator 3 from Scotland. The three of them have a university degree. It was the first time that they participated in this type of experiment. They received a small amount of money for their participation.

The material used consisted of 99 excerpts containing instances of the SP, which were randomly chosen from the English side of Grisot (2015) parallel corpus (21% of the total number of excerpts in this corpus). The excerpts are similar to those provided below, where the SP written in italics in (8) has a non-subjective usage and the SP written in italics in (9) has a subjective usage.
  1. (8)

    The military recently blocked a government move to place Pakistan’s infamous intelligence service, the ISI, under the control of the interior minister rather than the prime minister. (Journalistic sub-corpus).

     
  2. (9)

    She had an excellent heart—her disposition was affectionate, and her feelings were strong—but she knew how to govern them: it was a knowledge which her mother had yet to learn; and which one of her sisters had resolved never to be taught. (Literature sub-corpus).

     
The material contained 21 excerpts from the Europarl sub-corpus, 31 from the Journalistic sub-corpus and 47 from the Literature sub-corpus. For all excerpts, English was the original language.

The experiment was carried out according to the procedure described in “Methodology and procedure” section. The evaluation of the results was performed by calculating pairwise agreement for each item and the Қ value for each pair of annotators.

The results are as follows. The mean Қ value is 0.0508 (0.0045 for the pair A1–A2, 0.0479 for the pair A1–A3 and 0.10 for the pair A2–A3). This Қ value is close to chance agreement. In other words, in a systematic manner, the annotators did not agree when they were asked to identify subjectivity at it was defined in this research and use the perspective and perception list of linguistic sources.

After performing pairwise agreement, the SP was judged as being objective in 55 cases (56%) and subjective in 44 cases (44%). The results of the analysis of subjective and non-subjective usages of the SP with respect to the stylistic register are given in Table 1. Using a Chi Square test, this distribution is shown to be statistically significant (χ2 = 10.79, df = 2, p < 0.05). The analysis of standardized residuals for each cell shows that the significance comes from the literature register, where the number of observed non-subjective sentences is smaller than the expected one whereas the number of subjective sentences is larger than the expected one. Hence, the judgments performed on the literary texts are statistically different than the judgements made on the excerpts from the journalistic sub-corpus and the Europarl sub-corpus.
Table 1

Distribution of subjective and non-subjective usages of the SP type of sub-corpus

 

Subjective

Non-subjective

Total

Abs. freq.

Row (%)

Abs. freq.

Row (%)

Europarl

6

29

15

71

21

Journalistic

9

29

22

71

31

Literature

29

62

18

38

47

Total

44

 

55

 

99

In 26 cases, all the three annotators agreed on the label provided (that is, they made the same judgment). Among these, the SP was judged as being objective in 18 cases (69%) and subjective in 8 cases (31%).

Among the SP occurrences that received an identical label, the sources of subjectivity mentioned by the annotators are very heterogeneous, ranging from type of discourse, type of perception and type of information (factual or personal opinion). In very few cases, the judges said that they based their decision on a perspectival cue given in the annotation guidelines (that is, temporal or spatial deictics, reflexive and possessive pronouns and imperfective viewpoint). This qualitative analysis seems to point to the fact that subjectivity may arise from a large array of linguistic and non-linguistic sources and that it cannot be reduced to Smith’s fourth class of sources. This was also shown by a pilot experiment that was carried out following the same methodology and the same procedure, with two different judges and 60 new excerpts containing SP occurrences. In the pilot experiment, all of Smith’s (2003) four types of sources of subjectivity were considered (communication verbs, contents of mind, evaluation and evidentials, and perception and perspective). The agreement rate corresponds to a Қ value of 0.22. As it can be seen, when subjectivity is not reduced only to one type of sources, annotators have slightly better results. However, this Қ value remains under the threshold of 0.4 from the scale proposed in Grisot (2014).

Experiment 2

The purpose of Experiment 2 was to test whether French native speakers are able to consistently identify subjectivity when judging corpus data containing occurrences of PS, PC and IMP. The second aim was to test the existing theoretical assumptions regarding the link between these verbal tenses and subjectivity. Precisely, in the literature it is suggested that the PS has more frequently a non-subjective usage as in examples (10) and less frequently a subjective usage as in (11).
  1. (10)

    Max entra dans le bar. Il alla s’asseoir au fond de la salle.

    ‘Max entered the bar. He sat in the back.’

     
  2. (11)

    Aujourd’hui personne ne lui adressa la parole.

    ‘Today, nobody talked with him.’

     
A similar association is made in the literature for the IMP, which arguably has more frequently subjective usages as in examples (12), and less frequently non-subjective usages, as in (13) (from Moeschler et al. 2012).
  1. (12)

    Marie sauta dans le train. Cinq minutes plus tard, le train déraillait.

    ‘Mary jumped in the train. Five minutes later, it was derailing.’

     
  2. (13)

    Les dinosaures vivaient il y a des centaines de millions d’années.

    ‘Dinosaurs lived hundreds of millions of years ago.’

     

There were 105 participants, all native speakers of French. At the time when the experiment was carried out, they were 1st year Bachelor students at the Faculty of Humanities at the University of Neuchâtel. Their participation in the experiment took place during a Linguistics class. It was the first time they participated in this type of experiment. The training done on 6 excerpts containing instances of the three target verbal tenses was followed by an interactive discussion regarding the judgments given for each excerpt. They did not receive any other training than the one described above. After the training, each participant received on a paper new excerpts containing instances of the three target verbal tenses, as explained below.

The material used consisted of 214 excerpts containing 65 occurrences of the PS, 72 occurrences of the PC and 77 occurrences of the IMP (20.5% of the total number of excerpts containing a PS, a PC or an IMP in French as source and as target language). They were randomly chosen from the French side of Grisot (2015) parallel corpus. The excerpts were similar to those of examples (14)–(19). Specifically, there were 121 excerpts originally written in French (23% of the total number of excerpts containing a PS, a PC or an IMP in this corpus) and 92 excerpts originally written in English and randomly selected from the French side of the English-French corpus (17.5% of the total number of excerpts containing a PS, a PC or an IMP in this corpus). Original and translated data were used in order to test whether the participants’ judgement about subjectivity is influenced by the status of the language. The 214 excerpts were distributed in 21 sets, 20 sets contained 10 excerpts and one set 14 excerpts. Each participant received only one set. Each set was annotated by 5 participants. The evaluation of the results was performed by calculating pairwise agreement for each item and the Қ value for each pair of annotators.

The results are the following. The clean data consisted of 203 excerpts. There were 11 missing data points due to the fact that some participants did not judge all the excerpts from the set they received. The mean Қ value for the inter-judge agreement rate among the five annotators was 0.29.5 Pairwise agreement analysis showed that 65 excerpts (32%) received an identical judgement by all the annotators and 138 excerpts (68%) received an identical judgement by a majority of annotators (half + 1 or half + 2). All the three verbal tenses analysed were judged as having subjective and non-subjective usages, as shown in Table 2.
Table 2

Results in data containing 203 agreements

 

Non- subjective

Subjective

Total

Abs. freq.

(%)

Abs. freq.

(%)

Abs. freq.

IMP

32

44

41

56

73

PC

41

60

27

40

68

PS

31

50

31

50

62

The same analysis was carried out on the data containing only occurrences of the three target verbal tenses that received an identical judgement by all five annotators, as shown in Table 3.
Table 3

Results in data containing 65 perfect agreements

 

Non- subjective

Subjective

Total

Abs. freq.

(%)

Abs. freq.

(%)

Abs. freq.

IM

8

36

14

64

22

PC

18

67

9

33

27

PS

7

44

9

56

16

The results are similar in the two types of analysis: the IMP and the PS are primarily used subjectively whereas the PC is primarily used non-subjectively. Using a Chi square test, this association was shown not to be statistically significant (χ2 = 4.87, df = 2, p = 0.0876). Hence, the null hypothesis cannot be rejected and I conclude that none of these verbal tenses seems to be specialized for one or the other type of interpretation. The pairs of examples (14) and (15), (16) and (17), (18) and (19) illustrate the subjective and non-subjective usages of the IMP, PS, and respectively, PC.
  1. (14)

    Or mon petit bonhomme ne me semblait ni égaré, ni mort de fatigue, ni mort de faim, ni mort de soif, ni mort de peur.

    ‘And yet my little man seemed neither to be straying uncertainly among the sands, nor to be fainting from fatigue or hunger or thirst or fear.’

     
  2. (15)

    A Cuba, cela se traduisit dès 1969 par une lettre pastorale dans laquelle l’épiscopat prenait ses distances avec l’exil.

    ‘In Cuba, these events were reflected as early as 1969 in a pastoral letter in which the bishops distanced themselves from the extreme position of the Cubans in exile.’

     
  3. (16)

    Sans résultat. Les catholiques, flairant le piège, se rallièrent autour de leur Souverain pontife, et quand on les contraignit à choisir entre leur foi et leur fidélité à l’Etat, penchèrent souvent en faveur de la première.

    ‘Catholics scented another agenda, rallied round their Pontiff, and when forced to choose between faith and loyalty to the state, often chose the former.’

     
  4. (17)

    Une indemnité d’encouragement fut votée, séance tenante, en faveur du docteur Fergusson, et s’éleva au chiffre de deux mille cinq cents livres.

    ‘So a subscription to encourage Dr. Ferguson was voted there and then, and it at once attained the handsome amount of two thousand five hundred pounds.’

     
  5. (18)

    C’est la raison pour laquelle nous avons vraiment préparé, en commission, cette deuxième lecture en un temps record, et nous l’avons présentée en session plénière afin d’avoir le temps de mener les négociations sur une base sérieuse avant la fin de cette année.

    ‘This means that only a conciliation procedure can resolve the problem, so, in the committee, we prepared the second reading and presented it to plenary in what really was record time, so that we would have enough time to conduct the negotiations, on a serious basis, before the year is over.’

     
  6. (19)

    Ce n’est pas à la Grande mosquée de Paris, mais dans la cathédrale Notre-Dame qu’un groupe d’activistes gays a orchestré une “cérémonie de mariage” homosexuel il y a six ans, au cours de laquelle le pape Benoît XVI a été insulté.

    ‘It was not in the Great Mosque of Paris, but in Notre Dame Cathedral that a group of gay activists staged a homosexual “wedding ceremony” six years ago, during which words offensive to Pope Benedict XVI could be heard.’

     
The results of the analysis of subjective and non-subjective usages of French verbal tenses analysed with respect to the stylistic register are given in Table 4. Using a Chi Square test, this distribution is shown to be statistically significant (χ2 = 13.13, df = 2, p < 0.05). The analysis of standardized residuals indicates that the source of this significance is the journalistic register, where the number of observed subjective sentences is smaller than the expected one whereas the number of non-subjective sentences is larger than the expected one.
Table 4

Distribution of subjective and non-subjective usages of French verbal tenses by type of sub-corpus

 

Subjective

Non-subjective

Total

Abs. freq.

(%)

Abs. freq.

(%)

Europarl

34

53

30

47

64

Journalistic

16

29

40

71

56

Literature

49

59

34

41

83

Total

99

 

104

 

99

The analysis per type of language (French as original language vs. translated language) carried out on data consisting of the 65 perfect agreements (35 excerpts for the source language and 30 excerpts for the target language) confirms the preferences found for each verbal tense analysed. The preferences are more marked for French as original language, as shown in Table 5. However, the difference between French as original and as translated language was shown not to be statistically significant (χ2 = 0.06, df = 1, p = 0.8065).
Table 5

Results for French as source and as target language, in data containing 65 perfect agreements

 

IMP

PC

PS

Source (%)

Target (%)

Source (%)

Target (%)

Source (%)

Target (%)

Non-subjective

43

25

58

73

56

29

Subjective

57

75

42

27

44

71

These results are similar to the results of a pilot experiment, in which two annotators judged 80 new excerpts (27 PS, 26 PC and 27 IMP) randomly chosen from the French side of Grisot’s (2015) parallel corpus. The pilot experiment was carried following the methodology and the procedure described in “Methodology and procedure” section. The inter-judge agreement rate corresponded to a Қ value of 0.31. Each verbal tense was judged as having both subjective and non-subjective usages. A qualitative analysis was performed on the explanations given by the two judges for each label chosen during the debriefing session. The judges explained that in 81% of the cases, their judgement was based on linguistic cues, such as grammatical aspect. Specifically, the imperfective aspect was correlated to subjective usages (and therefore, by exclusion, the perfective aspect was correlated to non-subjective usages). Other cues, such as reflexive pronouns, mental state verbs (e.g. believe), verbs indicating direct perception (e.g. see), and inferred perception were signalled in 19% of the cases. In other words, it might be possible that the annotators used their intuitive knowledge that the IMP is imperfective and that the PS and the PC are perfective, to make their judgement regarding subjectivity.

Experiment 3

The purpose of Experiment 3 was to test the link between subjectivity and the speaker’s viewpoint when it is grammatically expressed, as it is in Serbian. The data used in this experiment was randomly chosen from Grisot’s (2015) sub-corpus of SP occurrences, and then translated into Serbian by a Serbian native speaker. According to Smith’s (2003) classification of sources of subjectivity, grammatical aspect is included in her fourth category: perception and perspectival cues. More precisely, according to her, the imperfective viewpoint expresses subjectivity whereas the perfective viewpoint does not. In this experiment, I tested this prediction.

There were three participants (a man and two women, age mean 38, range 19–50), who did not have prior knowledge of linguistics. They are Serbian native speakers and live in the French-speaking part of Switzerland. The three of them have a university degree. It was the first time they participated in this type of experiment. They received a small amount of money for their participation.

The material used consisted of 109 excerpts randomly chosen from the Serbian side of Grisot’s (2015) multilingual corpus (23% of the total number of excerpts in this corpus). The excerpts were similar to those of examples (20)–(23). Grammatical aspect was distributed as follows: 54 perfectives and 55 imperfectives. The material contained 25 excerpts from the Europarl sub-corpus, 33 from the Journalistic sub-corpus and 51 from the Literature sub-corpus. For all the excerpts, Serbian had the status of translated language.

The experiment was carried out according to the procedure described in “Methodology and procedure” section. The evaluation of the results was performed by calculating pairwise agreement for each item and the Қ value for each pair of annotators. The annotators were asked to say if the grammatical aspect morpheme was decisive for their judgement, and if that was not the case, on what basis they made the decision (linguistic cues, contextual cues or general world knowledge).

The results are as follows. The mean Қ value is 0.4 (0.36 for the pair A1–A2, 0.43 for the pair A1–A3 and 0.41 for the pair A2–A3). Pairwise agreement analysis of the 109 excerpts judged showed that both the perfective and the imperfective grammatical aspects are more frequently associated with subjective interpretations (76% of the cases for the imperfective and 54% of the cases for the perfective), as illustrated by examples in (20)–(23). The pair of examples (20) and (21) illustrate imperfective non-subjective and imperfective subjective interpretations, whereas the pair of examples (22) and (23) illustrate perfective non-subjective and perfective subjective interpretations.
  1. (20)

    Iako su SAD videle Mušarafa kao vršioca promene, on nikad nije dosegao unutrašnji politički legitimitet, a na njegovu politiku su gledali kao na opširnu i protivurečnu.

    ‘Although the US viewed Musharraf as an agent of change, he has never achieved domestic political legitimacy, and his policies were seen as rife with contradictions.’

     
  2. (21)

    Bio je izrazito lep, prelep. Ljudi koji ga nisu voleli, filistri i učitelji i crkveni mladići bi govorili da je bio samo lep.

    ‘He certainly was wonderfully handsome. People who did not like him, philistines and college tutors, and young men reading for the Church, used to say that he was merely pretty.’

     
  3. (22)

    Gospodine predsedniče, kada se radi o stavki 11 zapisnika o poslovanju, juče smo se složili da danas na dnevnom redu bude Burlanžov izveštaj.

    ‘Mr President, concerning item 11 of the Minutes on the order of business, we agreed yesterday to have the Bourlanges report on today’s agenda.’

     
  4. (23)

    Želeo bih, takođe, da se zahvalim premijeru koji me je počastvovao time što me je zamolio da premestim adresu u odgovoru u govor sa prestola.

    ‘I would also like to thank the Prime Minister, who honoured me by asking me to move the Address in Reply to the Speech from the Throne.’

     
The same analysis conducted only on the 63 occurrences of the perfective and imperfective aspects for which the three annotators made the same judgement (perfect agreement) shows that 90% of the imperfectives were judged as subjective whereas 57% of the perfectives were judged as non-subjective. This distribution is statistically significant (χ2 = 13.77, df = 1, p < 0.05). A qualitative analysis was performed on the justifications provided by the participants for their decisions for each experimental item. Among the most frequent type of explanations were factuality, modality, world knowledge, perception and grammatical aspect. Although a statistically significant correlation was found between [±subjectivity] and grammatical aspect, it does not seem to the main source of information used by the participants for making their decisions (Table 6).
Table 6

Grammatical aspect and subjectivity in 63 perfect agreements

 

Subjective

Non-subjective

Total

Abs. Freq.

(%)

Abs. Freq.

(%)

Imperfective

36

90

4

10

40

Perfective

10

43

13

57

23

Total

46

 

17

 

63

The results of the analysis of subjective and non-subjective usages of Serbian perfective and imperfective grammatical aspects as analysed with respect to the stylistic register are provided in Table 7. Using a Chi Square test, this distribution is shown not to be statistically significant (χ2 = 5.87, df = 2, p = 0.0531).
Table 7

Distribution of subjective and non-subjective usages of Serbian grammatical aspect by type of sub-corpus

 

Subjective

Non-subjective

Total

Abs. freq.

(%)

Abs. freq.

(%)

Europarl

15

60

10

71

25

Journalistic

17

52

16

71

33

Literature

39

76

12

38

51

Total

71

 

38

 

109

To conclude, these three experiments show that subjectivity seems to be a heterogeneous phenomenon, which is interpreted at the global level and which is not directly triggered by the linguistic sources investigated in this paper. The results of Experiment 3 may be interpreted as a correlation between [±subjectivity] and grammatical aspect, which may or may not involve a causal relationship. I will discuss the status of subjectivity as a pragmatic feature in “Subjectivity: a pragmatic feature” section.

Subjectivity: A Pragmatic Feature

Discussion of Results

In general, statistically not significant results (p > 0.05) and low Қ values (Қ < 0.7) are interpreted as inconclusive. They may point either to the inappropriateness of the design of the experiment or to the lack of evidence for rejecting the null H0 hypothesis for the value of p and for considering that the data is reliable for Қ values. The methodology and the design of the experiments discussed in this research have been previously validated for other types of information, such the [±narrativity] feature (referring to temporal ordering of eventualities) and the [±boundedness] feature (referring to bounded or unbounded eventualities) among others (cf. Grisot 2015). For this reason, I will consider that the low Қ values are informative.

Experiment 1 showed that subjectivity is a feature hard to pin down for native speakers of English. This applies both to cases where the list of sources consisted of all four categories proposed by Smith (2003) and when it consisted only of the fourth category. Experiment 2 indicated that in French the [±subjectivity] feature is identifiable by native speakers. The agreement rate reaches its limit at a Қ value of 0.3. This value remains constant whether two or more judges participate in the experiment. When the agreements are analysed, they indicate that French verbal tenses expressing past time are not specialized for one of the two values of the [±subjectivity] feature: the IMP and the PS are preferred for expressing the speaker’s subjective perspective whereas the PC is preferred for describing a situation in a non-subjective manner.

The results of Experiments 1 and 2 question the link that has been made in the literature between the [±subjectivity] feature and verbal tenses. The English SP and the French PS were expected to have non-subjective usages, hence allowing the speaker to describe a situation or a series of situations without including her subjective perspective and viewpoint. Based on these experiments, this hypothesis was shown to be too strong. The opposite hypothesis was made in the literature for the IMP. Experiments 2 indicated that in the data containing 65 perfect agreements (judgements made by 5 annotators) the IMP was judged as having both subjective (64%) and non-subjective (36%) usages. When the status of language was considered, that is source vs. target, it was found that verbal tenses have the same behaviour in a text written originally in French and in a text written in another language and translated into French.

Experiment 3, which targeted grammatical aspect in Serbian, indicated that subjectivity is identifiable by native speakers with a Қ value above 0.4. Subjective and non-subjective interpretations of a sentence are correlated with imperfective vs. perfective grammatical aspects (χ2 = 13.77, df = 1, p < 0.05). However, the experiment did not provide evidence for a deterministic relation of the type imperfective → subjective and perfective → non-subjective. These results are similar to Trnavac’s (2006) corpus study in English, Dutch, Serbian and Russian, according to which the four combinations of the two variables and their levels are possible.

The analyses per sub-corpus in each experiment showed that participants were sensitive to stylistic register. For English, the source of significance of the observed distribution is the literary register whereas for French it is the journalistic register. As for Serbian, the distribution of the subjective and non-subjective judgments was shown not to be statistically significant.

Based on Experiments 1–3, I suggest that subjectivity is a highly context- and language-dependent feature. It might arise in French and in English through pragmatic inference (Қ values below 0.4). In English, however, the [±subjectivity] feature seems to be particularly difficult to identify based on the linguistic cues investigated in this research. Using a larger array of linguistic sources, such as the four classes of markers proposed by Smith (2003), has a positive impact on identifying subjectivity. As for Serbian, two possible scenarios can be identified. The first is that the [±subjectivity] feature is procedurally triggered by grammatical aspect, hence through the notion of viewpoint which is stipulated in grammar. The second is that the correlation found between subjectivity and grammatical aspect does not involve a causal relation between the two (scenario supported by the participants’ explanations of their decisions). Further experimental research is required in order to investigate these two scenarios. At this point, it can be suggested that subjectivity is not a local phenomenon, i.e. which is not identifiable locally by looking at linguistic sources such as those included in Smith’s (2003) fourth category. However, in order to validate this suggestion, further research should test the other three categories given by Smith (2003). Subjectivity might therefore be a global phenomenon whose specific sources are not identifiable. When a subjective point of perspective is identified in the utterance, its presence ‘contaminates’ the whole utterance and influences the interpretation of verbal tenses, grammatical aspect and indexical markers (Kuno and Kaburaki 1975, 2004; Grisot 2017).

A Pragmatic Model for Verbal Tenses and Subjectivity

Moeschler et al. (2012) and Moeschler (2016) propose the mixed conceptual-procedural model of verbal tenses (MCPM). The MCPM is based on a classical Reichenbachian analysis of verbal tenses (Reichenbach 1947), supplemented by further pragmatic features. The use of temporal coordinates S (speech moment), R (reference point) and E (event moment), as well as temporal relations of precedence and simultaneity (both co-extensional and inclusive) provide a general template for tense system distinguishing between two sub-systems: one for past tenses and the other for present and future tenses (Moeschler 2016, 130).

The MCPM predicts six pragmatic uses of verbal tenses, based on the following hierarchy of features: [±narrative] > [±subjective] > [±explicit]. Firstly, the [±narrative] feature refers to whether or not temporal ordering is obtained by use of the current verbal tense. Secondly, the [±subjective] denotes the presence or the absence of a viewpoint or a psychological perspective in the description of a situation or series of situation. Finally, the [±explicit] feature signals whether the perspective is explicitly mentioned or implicitly accessed.

Moeschler et al. (2012) suggested that the [±narrative] and the [±subjective] features are procedural information encoded by verbal tenses. In Grisot (2015), I show that the procedural nature of the first feature was validated experimentally by looking at the simple past (in English, French, Romanian and Italian), the compound past and the imperfect (in French, Romanian and Italian). Offline experiments with linguistic judgement task, which had the same procedure as in the three experiments described in this article, were carried out. More precisely, for the [±narrative] feature, inter-annotator agreement rates were above 0.4 in all four languages investigated. According to the scale proposed in Grisot (2014), this value signals that the information judged is procedural. Moeschler et al. (2012) argue in favour of a similar treatment for the [±subjective] feature: it is procedural information encoded by verbal tenses and it represents the instruction for the hearer to look for a perspective/viewpoint on the eventuality expressed. The perspective/viewpoint can be explicit (i.e. expressed lexically) or implicit (i.e. inferred contextually).

Other scholars, such as Saussure and Sthioul (1999, 2005), Boogaart (2007) and Saussure (2013), adopt a similar position with respect to the procedurally encoded nature of the notions of perspective and viewpoint. However, they provide different arguments. Boogaart (2007) proposed that modal and subjective readings of the imperfect in Romance languages are due to specific instantiations of its underlying anaphoric semantics. More precisely, this concerns the dependence of the imperfect on another verbal tense in the discourse, which can provide it with a reference point R. Hence, the imperfect would impose the constraint that the event or state be simultaneous to some previously established R (E = R). Saussure (2013) provides a more detailed formula, given in (24), which describes this procedure encoded by the IMP. (24) is read as ‘the reference point R, inherited from some other sentence with a perfective past or a punctual adverb, is situated inside on ongoing eventuality R, […] and R precedes S’ (Saussure 2013, 49).
  1. (24)

    R ⊂ E & R-S

     
The reference point R can be instantiated as a subjective point of perspective or a point of evaluation for the truth-conditional content of the clause. When R is saturated as a subjective point of perspective, the imperfect has a subjective modal interpretation. For example, in (25) the IMP’s R is given by the event presented in the preceding discourse, which is a PS in this case. The PS’s R is a point of perspective in which John notices the room was dark. It can be identified by means of pragmatic inferencing: it was probably dark before, at and after the moment in which John entered the room.
  1. (25)

    Jean entra dans la chambre. Il faisait noir comme dans un four.

    John entered.PS the room. It was.IMP pitch dark.

    ‘John entered the room. It was pitch dark.’

     
Sausure and Sthioul (1999, 2005) and Saussure (2013) suggest that for narrative, counterfactual and indirect-speech subjective usages of the IMP, R is saturated by a moment at which a third party perceives the eventuality as being in the course of happening. In these subjective usages, the procedure encoded by the IMP changes from (24) to (26), where the variable to saturate is not R anymore but S’, which is a projected non-egocentric deictic point, and this projected non-egocentric deictic point saturated as a third-party perspective P precedes S.
  1. (26)

    S’ ⊂ E & P-S

     

The experimental work presented in this article does not seem to provide evidence supporting this hypothesis with respect to French verbal tenses. As discussed above, a deterministic relation between a verbal tense and one of the two possible interpretations: subjective or non-subjective was not found. Based on the value of the Қ coefficient, I suggest that this information is, in French, recovered through context-dependent general pragmatic inferences. However, this interpretation should be nuanced for two reasons.

The first is that the procedural account proposed by Moeschler et al. (2012), Sausure and Sthioul (1999, 2005) and Saussure (2013) does not make the prediction of a deterministic relation. On the contrary, they give examples of cases in which the IMP (but also the PC and the PS) has non-subjective usages corresponding to the prototypical background and descriptive IM, and non-prototypical subjective usages, corresponding to non-background usages, such as narrative IMP, counterfactual and indirect-discourse usages. In Experiment 2, the IMP was judged as having subjective and non-subjective usages. However, the judgement in terms of subjectivity is not dependent on prototypical and non-prototypical usages of the IMP.

The second reason is that the data used in these experiments is corpus data, which might be less straightforward to judge than artificial data. A difference in the complexity of linguistic judgements involving corpus data vs. artificial data was also found for other types of linguistic information. For example, in Grisot (2015) I show that a simple task like conjugating a verbal tense provided in the infinitive is more difficult for corpus data than for artificial data. When performing this task on 36 artificial sentences the two judges agreed on all the items (Қ = 1), whereas when performing on 90 sentences from a corpus, they agreed on the correct label in 93.3% of the cases (Қ = 0.86). Similar observations were made for connectives. Cartoni et al. (2013) show that, if intuitively speakers agree on the various senses (i.e. contextual usages) of the French alors que ‘while’ and of the English while, when they are asked to identify these senses by looking at natural data, the task is more difficult. In their annotation experiments, inter-annotator agreement rates did not go beyond the Қ of value of 0.42, and this both for alors que and for while.

Based on these observations, it is possible that the Қ value of 0.3 from Experiment 2 is actually due to the fact that the data is only natural data, and not to the fact that subjectivity is recovered through general pragmatic inferences. In order to determine this, the same experiment should be carried out on artificial data as well. This corresponds to one of the requirements given in Grisot (2014) for using the Қ scale with respect to the conceptual/procedural distinction: both natural and artificial data must be used in the experiment in order to control for variation coming from the type of data.

Furthermore, the results of Experiment 3 indicate that in Serbian subjectivity is correlated to grammatical viewpoint. As noted above, based on the current results, it cannot be established whether this correlation involves a causal relationship. The Қ value of 0.4 could be interpreted as signalling that the information judged is procedural information. As in the other experiments, the data tested is natural data. Hence, one might expect to have a higher value if artificial data was tested as well. If the [±subjectivity] feature is encoded by grammatical viewpoint, this might explain the results of the qualitative analysis of the explanations provided by French native speakers, according to which imperfective and perfective viewpoints were relevant for their judgment of French verbal tenses in terms of subjectivity. However, further experimental research is required in order to determine if a causal relation of a procedural nature exists between subjectivity and grammatical aspect.

Conclusion

In this paper, I investigated the notion of subjectivity, operationalized as the [±subjectivity] feature, in offline experiments with a linguistic judgement task. It was tested by looking at English and French verbal tenses expressing past time, and at grammatical aspect expressed morphologically in Serbian.

Evidence in favour of a high context- and language-dependent status of this feature was found. More precisely, native speakers of English found it very difficult to deal with this feature. Native speakers of French had better results at the task than English speakers, and used perspectival and perception cues for identifying subjective and non-subjective usages of French verbal tenses. Finally, native speakers of Serbian performed better than French speakers at the task when they used information coming from grammatical aspect. The experiments were carried out only on corpus data, and I made the suggestion that low agreement rates might also be due to the complexity of the corpus data compared to artificial data.

This research pointed out the great complexity of subjectivity, for which further systematic research is required in order to have a clearer picture. Consequently, I propose a series of future lines of inquiry and possible improvements of the current study, which can be summarized as follows. Firstly, there are two methodological issues regarding the material and the participants. The first concerns the data. Natural corpus and translated corpus data should be used along with built data, which has the advantage of being carefully controlled. The second regards the participants. They should be native speakers of the same L1 variety of language and be comparable with respect to their age and their educational background, in order to control for possible confounds in their assessment of subjectivity.

Secondly, there are a series of methodological suggestions regarding the type of analyses that could be carried out. For example, a link between aspectual classes and subjectivity is to be investigated by performing a fine-grained analysis of the semantics of the verbs used in the excerpts tested in the current experiments. Also, the role played by the other linguistic cues given in Smith’s (2003) classification for the expression of subjectivity should be assessed. This suggestion is supported by the qualitative analysis performed on the participants’ explanations and justifications for their decisions. The range of explanations they gave is very broad and it goes beyond the linguistic sources given in the annotation guidelines. This could be interpreted as evidence in favour a global account, rather than a local one, of subjectivity. Additionally, the potential causal relation between subjectivity and grammatical aspect should be investigated on the basis of more classic experimental paradigms, in which variables are carefully manipulated in order to test their effect on the variable of interest.

Finally, there are two theoretical issues that should be further considered. The first is defining the concept of subjectivity in a different framework than the linguistic and pragmatic approach chosen in this study. As mentioned in “Different approaches to subjectivity” section, the tools provided by Langacker’s framework such as the construal relationship, the conceptualizer and object of conceptualization and their relation to the ground, might provide new insights into the study of subjectivity. The second issue is linked to the definition of subjectivity as referring to the speaker’s psychological perspective and viewpoint. In a discourse, a third party’s psychological perspective and viewpoint can be expressed which might be different than the speaker’s own subjectivity.

Footnotes

  1. 1.

    The author is grateful to the three anonymous reviewers and to the editor for their very useful and relevant comments and suggestions on earlier versions of this paper. They have helped me to make this paper more precise. “I would like to thank Jacques Moeschler and Gaëtanelle Gilquin for their valuable comments and suggestions for this research, which is financed by the Swiss National Science Foundation. The financing was obtained for the VTS project (Verbal tenses and subjectivity: an empirical cognitive approach”, application no. 100015_170008/1)

  2. 2.

    Gold standard data refers to trustworthy and reliable annotated data, generally coming from human annotation experiments and used for training and meaningful evaluation of algorithms in Machine Learning. More generally, gold standard refers to scientific procedures or collections which are accepted standards (Wissler et al. 2014).

  3. 3.

    Here, the authors make reference to Sanders and Spooren’s (2009) study in which they annotated two Dutch connectives: omdat, which is most frequently used in objective causal relations (that is, they express causality between events in the real world), and want, which is considered to be a prototypical marker of subjective causal relations holding between the speaker’s conclusions on the basis of events in the world (Degand and Pander Maat 2003; Pit 2003; Canestrelli et al. 2013).

  4. 4.
  5. 5.

    0.37 for the pair A1–A2, 0.31 for the pair A1–A3, 0.34 for the pair A1–A4, 0.45 for the pair A1–A5, 0.26 for the pair A2–A3, 0.144 for the pair A2–A4, 0.27 for the pair A2–A5, 0.24 for the pair A3–A4, 0.27 for the pair A3–A5, 0.28 for the pair A4–A5. These values reflect inter-subject analyses because each annotator saw only one set and each set was judged by five annotators.

References

  1. Artstein, R., & Poesio, M. (2008). Inter-coder agreement for computational linguistics. Computational Linguistics, 34(4), 555–596.CrossRefGoogle Scholar
  2. Banfield, A. (1982/1995). Unspeakable sentences: Narration and representation in the language of fiction. Boston: Routledge & Kegan Paul.Google Scholar
  3. Bayerl, P. S., & Paul, K. I. (2011). What determines inter-coder agreement in manual annotations? A meta-analytic investigation. Computational Linguistics, 37(4), 699–725.CrossRefGoogle Scholar
  4. Benveniste, E. (1966/1971). Problèmes de linguistique générale. Paris: Gallimard.Google Scholar
  5. Boogaart, R. J. U. (1999). Aspect and temporal ordering: A contrastive analysis of Dutch and English (Ph.D. Dissertation). The Hague: Holland Academic Graphics.Google Scholar
  6. Boogaart, R. (2007). The past and perfect of epistemic modals. In L. de Saussure, J. Moeschler, & G. Puskas (Eds.), Recent advances in the syntax and semantics of tense, mood and aspect (pp. 47–70). Berlin: Walter de Gruyter.Google Scholar
  7. Caenepeel, M. (1989). Aspect, temporal ordering and perspective in narrative fiction (Ph.D. Dissertation). Edinburgh: University of Edinburgh.Google Scholar
  8. Canestrelli, A. R., Mak, W. M., & Sanders, T. J. (2013). Causal connectives in discourse processing: How differences in subjectivity are reflected in eye movements. Language and Cognitive Processes, 28(9), 1394–1413.CrossRefGoogle Scholar
  9. Carletta, J. (1996). Assessing agreement on classification tasks: The kappa statistic. Computational Linguistics, 22(2), 249–254.Google Scholar
  10. Cartoni, B., Zufferey, S., & Meyer, T. (2013). Annotating the meaning of discourse connectives by looking at their translation: The translation-spotting technique. Dialogue & Discourse, 4(2), 65–86.CrossRefGoogle Scholar
  11. Chen, W. (2008). Dimensions of subjectivity in natural language. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies: Short Papers (pp. 13–16). Association for Computational Linguistics.Google Scholar
  12. Comrie, B. (1976). Aspect: An introduction to the study of verbal aspect and related problems. Cambridge: Cambridge University Press.Google Scholar
  13. Degand, L., & Pander Maat, H. (2003). A contrastive study of Dutch and French causal connectives on the Speaker Involvement Scale. LOT Occasional Series, 1, 175–199.Google Scholar
  14. de Saussure, L. (2003). Temps et pertinence: éléments de pragmatique cognitive du temps. Bruxelles: De Boeck Duculot.CrossRefGoogle Scholar
  15. de Saussure, L., & Sthioul, B. (1999). L’imparfait narratif: point de vue (et images du monde). Cahiers de Praxématique, 32, 167–188.Google Scholar
  16. de Saussure, L., & Sthioul, B. (2005). Imparfait et enrichissement pragmatique. In P. Larivée & E. Labeau (Eds.), Nouveaux développements de l’imparfait (pp. 103–120). Amsterdam: Rodopi.Google Scholar
  17. de Saussure, L. (2013). Perspectival interpretations of tenses. In K. M. Jaszczolt & L. de Saussure (Eds.), Time, language, cognition and reality (pp. 46–69). Oxford: Oxford University Press.CrossRefGoogle Scholar
  18. Dyvik, H. (1998). A translational basis for semantics. In S. Johansson & S. Oksefjell (Eds.), Corpora and cross-linguistic research: Theory, method and case studies (pp. 51–86). Amsterdam: Rodopi.Google Scholar
  19. Eckardt, R. (2014). The semantics of free indirect discourse: How texts allow us to mind-read and eavesdrop. Leiden: Brill.Google Scholar
  20. Escandell-Vidal, V., & Leonetti, M. (2011). On the rigidity of procedural meaning. In V. Escandell-Vidal, M. Leonetti, & A. Ahern (Eds.), Procedural meaning: Problems and perspectives (pp. 81–102). Bingley: Emerald Group Publishing.Google Scholar
  21. Fleischman, S. (1990). Tense and narrativity. London: Routledge.Google Scholar
  22. Fleischman, S. (1995). Imperfective and irrealis. In J. Bybee & S. Fleischman (Eds.), Modality in grammar and discourse (pp. 519–551). Amsterdam: John Benjamins.CrossRefGoogle Scholar
  23. Genette, G. (1972). Discours du récit, essai de méthode, Figures III. Paris: Seuil.Google Scholar
  24. Grisot. (2014). The conceptual/procedural distinction: empirical perspectives and quantitative measures. In Oral Presentation at Intercultural Pragmatics Held at University of Malta, Malta in May 30–June 1.Google Scholar
  25. Grisot. (2015). Temporal Reference: Empirical and Theoretical Perspectives. Converging Evidence from English and Romance. Ph.D. Dissertation.Google Scholar
  26. Grisot. (2017). Le statut de la subjectivité: Information encodée ou inférée? In Oral presentation at the conference Le triple visage du langage: forme, sens, expression held at University of Banja Luka, Bosnia and Herzegovina on April 7.Google Scholar
  27. Koehn, P. (2005). Europarl: A parallel corpus for statistical machine translation. In MT summit (Vol. 5, pp. 79–86). Available online http://www.statmt.org/europarl/.
  28. Kuno, S. (2004). Empathy and direct discourse perspectives. In L. Horn & G. Ward (Eds.), The handbook of pragmatics (pp. 315–343). Oxford: Blackwell Publishing.Google Scholar
  29. Kuno, S., & Kaburaki, E. (1975). Empathy and syntax. Linguistic Inquiry, 627–672.Google Scholar
  30. Langacker, R. W. (1991). Concept, Image, and Symbol. The cognitive basis of grammar. Berlin: Mouton de Gruyter.Google Scholar
  31. Langacker, R. W. (2002). Remarks on the English grounding systems. In F. Brisard (Ed.), Grounding: The epistemic footing of deixis and reference (pp. 29–38). Berlin: Mouton de Gruyter.Google Scholar
  32. Langacker, R. W. (2006). Subjectification, grammaticization, and conceptual archetypes. In A. Athanasiadou, C. Canakis, & C. Bert (Eds.), Subjectification: various paths to subjectivity (Vol. 31, pp. 17–40). Berlin: Mouton de Gruyter.Google Scholar
  33. Lyons, J. (1982). Deixis and subjectivity: Loquor, ergo sum. In R. J. Jarvella & W. Klein (Eds.), Speech, place, and action: Studies in deixis and related topics (pp. 101–124). New York: Wiley.Google Scholar
  34. Lyons, J. (1995). Linguistic semantics: An introduction. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  35. Moeschler, J. (2014). Subjectivité et langage: l’exemple du présent historique. In B. Pavelin Lesic (Ed.), Francontraste: L’affectivité et la subjectivité dans le langage (pp. 29–40). Mons: Ed. du CIPA.Google Scholar
  36. Moeschler, J. (2016). Where is procedural meaning? Evidence from discourse connectives and tenses. Lingua, 175–176, 122–138.CrossRefGoogle Scholar
  37. Moeschler, J., Grisot, C., & Cartoni, B. (2012). Jusqu’où les temps verbaux sont-ils procéduraux? Nouveaux Cahiers de Linguistique Française, 30, 119–139.Google Scholar
  38. Moeschler, J., Jayez, J., Kozlowska, M., Luscher, J.-M., de Saussure, L., & Sthioul, B. (1998). Le temps des événements: pragmatique de la référence temporelle. Paris: Kimé.Google Scholar
  39. Narrog, H. (2012). Modality, subjectivity, and semantic change: A cross-linguistic perspective. Oxford: OUP.CrossRefGoogle Scholar
  40. Noël, D. (2003). Translations as evidence for semantics: An illustration. Linguistics, 41(4), 757–785.CrossRefGoogle Scholar
  41. Pit, M. (2003). How to express yourself with a causal connective: Subjectivity and causal connectives in Dutch, German and French. Amsterdam: Rodopi.Google Scholar
  42. Pit, M. (2007). Cross-linguistic analyses of backward causal connectives in Dutch, German and French. Languages in Contrast, 7(1), 53–82.CrossRefGoogle Scholar
  43. Reboul, A. (1992). Rhétorique et stylistique de la fiction. Nancy: Presses Universitaires de Nancy.Google Scholar
  44. Reboul, A., Delfitto, D., & Fiorin, G. (2016). The semantic properties of free indirect discourse. Annual Review of Linguistics, 2(1), 255–271.CrossRefGoogle Scholar
  45. Reichenbach, H. (1947). Elements of symbolic logic. New York: Mcmillan.Google Scholar
  46. Sanders, T. (2005). Coherence, causality and cognitive complexity in discourse. In Proceedings SEM-05, First International Symposium on the Exploration and Modelling of Meaning (pp. 105–114).Google Scholar
  47. Sanders, J., & Redeker, G. (1996). Perspective and the representation of speech and thought in narrative discourse. In G. Fauconnier & E. Sweetser (Eds.), Spaces, worlds and grammar (pp. 290–317). Chicago: University of Chicago Press.Google Scholar
  48. Sanders, J., Sanders, T., & Sweetser, E. (2012). Responsible subjects and discourse causality. How mental spaces and perspective help identifying subjectivity in Dutch backward causal connectives. Journal of Pragmatics, 44(2), 191–213.CrossRefGoogle Scholar
  49. Sanders, T., & Spooren, W. (2009). Causal categories in discourse–converging evidence from language use. In T. Sanders & E. Sweetser (Eds.), Causal categories in discourse and cognition (pp. 205–246). Berlin: Mouton de Gruyter.CrossRefGoogle Scholar
  50. Sanders, T. J., Spooren, W. P., & Noordman, L. G. (1992). Toward a taxonomy of coherence relations. Discourse Processes, 15(1), 1–35.CrossRefGoogle Scholar
  51. Sanders, T. J., Spooren, W. P., & Noordman, L. G. (1993). Coherence relations in a cognitive theory of discourse representation. Cognitive Linguistics, 4(2), 93–133.CrossRefGoogle Scholar
  52. Schlenker, P. (2004). Context of thought and context of utterance: A note on free indirect discourse and the historical present. Mind and Language, 19(3), 279–304.CrossRefGoogle Scholar
  53. Scholman, M. C., Evers-Vermeul, J., & Sanders, T. J. (2016). Categories of coherence relations in discourse annotation: Towards a reliable categorization of coherence relations. Dialogue & Discourse, 7(2), 1–28.Google Scholar
  54. Smith, C. S. (1997). The Parameter of Aspect (2nd edition). Dordrecht: Kluwer Academic Publishers.Google Scholar
  55. Smith, C. S. (2003). Modes of discourse: The local structure of texts. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  56. Spooren, W., & Degand, L. (2010). Coding coherence relations: reliability and validity. Corpus Linguistics and Linguistic Theory, 6(2), 241–266.CrossRefGoogle Scholar
  57. Spooren, W., Sanders, T., Huiskes, M., & Degand, L. (2010). Subjectivity and causality: A corpus study of spoken language. In S. Rice & J. Newman (Eds.), Empirical and experimental methods in cognitive/functional research (pp. 241–255). Stanford: CSLI Publications.Google Scholar
  58. Sthioul, B. (1998). Temps verbaux et point de vue. In J. Moeschler, J. Jayez, J. M. Luscher, L. de Saussure, & B. Sthioul (Eds.), Le temps des événements (pp. 197–220). Paris: Kimé.Google Scholar
  59. Sthioul, B. (2000). Passé simple, imparfait et sujet de conscience. In A. Carlier, V. Lagae, & C. Benninger (Eds.), Passé et parfait (Vol. 6, pp. 79–93). Amsterdam: Rodopi.Google Scholar
  60. Stukker, N., & Sanders, T. (2012). Subjectivity and prototype structure in causal connectives: A cross-linguistic perspective. Journal of Pragmatics, 44(2), 169–190.CrossRefGoogle Scholar
  61. Tahara, I. (2000). Le passé simple et la subjectivité. Cahiers de Linguistuque Francaise, 22, 189–218.Google Scholar
  62. Traugott, E. C. (1989). On the rise of epistemic meanings in English: An example of subjectification in semantic change. Language, 65(1), 31–55.Google Scholar
  63. Traugott, E. C. (1995). Subjectification in grammaticalization. In D. Stein & S. Wright (Eds.), Subjectivity and subjectivisation (pp. 31–54). Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  64. Traugott, E. C. (1999). The rhetoric of counter-expectation in semantic change: a study in subjectification. In A. Blank & P. Kock (Eds.), Historical semantics and cognition (pp. 177–196). Berlin/New Work: Mouton de Gruyter.Google Scholar
  65. Traugott, E. C., & Dasher, R. B. (2002). Regularities in semantic change. Cambridge: Cambridge University Press.Google Scholar
  66. Trnavac, R. (2006). Aspect and subjectivity in modal constructions (Ph.D. Dissertation). Utrecht: LOT.Google Scholar
  67. Vuillaume, M. (1990). Grammaire temporelle des récits. Paris: Éditions de Minuit.Google Scholar
  68. Wiebe, J. M. (1994). Tracking point of view in narrative. Computational Linguistics, 20(2), 233–287.Google Scholar
  69. Wiebe, J. M., Bruce R. F. & O’Hara T. P. (1999) Development and use of a gold-standard data set for subjectivity classifications. In Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics on Computational Linguistics (pp. 246–253). Association for Computational Linguistics.Google Scholar
  70. Wissler, L., Almashraee, M., Díaz, D. M., & Paschke, A. (2014). The gold standard in corpus annotation. In IEEE Germany Student Conference. June 2014, University of Passau.Google Scholar
  71. Zufferey, S., & Gygax, P. M. (2016). The role of perspective shifts for processing and translating discourse relations. Discourse Processes, 53(7), 532–555.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Department of LinguisticsUniversity of GenevaGenevaSwitzerland
  2. 2.Cognitive Science CentreUniversity of NeuchâtelNeuchâtelSwitzerland

Personalised recommendations