Linguistic inferences from pro-speech music

Abstract

Language has a rich typology of inferential types. It was recently shown that subjects are able to divide the informational content of new visual stimuli among the various slots of the inferential typology: when gestures or visual animations are used in lieu of specific words in a sentence, they can trigger the very same inferential types as language alone (Tieu et al., 2019). How general are the relevant triggering algorithms? We show that they extend to the auditory modality and to music cognition. We tested whether pro-speech musical gestures, i.e. musical excerpts that replace words in sentences, can give rise to the same inferences. We show that it is possible to replicate the same typology of inferences using pro-speech music. Minimal and complex musical excerpts can behave just like language, gestures, and visual animations with respect to the logical behavior of their content when embedded in sentences. Specifically, we found that pro-speech music can generate scalar implicatures, presuppositions, supplements, and homogeneity inferences.

Appendices

Appendix I: Musical gestures and vocal gestures

As mentioned in Sect. 3.3, we ran a parallel experiment on a different pool of participants using pro-speech onomatopoeias, i.e. iconic vocal sounds replacing one or several words in sentences, that we call ‘vocal gestures’, instead of musical gestures. The same four inference types were tested: scalar implicatures, presuppositions, supplements and homogeneity inferences, using paradigms and stimuli that were either perfectly analogous to the ones described throughout this paper, or very slightly different when vocal counterparts to musical stimuli could not be found. For instance, wherever an upward scale was used in our stimuli to evoke a rise in space to test for presuppositions, we used a whistle rising in frequency, and wherever a choir singing the French national anthem was used to evoke the action of singing to test for homogeneity inferences, we used a similar stimulus where the very same song was whistled instead of sung. Wherever a drum was used to evoke someone boxing to test for scalar implicatures, we used the onomatopoeia boom vocally pronounced.

We found a difference in the results collected from the experiment on musical gestures and the one on vocal gestures, which is surprising as most paradigms were perfectly symmetric. However, this experiment was ran on a smaller pool than the musical gestures experiment, and the lack of systematically significant effects of the inference type might thus be explained by a lack of power. We found indeed that the differences in the endorsement rates across both experiments were mainly not or marginally significant themselves, i.e. the responses given for the experiment on vocal gestures were not significantly different from the ones given for the experiment on musical gestures. Figure 7 summarizes the comparisons of the distribution of the data between both experiments. For each inference type, we computed the interaction between inference type and the Experiment factor, whose two levels corresponded to the two experiments.¹⁷

Fig. 7

Comparison between musical gestures and vocal gestures experiments

Figure 7 displays the figures assessing the significance of the difference in distribution of the data collected across experiments. None of the data subsets were significantly different across both experiments, except for supplements, for which we found no contrast in endorsement due to an unexpected interpretation of the vocal stimulus.

Appendix II: Statistical models

In this appendix, we provide the generalized linear mixed-effects models we used to analyze the data. They were all inspired by that of Tieu et al. (2019), whose analysis script is open source. For each model, we justify the structure of the model and in particular its random effects structure by the design [as recommended in Barr et al. (2013)]. We also provide the coefficients found for each model. For the sake of clarity, we report the statistical models in a raw fashion (i.e. the R code lines). Useful information about the syntax of R, which can be helpful to read these formulas, can be found online and are accessible at this link: https://github.com/clayford/LMEMInR/blob/master/lme4_cheat_sheet.Rmd.

1.1 Scalar implicatures

In the scalar implicatures paradigms, two main factors could have had an effect on the endorsement of the target inferences: the gesture factor (GestureC), which two levels correspond to the target and control premises contrasting two alternative realizations of the drum sound mimicking someone boxing, drum$\times$1 and drum$\times$2; and the inference factor (InferenceC), which two levels correspond to the target and baseline inferences contrasting two possible interpretations of the the drum sounds (‘some’ vs. ‘a lot’ in the positive environment; ‘some’ vs. ‘none’ in the negative environment).

Here, we were interested in the GestureC * InferenceC interaction to ensure that the contrast found for the target premise was not due to a default bias in endorsement for one inference over the other, by comparing the difference between the endorsement of the target and that of its negation for the target premise and the same difference for the control premise. We did not include the environment factor in the model (positive vs negative) because there was no theoretical reason to expect a difference in these interactions between the positive and the negative environment. GestureC, InferenceC, and their interaction were thus used as fixed effects in our model, while this interaction by subject was used as a random effect, accounting for the variability across participants in (i) the interpretation of the premise, (ii) the interpretation of the inference, and (iii) the interaction between (i) and (ii).

Model

value $\sim$ GestureC * InferenceC +(1 + GestureC * InferenceC|SubjID)

This first model failed to converge. We followed the same procedure as (Tieu et al., 2019) in simplifying the random effects structure, and removed the interaction between the two factors GestureC and InferenceC from the random effects as follows:

Simplification of random effects structure (as in Tieu et al., 2019)

value $\sim$ GestureC * InferenceC+(1+ GestureC+ InferenceC|SubjID)

$$\begin{aligned} \begin{array}{ c c c c c } \text { Environment} &{} \text {(Intercept)} &{} \text {GestureC} &{} \text {InferenceC} &{} \text {GestureC:InferenceC} \\ \text {Positive} &{} 52.60377 &{} -\,0.79245 &{} -\,0.07547 &{} 116.60377 \\ \text {Negative} &{} 57.005 &{} 1.557 &{} 7.575 &{} 44.698\\ \end{array} \end{aligned}$$

1.2 Presuppositions

There was no theoretical reason for predicting an interaction in presuppositional behavior between question formation and projection under ‘none’, as the two projection tests are independent. Inferences from questions and from ‘none’ were thus analyzed separately. We are rather interested in the effect of the inference type (presupposition vs no presupposition) on the responses.

To ensure that the inference containing presupposition was not due to a by-default preference for this kind of inference, the endorsement of the presupposition was contrasted with that of its negation (baseline inference), as a control. The model thus used the inference factor InferenceC, which levels correspond to the presupposition/no presupposition, as a fixed effect, and this same factor by subject was used as a random effect, to capture the possibility that each participants may simply tend to endorse presuppositions differently.

Model

This first model failed to converge, leading us to simplify the random effect structure by only keeping the SubjID factor as a random effect, to capture the variability in intercept across participants:

Simplification of random effects structure

$$\begin{aligned} \begin{array}{ c c c } \text { Environment } &{}\qquad \text { (Intercept) } &{}\qquad \text { InferenceC }\\ \text { Question } &{}\qquad 49.48 &{}\qquad -\,15.60 \\ \text { `none' } &{}\qquad 51.65 &{}\qquad 2.74\\ \end{array} \end{aligned}$$

1.3 Supplements

The case for supplements is symmetric to the paradigm for scalar implicatures in 6, where both the gesture factor GestureC, describing the two types of premise (the target premise where the musical gesture is expected to be interpreted as a non-restrictive relative clause, and the control premise where the musical gesture is made at-issue by using ‘like this’) and the inference factor InferenceC, describing the two types of inference (supplemental or not) were used as fixed effects in the model, while the interaction by participant was used as a random effect in the maximal model below:

Here, we are interested in the interaction between the Gesture factor, which two levels represent the two forms of sentences (with a pro-speech gesture standing for a non-restrictive relative clause, and with the deictic ‘like this’) and the Inference factor, which two levels correspond to the supplemental inference (if X, then X would have happened in a certain way) and the inference without the supplemental information (if X, then X would not have necessarily happened in this same way). As the control inference using ‘not necessarily’ was not the exact negation of the target inference for reasons of simplicity, we had no strong prediction as to how differently the target and control inference would be endorsed for the control premise using ‘like this’; but we expected this difference to be important for the target inference using a post-speech musical gesture (without ‘like this’) which was expected to trigger a supplemental inference just as post-speech gestures do (Tieu et al., 2019; Schlenker, 2018). We thus expected an interaction between the two factors, with a higher difference in endorsement between the target inference and the baseline inference in response to the target premise than in response to the control premise.

Model

value$\sim$ GestureC * InferenceC + (1 + GestureC * InferenceC|SubjID)

This first model failed to converge, leading us to simplify as before the random effect structure by removing the interaction:

Simplification of random effects structure 1

value$\sim$ GestureC * InferenceC + (1 + GestureC + InferenceC|SubjID)

This second model converged but did not allow for interaction testing (removing the interaction from the model prevented the model from converging), leading us to simplify the random effect structure even more:

Simplification of random effects structure 2

$$\begin{aligned} \begin{array}{ c c c c } \text { (Intercept) }&{} \text { GestureC }&{} \text { InferenceC }&{} \text { GestureC:InferenceC }\\ \text { 57.175 }&{} -\,7.085 &{} -\,26.858 &{} 8.774 \\ \end{array} \end{aligned}$$

1.4 Homogeneity inferences—NP and predicate

The case for homogeneity is analogous to that of presuppositions in 6, where only inference type was used as a fixed effect in the model, while the effect of inference type by participant (testing whether each participant had a personal tendency to endorse the inferences) was used as a random effect. As we were interested in whether the homogeneous (‘all’ or ‘none’) inference would be preferred to the non-homogeneous one in each environment (positive and negative), we are here only interested in the effect of inference type (homogeneous vs. non-homogeneous).

Model

This model did not converge, so we decided, as with presuppositions, to go with the most minimal random effect structure only accounting for the difference in intercepts between participants:

Simplification of random effects structure

$$\begin{aligned} \begin{array}{ c c c c } \text { Hypothesis tested } &{}\text { Environment } &{}\text { (Intercept) } &{}\text { InferenceC } \\ \text { NP } &{}\text { Positive } &{}51.49 &{}-\,59.85 \\ \text { NP } &{}\text { Negative } &{}47.91 &{}-\,38.87 \\ \text { Predicate } &{}\text { Positive } &{}54.71 &{}-\,61.34 \\ \text { Predicate } &{} \text { Negative } &{}40.31 &{}-\,37.34 \\ \end{array} \end{aligned}$$

Note that in our analysis, we were interested in inference type, i.e. whether the homogeneous inference was significantly more endorsed than its non-homogeneous counterpart. However, as the editor pointed out, it would also have made sense to analyze the inference type in terms of universal or existential. As described in the paradigm in Sect. 3.4, we expected musical gestures (that were punctuated repetitions) to have a universal reading in the positive environment (‘all harps’) and to get an existential reading in the negative environment (‘at least one harp’, i.e. ‘some harps’), just as the reading of gestures and visual animations were shown to flip from universal to existential depending on the context in Tieu et al. (2019). If we perform such an analysis, we are interested in the interaction between environment (positive vs. negative) and inference type (understood as universal vs. existential, and not as homogeneous vs. non-homogeneous anymore). We plot the results with this new analysis and report the significance and coefficients of what the model would have been in this situation below:

Fig. 8

Reading flipping under negation

Model

This model failed to converge, leading to a simplification of the random effects structure as follows:

Simplified model

value$\sim$ InferenceTypeC * Environment + (1 + InferenceTypeC + Environment|SubjID)

The comparison of this model to the same model without the interaction between inference type and environment showed a significant difference between the two models ($\chi ^2$ = 84, p < 0.001). This supports the idea that the reading of a same musical gesture flips under negation, which is consistent with our first analysis.

1.5 Iconicity controls

For each iconicity control, we were interested in the difference in endorsement between the inference containing the matching interpretation of the iconic modulation and the inference that contained the opposite non-matching interpretation. For instance, we wanted to know whether the inference containing the interpretation of a long upward scale as a high mountain was more endorsed than the inference containing the interpretation of a long upward scale as a low mountain. We thus counted inference type as a fixed effect. To model the possible differences in the tendencies to endorse both types of inferences in a certain way for each participant, we included inference type by subject as a random effect in the model, as shown:

Model

value$\sim$ InferenceTypeC + (1 + InferenceTypeC|SubjID)

$$\begin{aligned} \begin{array}{ c c c c } \text { Control } &{}\quad \text { (Intercept) } &{}\quad \text { InferenceC } \\ 1 &{}\quad 55.09 &{}\quad 52.11 \\ 2 &{}\quad 52.75 &{}\quad -\,28.37\\ \end{array} \end{aligned}$$

Appendix III: Inferential mechanisms

Here, we provide a loose description of the underlying mechanisms responsible for the four types of inferences tested. Although an important part of its content is itself subject to debate, the aim of this table is merely to provide some very basic analysis of each inference to facilitate the reading of each section; it does not aim at exhaustively accounting for all possible theoretical models.¹⁸$^{,}$¹⁹

Fig. 9

Description of each inferential type