Analysis
Structural equation modeling (SEM) methodology was adopted in R (Rosseel, 2012), to model the relations between interaction quality, self-regulation and social-emotional competence. Broadly speaking, SEMs combine a measurement model, as obtained with a common factor analysis, with a structural model that reflects the relations between the to-be-investigated latent constructs similar to a common linear regression. This approach is favorable in terms of reducing measurement error compared to a common regression analysis based on sum or average scores.
As regards the measurement model, interaction quality was considered to be a construct composed of the dimensions ‘positive climate’, ‘negative climate’, ‘teacher sensitivity’, ‘regard for child perspectives’, and ‘behavioral guidance’. All five dimensions represent the higher-level domain ‘emotional and behavioral support’ of the CLASS Toddler scale. We chose to include only this domain because it seemed most relevant for the present purposes.
Regarding the construct ‘self-regulation’, we conceptually distinguished between (a) tasks used to measure children’s self-regulatory capacities and (b) teacher’s perceptions of children’s self-regulation. Dimensions of the former were ‘working memory’, ‘selective attention’, and ‘inhibitory control’, whereas dimensions for the latter were ‘classroom’ and ‘interpersonal’ self-regulation.
On a technical note, we would like to point out that although the data included multiple levels (nested observations) which could explicitly be modeled, we refrained from doing so, because our sample size was too small. In the following, we will report separate analyses for the multiple-item measure and the single-scoring measure of interaction quality with the Class Toddler instrument to compare both scoring approaches.
Descriptive Statistics
Interaction Quality (multiple-item-based analysis). Means, standard deviations, minimum and maximum values, as well as skew and kurtosis were computed to investigate the response characteristics of each dimension of interaction quality. As can be seen in Table 1, most indicators seemed to be properly distributed suggesting that the measurement of interaction quality was overall successful. However, the dimension ‘negative climate’ exhibited signs of ceiling effects for all indicators as indicated by a high mean, low standard deviation and extreme skew and kurtosis values. This being confirmed by a visual inspection of the data, we decided to remove this dimension from further analyses. For ‘behavioral guidance’, the indicator ‘problem behavior’ likewise showed extreme skew and kurtosis values, and visual inspection, again, suggested a ceiling effect, so we decided to remove the item ‘Problem behavior’ from the dimension.
Table 1 Descriptive statistics for interaction quality To investigate whether the conceptual five-dimensional structure was reflected in the remaining data, we conducted an exploratory factor analysis with the number of factors being determined using a scree plot. As a result, a single dimension was suggested, and all items loaded on this factor with loadings higher than 0.5. Confirming this outcome, Cronbach’s Alpha value was estimated at α = 0.95 (very good). Taken together, this pre-analysis suggested that all items indeed measure a single construct (interaction quality), but there is not enough reason to separate this construct further into different dimensions. We therefore adopted the one-factor model and subsequently summarized the values of each indicator per teacher to obtain a per-teacher score that could be used for the later regression model.Footnote 1
Interaction quality (single-score analysis). In accordance with the multiple-item-based analysis, the dimension ‘negative climate’ exhibited signs of a ceiling effect (Table 1, lower part), as its mean was close to the maximum (M = 6.69, SD = 0.53) and its distribution considerably skewed (skew = − 1.37). For this reason, we chose to remove this dimension. In contrast to the multiple-item-based analysis, the dimension ‘behavioral guidance’ seemed to be properly distributed. To foreshadow the results, however, including this dimension into the regression models resulted in severe problems with model fit and, as a result, we removed this dimension completely from the analysis. To investigate whether the remaining three indicators formed separate dimensions or, as was the case with the multiple-item-based analysis, a single dimension, we conducted an exploratory factor analysis with the number of factors being determined using a scree plot. As a result, a single dimension was suggested, in accordance with the multiple-item-based analysis. Cronbach’s Alpha was estimated at α = 0.91(very good) suggesting a high internal consistency for this single-factor model.
Self-regulation (tasks). Self-regulation, as measured by behavioral tasks, consisted of the dimensions ‘working memory’, ‘inhibitory control’, and ‘selective attention’ (see Table 2). Inspection of the working memory measure revealed an appropriate difficulty of this task (M = 4.46, SD = 1.18), but also an obvious deviation from a normal distribution (skew = − 1.42, kurtosis = 2.69). Because we did not want to drop this measure altogether, we decided to apply a square power transformation on this measure, which alleviated these problems (skew = − 0.34, kurtosis = − 0.28). As our measure of selective attention, we scored the respective subtest of NEPSY and an overall score was calculated based on processing time and percentage of correct and incorrect reactions. This measure was of appropriate difficulty, as well as reasonably distributed (M = 8.67, SD = 1.75, skew = 0.47, kurtosis = − 0.16) and no transformation needed to be applied. Regarding the ‘inhibitory control’ measure, it was evident that 16 of the 64 children succeeded in not peeking at all at the wrapped toy as reflected by the number of seconds waiting being at the ceiling (60 s). The majority of children, however, seemingly could not wait longer than 20 s (N = 35), and the remaining children gave up halfway (N = 12). Since there did not seem to be an appropriate transformation that solves both the ceiling effect and the non-normal distribution, we decided to leave the measure as it was and to keep this problem in mind during the subsequent analyses. Finally, we checked whether the three tasks reflected a single construct of self-regulation by computing the correlations between them. It was evident, however, that performance in the three tasks only weakly correlated (r = [0.14; 0.27]), meaning that combining the tasks into a single variable was not the right choice. We therefore analyzed the effect of interaction quality on three separate facets of self-regulation.
Table 2 Descriptive Statistics for Self-Regulation Self-regulation (ratings). Regarding the teacher ratings of self-regulation (Table 2), all items of the shortened CBRS showed proper distributional characteristics. That is, item means were distributed between 2.80 (min) and 3.66 (max), standard deviations between 0.84 (min) and 1.26 (max), and both skew and kurtosis values were below the critical value of |1| for all items. We subsequently checked the factor structure of this questionnaire using exploratory factor analysis. As a result, a two-factor solution was preferred, in accordance with the conceptual distinction between classroom and interpersonal self-regulation. That being said, four items from the classroom dimension and two items from the interpersonal dimension did not load sufficiently on any factor (< 0.5) meaning that we had to drop these items from further analyses (see Table 2). For the remaining items, Cronbach’s Alpha was estimated at 0.89 and 0.88, respectively.
Social-emotional Skills. The MASCS questionnaire exhibited proper distributional characteristics (Table 3). In particular, item means were distributed between 1.86 (min) and 3.25 (max), standard deviations between 0.65 (min) and 1.06 (max), and both skew and kurtosis values were below the critical value of |1| for all items. We therefore kept all items for this scale. We further checked whether the two conceptual dimensions inherent in this scale (‘prosocial behavior’ and ‘antisocial behavior’) were reflected in the data at hand by conducting an exploratory factor analysis. As a result, a two-factor solution was indeed preferred, and all items loaded on their ‘correct’ dimensions having a loading of at least 0.5. Consistent with this result, Cronbach’s Alpha for each dimension were estimated at 0.84 (prosocial) and 0.89 (antisocial), respectively.
Table 3 Descriptive statistics for social-emotional skills Behavioral Self-Regulation Tasks and Interaction Quality
We subsequently regressed each task measure of self-regulation on interaction quality, age and gender being included as control variables. In the following, we report standardized regression coefficients (labeled ‘b’) and confidence intervals (labeled ‘CI’). In case of marginal significance, the confidence level of the CI is 90%, otherwise, it is 95%. Further note for the regressions of the multiple-item-based interaction quality score on the task measures of self-regulation: These models were just-identified meaning that it did not make sense to report fit criteria (fit is always perfect in these cases).
Multiple-item-based interaction quality. For ‘working memory’, the resulting model revealed effects in the expected direction; that is, higher interaction quality scores were associated with higher scores in the working memory task (b = 0.23; CI = [0.01; 0.45]). Second, there was a significant effect of gender (b = − 0.26; CI = [− 0.48; − 0.05]) suggesting that being male was associated with lower scores. Finally, age was positively associated with higher scores (b = 0.36; CI = [0.15; 0.57]). Regarding our measure of ‘selective attention’, there were no significant effects for interaction quality (b = − 0.01; CI = [− 0.26; 0.23]) and gender (b = − 0.19; CI = [-0.42; 0.04]). However, there was a significant positive effect of age (b = 0.34; CI = [0.11; 0.56]) suggesting that older children showed better selective attention. The adjusted R2 was estimated at 0.10. For ‘inhibitory control’, there was no significant effect for interaction quality (b = 0.10; CI = [− 0.13; 0.33]), whereas the control variable gender was marginally significant (b = − 0.21; CI = [− 0.39; − 0.02]). There was a significant positive effect of age (b = 0.40; CI = [0.20; 0.60]) suggesting that older children showed better inhibitory control. One must keep in mind, however, that our measure of inhibitory control contained a ceiling effect and non-proper distributional characteristics, meaning that one needs to be cautious with regard to interpreting this result.
Single-score interaction quality. The corresponding regressions for our single-score measure of interaction quality generally revealed analogous effects. For ‘working memory’, there were significant effects of interaction quality on the working memory task (b = 0.27; CI = [0.06; 0.49]) and of gender (b = − 0.28; CI = [− 0.49; − 0.07]) and age (b = 0.33; CI = [0.13; 0.54]). One must note, however, that model fit was not ideal (χ2 = 37.04, p > 0.05, df = 8, CFI = 0.90, TLI = 0.830, RMSEA = 0.25) meaning that results must interpreted with caution.
Regarding ‘selective attention’, there were again no significant effects of interaction quality (b = 0.01; CI = [− 0.22; 0.25]) and the control variable gender (b = − 0.20; CI = [− 0.42; 0.03]). The significant positive effect of age, however, was present also in this analysis (b = 0.33; CI = [0.11; 0.54]). Model fit was slightly better although, again, not ideal (χ2 = 29.49, p < 0.05, df = 8, CFI = 0.92, TLI = 0.87, RMSEA = 0.21). For ‘inhibitory control’, there was no significant effect for interaction quality (b = 0.17; CI = [− 0.05; 0.39]), whereas the control variable gender reached the 5% significance level (b = − 0.22; CI = [− 0.43; − 0.01]). In addition, there was a significant positive effect of age (b = 0.38; CI = [0.18; 0.57]) suggesting that older children showed better inhibitory control. Model fit was comparable to the other two models (χ2 = 33.45, p < 0.05, df = 8, CFI = 0.92, TLI = 0.85, RMSEA = 0.22).
Self-regulation Ratings and Interaction Quality
The ‘classroom’ and the ‘interpersonal’ dimension of the CBRS were modeled as latent variables and separately regressed on interaction quality, age and gender being additional control variables.
Multiple-item-based interaction quality. The first model with the ‘classroom’ dimension exhibited a very good fit, as indicated by common fit criteria for structural equation models (Kline, 2011): χ2 = 25.59, p > 0.05, df = 24, CFI = 0.997, TLI = 0.998, RMSEA = 0.04. However, the effect of interaction quality on classroom self-regulation was non-significant (b = − 0.07; CI = [− 0.42; 0.28]), and so was the effect of age (b = − 0.06; CI = [− 0.41; 0.30]). Gender, however, showed a significant association with self-regulation (b = − 0.42; CI = [− 0.72; − 0.12]) suggesting that being male was associated with lower classroom self-regulation scores. For the ‘interpersonal’ dimension, model fit was similarly good (χ2 = 21.34, p > 0.05, df = 17, CFI = 0.99, TLI = 0.996, RMSEA = 0.07), but again, there was no significant relation between interaction quality and self-regulation (b = 0.07; CI = [− 0.21; − 0.35]). Here, age showed a significant effect (b = 0.35; CI = [0.10; 0.59]) suggesting that older children showed better interpersonal self-regulation. The effect of gender, however, was non-significant (b = − 0.14; CI = [− 0.40; 0.13]).
Single-score interaction quality. Comparable results were obtained for the single-score analysis. Both models of self-regulation exhibited acceptable fits (classroom: χ2 = 52.69, p < 0.05, df = 33, CFI = 0.94, TLI = 0.95, RMSEA = 0.11; interpersonal: χ2 = 51.26, p < 0.05, df = 33, CFI = 0.96, TLI = 0.97, RMSEA = 0.098). For classroom self-regulation, there was an effect of gender (b = − 0.44; CI = [− 0.69; − 0.18]), but non-significant effects for interaction quality (b = − 0.10; CI = [− 0.31; 0.10]) and age (b = 0.03; CI = [− 0.27; 0.34]). For interpersonal self-regulation, there was a significant effect of age (b = 0.36; CI = [0.12; 0.60]), but non-significant effects of interaction quality (b = 0.09; CI = [− 0.10; 0.28]) and gender (b = − 0.13; CI = [− 0.39; 0.13]).
Social-emotional Skills and Interaction Quality
Multiple-item-based interaction quality. In accordance with the two-factor structure of the MASCS reported above, ‘prosocial behavior’ and ‘antisocial behavior’ were separately regressed on interaction quality, age and gender being included as control variables. As a result, both regression models exhibited acceptable fits (prosocial: χ2 = 54.24, p < 0.05, df = 41, CFI = 0.95, TLI = 0.96, RMSEA = 0.08; antisocial: χ2 = 57.01, p < 0.05, df = 32, CFI = 0.97, TLI = 0.98, RMSEA = 0.12). However, although pointing into the expected direction, the effect of interaction quality remained non-significant in both cases (prosocial: b = 0.03; CI = [− 0.24; 0.31]; antisocial: b = − 0.18; CI = [− 0.47; 0.11]). For prosocial behavior, there were significant effects of gender (b = − 0.42; CI = [− 0.64; − 0.20]) and age (b = 0.35; CI = [0.10; 0.59]) suggesting that this kind of behavior is more prevalent in girls than in boys and that it becomes more prevalent with age. For antisocial behavior, both effects remained non-significant (gender: b = 0.12; CI = [− 0.17; 0.40]; age: b = − 0.16; CI = [− 0.43; 0.11]).
Single-score interaction quality. In contrast to the multiple-item-based analysis, the model for prosocial behavior exhibited a relatively poor fit (χ2 = 74.98, p < 0.05, df = 63, CFI = 0.881, TLI = 0.897, RMSEA = 0.090). Although pointing into the right direction, the effect of interaction quality on prosocial behavior failed to reach significance (b = 0.16; CI = [− 0.06; 0.38]). However, there were significant effects of gender (b = − 0.43; CI = [− 0.66; − 0.22]) and age (b = 0.31; CI = [0.03; 0.59]), consistent with the multiple-item-based analysis. The regression modeling the relation between interaction quality and antisocial behavior exhibited an acceptable fit (χ2 = 87.92, p < 0.05, df = 52, CFI = 0.94, TLI = 0.945, RMSEA = 0.114). In contrast to the multiple-item-based analysis, there was a marginally significant effect of single-score interaction quality on antisocial behavior (b = − 0.20; CI = [− 0.39; − 0.02]) in the expected direction; that is, higher interaction quality scores were associated with fewer antisocial behavior. The effects of gender (b = 0.06; CI = [− 0.21; 0.35]) and age (b = − 0.20; CI = [− 0.48; 0.08]), however, were non-significant. To explore in more detail the relation between interaction quality and antisocial behavior, we carried out additional regressions separately for each sub-factor of antisocial behavior, that is, disruptiveness and impulsiveness. As a result, there was a significant effect of interaction quality on disruptiveness (b = − 0.29; CI = [− 0.8; − 0.09]) but not on impulsiveness (b = − 0.11; CI = [− 0.38; 0.15]) suggesting that higher interaction quality may be associated with lower disruptiveness. In addition, there was a significant effect of gender on impulsiveness (b = 0.28; CI = [0.02; 0.5]) but not on disruptiveness (b = − 0.06; CI = [− 0.33; 0.24]). Age did not significantly relate to either dimension of antisocial behavior (disruptiveness: b = − 0.23; CI = [− 0.50; 0.05]; impulsiveness: b = − 0.22; CI = [− 0.48; 0.04]).