The design included four waves of data collection. The first measurement was in the first academic year after pre-service teachers began their work placements, usually around 3 months after the start of the academic year. The second measurement occurred at the end of the first year. The third measurement was at the end of the second year, and the fourth measurement was at the end of the third year when pre-service training was completed.
Both quantitative and qualitative data were gathered in this study in a nested mixed-methods design with concurrent data collection (see Creswell et al. 2003). The quantitative data provided insight into pre-service teachers’ behaviour in ECEC practice, whereas the qualitative data provided insights into their individual learning experiences. The mixed-methods design of our study allowed us to study PD, taking into account both the growth of interaction skills in ECEC practice (i.e., learning outcomes) and their level of professional reflection (i.e., learning processes, see Measures below), following recommendations from Schachter (2015), Sheridan et al. (2009) and Snyder et al. (2012). The mixed-methods design made it also possible to explore the possible link between pre-service teachers’ growth of interaction skills (research question 1) and reflection on their PD (research question 2).
A total of 47 pre-service teachers in the Pedagogic Work course at the Regional Training Center of Amsterdam (‘ROC van Amsterdam’) participated in the study. They were, on average, 23.1 years (SD = 9.7) and most were female (96%), which is characteristic for the Dutch ECEC workforce. The Netherlands was the country of birth for two-thirds of the participants. The other participants came from Morocco, Suriname and the Antilles, Chile, Egypt, Ghana, the Cape Verde islands, Somalia, and Spain.
Out of the total of 47 pre-service teachers, 17 no longer participated after the second wave of data collection. Six students dropped out over in the next the 2 years period, because they quit the course. The total group of dropouts differed from the remaining pre-service teachers in terms of ethnicity: more native Dutch people dropped out, χ2(1) = 4.37, p = .037. There were no significant differences in terms of learning pathway (school-based or on-the-job), χ2(1) = 0.18, p = .67; Dutch as native language, χ2(1) = 1.02, p = .31; preservice teacher education level three or four, χ2(1) = .09, p = .76; age, F(1, 45) = 0.10, p = .75; or having their own children, χ2(1) = 0.79, p = .37. The final sample and the attrition group did not show statistically significant differences on the outcome measure, although there was a trend effect of trainees who were still included in the final wave with slightly lower scores at the first wave than the students who dropped out (3.19 vs. 3.56, respectively), F(1, 45) = 3.86, p = .056.
An a priori power analysis with G*POWER (Faul et al. 2007) indicated that the initial sample allowed a statistical test of detecting a linear trend with adequate power (β = .80) at the conventional alpha level (α = .05) for f2 values of 0.18 (corresponding to a R2 value of .15). The final sample (N = 23) still allowed a statistical test with adequate power for f2 values of .33 (corresponding to a R2 value of .25). A post-hoc analysis for multilevel repeated measures analysis, using optimal design software (Spybrook et al. 2011), indicated that the statistical power was .90 for the observed effect size from our study (ES = 1.67 for the aggregated CIP measure); in fact, statistical power was adequate for effect sizes ≥ 1.00. Hence, statistical power in our study was adequate for large effect sizes.
Caregiver Interaction Profile (CIP)
A research assistant made two 10 min video recordings of teacher–children interactions from practicum in both a lunch and a structured play situation. The educational materials of each session (e.g., a toy building set or drawing materials) enabled a teacher-driven approach to play and encouraged interaction both between the teacher and the children and among the children themselves. The videotapes of the pre-service teachers who were filmed at their work placement locations were evaluated using the CIP instrument for the assessment of interaction skills. These skills were measured on a seven-point scale with the following anchors: 7 = very high, 6 = high, 5 = moderately high, 4 = moderate, 3 = moderately low, 2 = low, and 1 = very low. Scores under 3.5 are defined as inadequate, scores between 3.5 and 4.5 as moderate, and scores above 4.5 as adequate to good. Also an aggregate score was calculated (CIP total), averaging the scores for the six skills. A validation study of this instrument showed adequate reliability, convergent validity, discriminant validity and predictive validity (see Helmerhorst et al. 2014 for a full description). In our study, trained observers, who met the criterion of an agreement score of .70 (intra-class correlation, mixed model, absolute agreement), independently assessed the filmed episodes. Assessors of the recordings had not visited the locations, in order to rule out contamination of judgment.
The individual interviews, which lasted about 45 min, included open questions, concerning pre-service teachers’ learning experiences during their pre-service training including the work placement. Key questions, which were included at each wave, were related to pre-service teachers’ development of the six interaction skills from the Dutch curriculum (e.g., What is difficult when you interact with children from the group? And, what is easy?). We also asked the participants to reflect on their development (e.g., What are the most important things you have learned so far when it comes to interacting with young children?) and their learning experiences (e.g., In what context did you learn most with respect to interaction skills: at school, in the work placement, or the combination of both?). At each wave, a number of key questions related to interaction skills was similar; additional questions were added at different waves (e.g., reflecting on one’s personal growth during the pre-service training in the final interview). In total, 125 interviews were gathered in a 3 years period.
Demographic characteristics of the pre-service teachers were gathered in the first year with a brief questionnaire, including country of birth, native language, and age.
The participants were informed about the study by their teacher educators in the beginning of the first school year. The teachers also indicated that each pre-service teacher was allowed to participate or not, and participation in the study was based on each students’ individual decision; students were also informed they could stop participating in the longitudinal study. After receiving informed consent, the research team made appointments with the coordinating teacher educator, the pre-service teacher and the center director of the work placement group. The parents at each center were informed about the goal of the study and the research procedures, and they could indicate whether their child was allowed to be video-recorded or not.
A trained research assistant visited the pre-service teachers in their work placement groups in child care settings. Second, the assistant interviewed each participant individually in a separate room with an interview guide. At the beginning of the first interview, the research assistant explained the study procedures once again and also enquired about possible questions of the pre-service teachers related to the study procedures. The interviews were recorded using an audio-recorder and transcribed for analysis. After the interviews, following a standardized protocol for our validated measure, the assistant made video recordings of the pre-service teachers in interaction with the children, in two situations; during lunch/snack-time and during structured play. The play materials changed at each measurement moment during the structured observations to reduce any instrumentation effect; the general procedure and instruction remained unchanged.
We analyzed all longitudinal quantitative CIP data with a multilevel growth model to take into account the hierarchical data structure with repeated measures nested under individual teachers; this technique also allowed us to analyze all available data. The professional growth is expressed as increase per academic year, measured from the start of the course. The fit of the unconditional means model, χ2(2) = 312.5, improved significantly to χ2(5) = 287.2 after adding random intercepts and random slopes, Δχ2(3) = 25.3, p < .001. The random part was fitted with an unstructured covariance model. In an explorative analysis, predictors at teacher level were finally added to this basic model in order to explain significant variation in outcomes.
All interviews were recorded, transcribed, and analysed with the MAXQDA software program. At first, we developed a deductive coding scheme with curriculum-based codes for the six interaction skills. Subsequently, inductive codes were added. In this way we could identify and summarise all relevant data from the interviews. The final coding scheme comprised six main codes and 118 sub-codes. In total, 125 interviews were included with 7584 coded segments. These coded segments consisted of words, sentences or paragraphs containing a shared theme. Three research assistants individually coded the transcripts. To check interrater agreement, a random sample of the transcripts was double coded (N = 292, approximately 4% of all 7584 segments) with the coding scheme. The interrater reliability for each main varied between .80 and .97 with a mean of .87. We selected interview codes related to the development of interaction skills to investigate the perspectives of pre-service teachers related to the PD of their interaction skills. We explored, for each distinguished skill, whether there were important developments in the interview data from wave 1–4 that would contextualise the quantitative results, such as problems related to the use of the skills in practice. Further, we aimed to integrate findings from the qualitative interview data related to their PD on interaction skills and the quantitative growth of these skills during pre-service training by quantifying the results from the interview into a variable called level of reflection.