Recurrence Quantification Analysis as a Methodological Innovation for School Improvement Research

Abstract

School improvement and educational change can be facilitated by learning through reflection, as this allows teachers to discover ways to develop and adapt to change. Higher levels of engagement in reflection have been found to be beneficial, but it is unclear from which everyday routine in engagement in reflection higher levels arise, and thus whether occasions to make knowledge explicit should be organized with a certain constancy. In this study, we therefore used a conceptualization of teacher learning through reflection as a situated and dynamic process in which available environmental information, learning activities, and professional practices are interconnected, and co-develop. Seventeen Dutch Vocational Education and Training teachers participated over a period of 5 months. We explored the use of daily and monthly logs as measurement instruments and Recurrence Quantification Analysis (RQA) as the analysis technique applied to the time-series generated from the daily logs. The findings indicated that teachers who make information from their working environment explicit more are also able to make new insights explicit more. The routine with which teachers make information explicit was found to be mostly unrelated to making new insights explicit. To reach their levels of engagement in reflection, some teachers organized opportunities to reflect with determined intervals, others seemed to recognize those opportunities as the working environment provided them, and some used a combination thereof. Moreover, the use of daily and monthly logs seemed to fit better to some participants than others. Only sometimes does organizing constancy in engagement in reflection seem to relate to the levels thereof. This study provides an example of how logs and RQA can be adopted to tap into professional learning as a dynamic and situated process in support of school improvement and educational change.

11.1 Introduction

In educational research and practice, teacher learning in schools is recognized as an important resource in support of school improvement and educational change. In their efforts to understand the mechanisms underlying school improvement, researchers have started to examine the role of teacher learning as a key component to building school-wide capacity to change. In practice, professional learning communities are being increasingly developed to stimulate the sharing of knowledge, information and expertise among teachers, with the goal to improve instruction and student learning. More specifically, by engaging in professional learning activities, teachers can make knowledge and information explicit, discover the proper scripts for future actions aimed at adaptation to changes such as ongoing reorganizations of work processes and accountability reforms, and to formulate and monitor goals for further development of for instance instructional methods and technological innovations (Korthagen, 2010; Oude Groote Beverborg, Sleegers, Endedijk, & van Veen, 2015a).

To understand more about how engagement in professional learning activities enables teachers to learn, scholars have called for more situated and longitudinal research (Feldhoff, Radisch, & Bischof, 2016; Feldhoff, Radisch, & Klieme, 2014; Korthagen, 2010). The few longitudinal studies conducted so far used analytic techniques (Structural Equation Modelling; SEM) that derive their power from large samples of participants and included a limited number of measurement occasions with relatively long intervals (e.g. yearly intervals) to assess the (reciprocal) relationships between variables under study. The findings suggest, among other things, that reflection is positively related to self-efficacy and changes in instructional practices (Oude Groote Beverborg, et al., 2015a; Sleegers, Thoonen, Oort, & Peetsma, 2014). Higher levels of engagement in professional learning activities, thus, seem beneficial to improve education. In addition, these studies pointed towards the importance of conditions at the school-level, such as transformational leadership and working in teams, to foster teacher learning. This suggests that a purposeful and empowering environment can help to structure uncertainty and ambiguity, and to enable teachers to come to a common understanding about changing their practice, and learn from one another (see also Coburn, 2004; Oude Groote Beverborg, 2015; Staples & Webster, 2008). As such, these longitudinal studies have their merit in validating and extending previous findings from cross-sectional studies on the structural relations between organizational conditions and improving education over time (see also Hallinger & Heck, 2011; Heck & Hallinger, 2009; Heck & Hallinger, 2010).

However, findings on structures at the school-level do not inform about how teachers use these organizational conditions in everyday regulation practices and how such use may fluctuate over time (Maag Merki, Grob, Rechsteiner, Rickenbacher, & Wullschleger, 2021, see chapter 12; see also Hamaker, 2012; Molenaar & Campbell, 2009). It remains for instance unclear how higher levels of engagement in professional learning activities translate to individual teachers’ routines of for instance reflection or knowledge sharing on a daily basis (see also Little & Horn, 2007). Are these higher levels based on for instance reflecting very regularly (every day a little) or in bursts (whenever there is a necessity or opportunity)? By extension, it remains unclear whether the regularity with which moments of teacher learning are organized also contributes to sustaining school improvement (think with regard to regularity for instance of the rhythm of reflection cycle phases for self-improvement, the periodicity of meetings of learning community members to develop instruction and curriculum, and even the intervals of appraisal interviews and classroom observations that can be used for quality development monitoring and accountability purposes) (e.g. Desimone, 2009; Korthagen, 2001; van der Lans, 2018; van der Lans, van de Grift, & van Veen, 2018).

In contrast to large survey studies, case studies have generated situated descriptions of what occurs during efforts to improve schools in specific contexts (see for instance Coburn, 2001, 2005, 2006). However, case studies do not have the aim to generalize their findings, and the validity and utility of those findings is limited. As such, the available research provides no systematic evidence of how (for what and when) teacher learning takes shape in its social context. Consequently, understanding more about the dynamics of everyday teacher learning and its link with school improvement and educational change requires studies that are situated, longitudinal, and aimed at finding systematic relations, and in addition, a corresponding situated and dynamic perspective (Barab et al., 1999; Clarke & Hollingsworth, 2002; Greeno, 1998; Heft, 2001; Horn, 2005; Lave & Wenger, 1991; Reed, 1996).

From a situated and dynamic perspective, school improvement is seen as an ongoing, embedded, complex, and dynamic process of adapting to continuously changing challenges that arise out of schools’ unique circumstances. School improvement emerges from the many interactions between actors within and outside schools, making the school improvement journey highly context-sensitive, and the occurrence of meaningful developments (or milestones) unpredictable (van Geert & Steenbeek, 2014; see also Ng, 2021, chapter 7). Similarly, teacher learning is seen as a cyclical process in which available environmental information, professional learning activities, and productive practices are interconnected and co-develop (Barab et al., 1999; Clarke & Hollingsworth, 2002), that is, teachers attend to, interpret, adapt, and transform information from their environment and make use of their (social) environment to learn what is needed (Barab & Roth, 2006; Gibson, 1979/1986; Greeno, 1998; Little, 1990; Maitlis, 2005).

Investigating ongoing micro-level change processes, such as the routine with which individual teachers make environmental information and changes in meaning, knowledge, or accommodation of teaching practices, explicit, requires analytic techniques that assess intra-individual variability over time, such as State Space Grid analysis (Granic & Dishion, 2003; Lewis, Lamey, & Douglas, 1999; Mainhard, Pennings, Wubbels, & Brekelmans, 2012) or Recurrent Quantification Analysis (RQA). In contrast to commonly used statistical modelling techniques, such as SEM, these techniques are based on dense time-series, whose temporal structures are kept intact. They provide measures about the stability or flexibility of a developmental process. RQA has been applied to analyse coordination in conversations, reading fluency, emergence of insights and behavioural changes (Dale & Spivey, 2005; Lichtwarck-Aschoff, Hasselman, Cox, Pepler, & Granic, 2012; O’Brien, Wallot, Haussmann, & Kloos, 2014; Richardson, Dale, & Kirkham, 2007; Stephen, Dixon, & Isenhower, 2009; Wijnants, Hasselman, Cox, Bosman, & Van Orden, 2012; see also Wijnants, Bosman, Hasselman, Cox, & Van Orden, 2009).

This study aims to examine the overall level and the routine of learning through reflection in the workplace. More specifically, this study focusses on the relation between the temporal pattern of becoming aware of information in the (social) environment and experiencing new insights by making both explicit through reflection. It does so by collecting dense intra-individual (teacher) longitudinal measurements (logs), and by illustrating how RQA can be applied to these time-series. We will explore the application of RQA as a promising analytic technique for understanding the co-evolution of teacher learning and school-wide capacity for sustained improvement.

11.2 Theoretical and Methodological Framework

In this section, we will first describe teachers as active interpreters of their specific circumstances and as reflective practitioners (e.g. Clarke & Hollingsworth, 2002). Next, we will discuss and describe logs as measurement instruments that can capture this situated process over time. Thereafter, we will extensively discuss RQA and we will present examples of studies to provide some research context as to how it can be applied. We will end this section by showing how this conceptualization, measurement instrument, and analysis strategy come together in the present study.

11.2.1 Information and Reflection in a Situated and Ongoing Learning Process

Within the situated perspective, teacher learning is considered an acculturation process (Greeno, 1998; Lave & Wenger, 1991). Teachers are considered active, intentional perceivers, constructing a meaningful practice by integrating new experiences with old experiences (Coburn, 2004; Sleegers & Spillane, 2009; Spillane & Miele, 2007). These experiences are provided by the community while the person is engaged in it (Lave & Wenger, 1991; Little, 2003; Wenger, 1998). Central to this perspective is that knowledge is distributed over a situation (Greeno, 1998; Hutchins, 1995; Putnam & Borko, 2000), that a person makes sense of it through action (Little, 2003; Spillane, Reiser, & Reimer, 2002; Weick, 2011), and that sensemaking is embedded in a person’s history (Coburn, 2001; Coburn, 2004; Sleegers, Wassink, van Veen, & Imants, 2009), as well as in a social and cultural context (Sleegers & Spillane, 2009). While acting, a person selects the information that affords continued action and that fits the understanding of the purpose in the situation (Coburn, 2001; Sleegers et al., 2009; Spillane et al., 2002). Learning can thereby also be characterized as a process of continuously attuning (Barab et al., 1999; Clarke & Hollingsworth, 2002; Granic & Dishion, 2003; Guastello, 2002). As such, teachers can regulate what information in the (social) environment they attend to, so that, over a longer period of time, experiences of interactions with the (social) environment consolidate into new, or differentiations of, meanings, knowledge, and skills (Korthagen, 2010; Kunnen & Bosma, 2000; Lichtwarck-Aschoff, Kunnen, & van Geert, 2009; Steenbeek & van Geert, 2007; van Geert & Steenbeek, 2005). In addition, of course, teachers can develop and adapt by regulating their activities through reflection (Argyris & Schön, 1974; Korthagen & Vasalos, 2005; Schön, 1983).

Teacher engagement in reflection, then, can be seen as an introspective activity that refers to a person recreating an experience of acting in a given situation. In making this experience explicit later, a person supplements the memory of the experience with new ideas that can either be self-generated or based on information gained from others (Oude Groote Beverborg, Sleegers, & van Veen, 2015b). This creates an altered and thus new experience, which can then serve as the basis for future action. In this way, reflection directs what information in the environment is to be attended to, thought about, and reacted to, and for what purpose (Clarke & Hollingsworth, 2002; see also Weick, 2006). Making information explicit in this way helps to put the knowledge that is distributed within teachers’ environments to focussed use and regulates development and adaptation by setting priorities for attention and actions. As such, making previously encountered information explicit shapes future experiences, what can be consequently reflection upon, and what will be made explicit thereafter. This interplay between environmental information and reflection stresses that the directions teachers’ and their school’s developments can take are based in a teacher’s specific circumstances.

Moreover, through repeated investigation of one’s own actions and encountered information, a teacher might, after a while, suddenly discover a new way of acting or looking at the world that is more functional in a given situation than the old one was (Clarke & Hollingsworth, 2002). Such learning experiences of change in meaning, knowledge, or skills, which were generated by one person, can also be reflected upon, made explicit, and shared as possibly of value for other individuals and the team (Nonaka, 1994; van Woerkom, 2004). That also helps to find solutions to ongoing changes and challenges at work, and to formulate and monitor goals for further development (of for instance shared meaning) and improvement (of for instance a school’s capacity for change) (Oude Groote Beverborg, et al., 2015a).

However, due to the circumstantial and temporal dependency of available information, meaning, knowledge, and skills, intensities of engagement in reflection on one’s working environment can fluctuate over time within persons and can differ between persons before new insights emerge (Endedijk, Brekelmans, Verloop, Sleegers, & Vermunt, 2014; Stephen & Dixon, 2009; see also Orton & Weick, 1990). The corresponding trajectories of individual teachers’ engagements in making information explicit may therefore look quite irregular and not alike. Additionally, learning experiences can also emerge with different intervals. Repeated engagement in reflection on one’s working environment therefore changes, continuously slightly (sensitivity to specific information) and occasionally more profoundly (experience of having learned something), the way the world is perceived, understood, and enacted (see also Coburn, 2004; Voestermans & Verheggen, 2007, 2013).

Nevertheless, it remains unclear with how much routine teachers engage in reflection in their everyday practices. Insights into the intra-individual variability in intensity of everyday reflection may provide valuable knowledge to schools as well as to the inspectorates of education about the ways, in which they can organize and support teacher learning in the workplace. In order to tap into these dynamics of reflection and their consequences, measurement instruments therefore need to be designed that allow for specific person-environment interactions and that can be administered densely (see also Bolger & Laurenceau, 2013). Moreover, the chosen analysis needs to provide measures that can represent temporal variability. In the next two sections, we will address the use of logs as a measurement instrument that can be administered densely and the use of RQA as an analytic technique that yields dynamics measures.

11.2.2 Logs

In order to tap into the dynamics of individual teachers’ reflection processes, it is necessary to look at them while and where they are happening – rather than by means of for instance interviews that are prone to hindsight bias or with standardized questionnaires that are insensitive to specific circumstances – to focus on the continuous interaction between the acting professional and the environment through time, and then reconstruct the learning process as a series of interactions over time (see for an example Endedijk, Hoekman, & Sleegers, 2014; Lunenberg, Korthagen, & Zwart, 2011; Lunenberg, Zwart, & Korthagen, 2010; Zwart, Wubbels, Bergen, & Bolhuis, 2007; Zwart, Wubbels, Bolhuis, & Bergen, 2008). This would give an account of professional development including prospective learning, and not only an account of retrospective learning.

In this study, we will therefore measure teachers’ reflection processes with logs (for other uses of logs in dynamic analyses, see: Guastello, Johnson, & Rieke, 1999; Lichtwarck-Aschoff et al., 2009; Maitlis, 2005; for other uses of logs in school improvement research, see Maag Merki et al., 2021, chapter 12; Spillane & Zuberi, 2021, chapter 9). Not everything that happens can be reported in a log. What is reported, is what is most salient in a teacher’s experience. Using open questions, this can be charted in a personalized and situated manner.

The use of logs presupposes that teachers have a sensitivity to information in their environment, that they monitor their development, and that they have an affinity for making information and knowledge explicit by using logs. Every time teachers fill in a log entry, they use an opportunity to make information, experiences, or knowledge explicit (as, in a sense, surveys with targeted items and interviews with targeted questions do as well). Participating in this study might therefore make teachers more aware of what is going on in their environment, of their purpose, and in what areas they develop (Geursen, de Heer, Korthagen, Lunenberg, & Zwart, 2010). By administering logs densely, the logs themselves can also become a familiar part of the working environment that teachers can choose to engage with. Nevertheless, teachers flow with the issues of the day, and may find it hard to disengage from the immediacy of their work to make time to reflect by using logs. Logs thereby not only measure the learning process. They do so by setting a model of the reflection process in terms of content and pace that may fit better or worse to different teachers within a certain period of time. Moreover, the interval with which logs are administered ought to be in accord with the expected rate of change of the frequency with which teachers are likely to reflect upon their environment and learning experiences.

For the assessment of reflection routines, it is important that logs can generate a dense time-series. From these time-series, the dynamics of engagement in reflection can be reconstructed with an RQA.

11.2.3 Recurrence Quantification Analysis

RQA is a nonlinear technique to quantify recurring patterns and parameters pertaining to the stability of the underlying dynamics from a time-series (with an intact temporal structure). An important advantage of RQA, unlike other time-series analysis methods, is that this technique does not impose constraints on data-set size (N). RQA does not make assumptions regarding statistical distributions or stationarity of data either. Nevertheless, for RQA to provide interpretable results, it has been suggested that the minimum requirements for the time-series are that they are long enough to contain at least two repetitions of the whole repeating dynamic pattern and that at least three measurement occasions fall within each repetition of the repeating dynamic pattern (Brick, Gray, & Staples, 2018). Needless to say that more robust and precise estimation will be permitted by measuring longer and denser, which may thus be required for noisier data. The technique reveals subtle time-evolutionary behaviour of complex systems by quantifying system characteristics that would otherwise have remained hidden (i.e., when only taking frequencies into account). To get an idea of what is meant by dynamics, consider Fig. 11.1. It shows five examples of hypothetical, idealized change trajectories (i.e. stability, growth, randomness, and two times regular fluctuation) of engagement in reflection of different persons. Trajectories (a, b, c, and d) all have different temporal patterns (rhythms). Their overall level of reflection does not distinguish them: Each trajectory has a mean of 1. In comparison, trajectories (d and e) differ in their means, but have the same rhythm. The differences between the change trajectories become apparent, because they have (relatively) many time-points.

Fig. 11.1

Five examples of change trajectories, shown as time-series graphs and recurrence plots, of engagement in reflection with different dynamics

Note: Change trajectories (a, b, c, d and e) represent hypothetical, idealized change trajectories (i.e. stability, growth, randomness, and two times regular fluctuation, respectively) of engagement in reflection of different persons. Trajectories (a, b, c and d) all have a mean of 1 but differ in the values of their dynamics (rhythm) measures. In comparison, trajectories (d and e) differ in their means, but have the same values of their dynamics measures. Each trajectory is represented by two graphs: one time-series and one recurrence plot (top and bottom graphs, respectively). The time-series have 36 time points (i.e. days) (x-axis of each graph) and engagement in reflection can have one of the following values at each time point: 0, 1, 2, or 3 (i.e, the number of reflection moments, or the amount of reflection intensity, per day) (y-axis of each graph). In the recurrence plots, both the x-axis and the y-axis represent the 36 time points, and thus the plots have 36*36 = 1296 cells. These cells can either be filled or empty (filling is in this case marked by a black square). Filled cells are called recurrence points. Recurrence points represent that the process had a value at a certain time point and that that value also occurred at another time point (i.e. the recurrence of one of the reflection intensity values). In these examples, the time-series are plotted against themselves in the recurrence plots (i.e. auto-recurrence), and thus the plots are symmetrical around the Line of Incidence (the center diagonal line, i.e. the time-series as it was measured). Auto-recurrence plots are generated for each single time-series separately. The Line of Incidence is excluded in the calculation of the dynamics measures. t = length of the time-series; m = mean of the values in the time-series; sd = standard deviation around the mean; %REC = Recurrence Rate (i.e. the percentage of recurrence points in the recurrence plot); %DET = Determinism (i.e. the percentage of recurrence points that form diagonal lines out of the total of recurrence points); Meanline = the mean length of all diagonal lines of recurrence points; ENTR = Shannon Entropy (i.e. a measure of complexity; it is calculated as the sum of the probability of observing a diagonal Line Length times the log base 2 of that probability). See also the Recurrence Quantification Analysis-section and Fig. 11.3

A distinction can be made between the application of RQA to categorical (nominal) data¹ and to continuous (scale) data. Categorical RQA is a simplified form of continuous RQA². This chapter will focus on categorical RQA. Moreover, RQA can be applied to single time-series (auto-RQA) or to two different time-series (cross-RQA). Fundamentally, auto-RQA is applied to answer questions concerning within-actor variability, whereas cross-RQA is applied to answer questions concerning variability in coordination between actors over time.

RQA combines the visualization of temporal dynamics in recurrence plots with the objective quantification of (non-linear) system properties. In auto-RQA, one time-series is placed on both the x-axis and the y-axis to generate the recurrence plot. In cross-RQA, one time-series is placed on the x-axis and another time-series is placed on the y-axis to generate the recurrence plot. In essence, a recurrence plot is a graphical representation of a binomial matrix that shows after what delays values in time-series recur (recurrence points³). The recurrence plot is then quantified and used to calculate complexity measures.

Consider Fig. 11.1 again. In the figure, engagement in reflection has one of the following values at each time point: 0, 1, 2, or 3 (i.e. the number of reflection moments, or the amount of reflection intensity, per day). The temporal order of these values is given in the time-series graphs. The recurrence plots on the other hand are composed of auto-recurrence points; that is, they show that any of these values occurred at a certain moment and that that also happened sometime else within the same time-series (earlier, at the same time, or later). In these examples, the time-series are plotted against themselves in the recurrence plots (i.e. auto-recurrence), and thus the plots are symmetrical around the Line of Incidence (the centre diagonal line, i.e. the actual time-series – in cross-RQA, this line is sometimes called the Line of Synchrony). Auto-recurrence plots are generated separately for each single time-series. The time-series graph of the stable process in (a) shows that at each time point the process had a value of 1. Therefore, the corresponding recurrence plot is fully filled. In comparison, the growth (and decline) process in (b) shows a steady increase from 0 to 3 followed by a sharp decrease to 0 again. Consequently, the recurrence plot shows neatly clustered recurrence points. The random process in (c) has the same time-series values as the time-series in (b), but in (c), the temporal structure of these values was changed by placing them in a random order. Consequently, the recurrence plot of the process in (c) is less characterized by diagonal lines (consecutive recurrences form diagonal lines). Therefore, the process in (c) has the same values as in (b) for the mean and the Recurrence Rate, but the other dynamics measures differ. The regularly fluctuating processes in (d and e) both have only two values (0 and 3, or 0 and 2, respectively), and in both trajectories, these values recur after the same period. Therefore, they have identical recurrence plots and thus identical dynamics measures.

When the same behaviour is repeated periodically or when different behaviours succeed each other periodically, diagonal lines are formed in the recurrence plot. Measures based on the temporal order of these recurrence-sequences in the recurrence plot inform about the dynamics of the system. The Line of Incidence is excluded in the calculation of the dynamics measures. We will introduce the measures Recurrence Rate, Determinism, Meanline, and Entropy (other measures are Maxline, Laminarity, and Trapping Time) (Marwan, Romano, Thiel, & Kurths, 2007; see also Cox, van der Steen, Guevara, de Jonge-Hoekstra, & van Dijk, 2016) and elaborate on three studies as examples of how to apply them.

Recurrence Rate is computed as the ratio of the number of recurrent points (the black regions in the recurrence plot) over the total number of possible recurrence points in the recurrence plot (i.e. the length of the time-series squared). The Recurrence Rate thus indicates how often behaviours in a time-series re-occur (or also occur in the case of cross-RQA). The Recurrence Rate is not based on the temporal order of the values in the time-series, and is thus a raw measure of variability of behaviour (or of coordination in the behaviours of two actors in the case of cross-RQA) over time.

Determinism is defined as the ratio of the number of recurrence points forming a diagonal pattern (i.e. a sequence of recurring behaviours) over the total number of recurrence points in the recurrence plot. Determinism thus informs about behaviours that continue to recur over time relative to isolated recurrences, indicating the persistence of those behaviours.

An example of a study using Recurrence Rate and Determinism was conducted by Dale and Spivey (2005). They applied categorical cross-RQA to assess lexical and syntactic coordination in conversations of dyads of children and caregivers at many measurement occasions (N_dyads = 3; N_participants = 6; N_{conversations} were 181, 269, and 415). They used the Recurrence Rates of words and of grammar as an indication of coordination between child and caregiver. Types of words are more numerous in conversations than syntactic classes, and types of words therefore gives lower Recurrence Rate values. Additionally, they used the Determinism of words and of grammar, but now based on the set of words that lay within about 50 words from each other in the conversations (i.e. within the band of about 50 words around the Line of Synchrony). This provides an indication of dynamic structures of coordination that are closer together in time and it forms a basis for the interpretation of the Recurrence Rate. Then, they computed both measures again, but now based on the child’s time-series at the same measurement occasion and the caregiver’s time-series at a measurement occasion one step ahead in development. They compared the 2 × 2 Recurrence measures and the 2 × 2 Determinism measures of each dyad using t-tests to assess the influence of the given conversation. Finally, they assessed the development of the Recurrence Rate and Determinism over time using regression analyses. For all comparisons of RQA measures, results indicated that coordination between child and caregiver was stronger within the same entire conversation than over conversations, and that coordination was stronger with greater temporal proximity within a conversation. Moreover, the results indicated that coordination diminished over development.

Meanline is an index of the average duration of deterministic patterns, and thus indicates how long on average the person (or dyad in the case of cross-RQA) remains in similar behavioural states over time. Meanline provides information about the stability of behaviour.

An example of a study using Meanline was conducted by O’Brien et al. (2014). They applied continuous auto-RQA to assess stability of reading fluency of children in different grades and that of adults (N_cohorts = 4; N_participants = 71; N_texts = 1). All participants read the same text. Additionally, each participant of each cohort was randomly assigned to either a silent reading or a reading out loud condition. The researchers used Meanline as a measure of the length of recurring stretches of word-reading-times (other measures relating to other aspects of reading were also used). ANOVAs were used to compare cohorts and conditions. Moreover, they applied continuous cross-RQA to each possible combination of two time-series of the participants within each cohort and within either condition. This analysis gave shared-Meanline values. With this measure, an assessment could be made of whether the reading dynamics of each group were more structured by the text (higher shared-Meanline) or more idiosyncratically (lower shared-Meanline), that is, whether more fluent readers are less constrained by the processing of each (subsequent) word and instead follow their own meanderings through the story to monitor their own understanding of the text. Because of concerns that using the pairwise cross-RQA metric may violate the assumption of independence of observations, the shared-Meanline values were submitted to a bootstrap procedure that drew 1000 subsamples per group, after which confidence intervals were constructed for each group. Using 99% confidence intervals, those groups, whose confidence intervals did not overlap, differed significantly from the other groups. The results indicated that adults had more stability in reading in both reading modes as compared to the other cohorts, and that, when reading out loud, the reading dynamics of both sixth graders and adults are structured more idiosyncratically than those of second and fourth graders and also than those of all cohorts during silent reading.

Entropy is computed as the Shannon Entropy of the distribution of the different lengths of the deterministic segments⁴. Entropy indicates the level of complexity of the sequences of behaviours. The Entropy measure, in RQA, thus indicates how much “disorder” there is in the duration of recurrent sequences.

In the form of peak-Entropy, Entropy can for instance be used as a measure of reorganization⁵. Lichtwarck-Aschoff et al. (2012) conducted a study on the course and effect of clinical treatment for externalizing behaviour problems of children (age-range = 7–12 years). A pattern of reorganization over the course of treatment would be an indication of improvement. Both parents and children received treatment once a week for 12 weeks. Bi-weekly 4 or 6-min observations of problem solving discussions between parent and child formed the raw data (N_dyads = 41; N_participants = 82; N_{conversations} = 6). The data of each participant were initially coded in real-time along nine mutually exclusive affect codes for each participant. The thus acquired time-series were collapsed into one time-series per dyad, resampled to have 72 data points, and recoded along four categories (plus a rest category) that reflected the affective state of the dyad (unordered categorical data). The researchers applied categorical auto-RQA to these dyadic time-series to calculate the Entropy of each conversation of each dyad. 15,000 bootstrap replications of the sample’s Entropy values were used to estimate 95% confidence intervals. The consecutive Entropy values formed the data for subsequent Latent Class Growth Analysis. This analysis was used to identify groups based on the form of the Entropy-trajectories, that is, to distinguish between conversations that could be characterized by a higher Entropy-level followed by a drop in Entropy (i.e. peak-Entropy) and conversations that did not show this pattern of reorganization. Moreover, improvement of children’s externalizing behaviour problems was independently assessed through pre- and post-treatment clinicians’ ratings. Based on criteria for clinically significant improvement, these ratings were also used to divide the sample into classes: improvers and non-improvers. Consequently, the two estimates of class membership were compared. The results showed that dyads in the peak-Entropy-class belonged more frequently to the improvers-class. To assess whether this finding could be simply attributed to either a decline in frequency of negative dyadic affective states or an increase in positive dyadic affective states, the researchers additionally calculated the Recurrence Rates of each coding category of each conversation (again, 95% confidence intervals were based on 15,000 bootstrap replications). The results from a non-parametric test (Kolmogorov Smirnov test) applied to these not normally distributed data showed no differences between classes in the level of recurrence of any of the affective state categories. This indicates that it might be necessary for people to have a period of unpredictability and flux, in which they try out and explore new behaviours, to develop.

11.2.4 Present Study

To reiterate, in this study, we are interested in teacher learning through reflection in the workplace. Building on a situated and dynamic perspective, learning experiences can be seen as emerging from acting upon information in the (social) environment after a period of time. Through reflection on their working environment, teachers make information explicit. Through reflection on learning experiences, teachers make new insights (developed or adapted meanings, knowledge, and skills) explicit. By making these things explicit, teachers can share them with colleagues, put them to focussed use, and set priorities concerning what to attend to and how to act in which situation. Moreover, attending to information can occur more frequently than having new insights, and therefore reflection on the working environment can occur more frequently than reflection on learning experiences. As an example of how to investigate teacher learning through reflection as an everyday and ongoing process, we designed a study to explore the routine with which teachers engage in making information explicit, and how that, in comparison to the overall levels thereof, relates to making new insights explicit. The routine of reflecting pertains to the temporal stability of that activity, and thus its dynamics should be assessed. This requires the collection of dense time-series from individual teachers.

Our measurement instruments, measurement intervals, and analytic measures were chosen in correspondence with this conceptualization. In accord with the different expected rates of change, we chose to use daily logs to measure reflection on the environment and monthly logs to measure reflection on learning experiences. We will explore whether these measurement instruments and measurement intervals are useful for the assessment of the dynamics of learning through reflection (see also Kugler, Shaw, Vincente, & Kinsella-Shaw, 1990).

We used the responses to the daily logs to generate time-series for each participant. Each point in these time-series represents the intensity of reflection on the environment, i.e. the number of reflection moments during a day. The analysis measures for the routine of reflection on the environment were calculated by applying a categorical auto-RQA to each time-series. Recurrence Rate was used as a raw measure of routine and informs about the overall regularity of the reflection process. Determinism was used as a measure of the persistence thereof. The analysis measures for the overall level of reflection on the environment and learning experiences were calculated by simply summing up all responses to the daily and monthly logs, respectively. To investigate the extent to which the overall level and the routine of the intensity of making information explicit co-occurs with the overall intensity of making insights explicit, we generated and inspected scatterplots.

11.3 Method

We used a longitudinal, mixed-method design with convenience sampling to assess the relation between the level and routine of teachers’ engagement in reflection on their environments to make information explicit and the level of reflection on learning experiences to make insights explicit. To do so, we asked teachers to fill in daily and monthly logs, including open questions about the salient information they attended to and the learning experiences they had, respectively, for a period of 5 months. Analyses were applied to the time-series of frequencies of filled in log entries.

11.3.1 Sample

This study was conducted in one VET college in the Netherlands in 2011 (see also Oude Groote Beverborg et al., 2015a). Team leaders were asked whether team members were willing to participate in this study, and participation was voluntary. A total of 20 teachers participated. The data from 1 teacher were excluded from the analysis, because the teacher had moved to a different employer (a college offering professional education), and the data from 2 other teachers were excluded, because they started 2 months late. Thus, the effective sample size was 17. The participants were employed in departments that taught law, business administration, ICT, laboratory technology, and engineering to students and that coached other teachers. Thirteen participants were female, and 4 were male. Working days per week ranged from 2 to 5. In order to generate enough data for a substantive time-series, but as a trade-off between practicality and rigor, the study ran for 5 months: from February until June. During this period, all participants had a 2 weeks’ holiday. One participant (P12) stopped participating after 2 months, and another participant (P10) after 3 months.

11.3.2 Measurement

The study consisted of two logs: a daily and a monthly log. The daily log (diary) asked teachers to make salient information explicit, and thus measured their engagement in reflection on the environment. The monthly log asked teachers to make their insights explicit, and thus measured their reflections on learning experiences. The logs were designed as short, structured interviews with a few open questions. Thereby, participants could report the information that was most relevant to them individually at each measurement occasion. More specifically, the diaries asked about the most salient information that day and the context, in which the information was attended to. The diary questions were focussed on information from colleagues (de Groot, Endedijk, Jaarsma, Simons, & van Breukelen, 2014). The main diary question was: “What did your colleague say or do that was most salient today?” It was made explicit that this could be something someone said, someone did, something that was read, and so on. Other open questions related to the task the participants worked on for which the reported information was relevant, and to how they responded to the information (see Appendix A for the complete specification of one diary entry translated into English). The diaries were designed in such a way that teachers could report their own experiences. The diaries were therefore sensitive to local and personal circumstances and measured with such a density that fluctuations could be expected to be measurable. The monthly logs were designed similarly and asked to report the learning experiences participants had had sometime in the last month as accurately as possible (Endedijk, 2010). The most important question was: “What have you learnt in the last month?” Additionally, questions about the context the learning experience came from, or in which context it had to be understood, were asked, such as about the task and the goal they related to, what means helped to learn it, the manner in which it was learnt, and in what manner participants realized they had learnt something. Lastly, the monthly log also asked questions about what teachers were satisfied with in their learning process and what could be improved in the future, what goals they would pursue in the future, and what they would attend to in the future (see Appendix B for the full specification of one monthly log entry translated into English).

Diaries were administered on each person’s working days and monthly logs on the first working day of the new month. In order to constrain the burden of repeatedly filling in logs, a maximum of three diary entries (making information explicit) and three monthly log entries (making insights explicit) could be filled in per measurement occasion. Also, participants were instructed to spend no more than 5 min on each diary entry (thus a maximum of 15 min per day), and no more than 10 min on each monthly log entry (thus a maximum of 30 min per month). Teachers were asked to fill in at least one log entry per measurement occasion, but this was not mandatory. Logs were administered online. For each participant’s measurement occasion’s log, an invitation was sent by email. On some measurement occasions, some invitations failed to be sent. See Fig. 11.2 and Table 11.1 for frequencies of reporting and descriptives. The analyses were applied to the time-series of frequencies of filled in log entries.

Fig. 11.2

Time-series of participants’ intensities of reflection on the environment

Note: Reflection intensity = the number of reflection moments per working day. P stands for participant. Numbers indicate the participants. For each graph, time is on the x-axis and reflection intensity is on the y-axis. The time-series only include those days, on which participants received invitations to fill out daily logs (working days). Consequently, the time-series vary in length. The largest number of working days of a participant during the measurement period was 82 and, to ease comparison, this value was set as the length of each x-axis. The time-series have been categorized based on the participants’ response patterns. (a): Mean amount of reflection on learning experiences per month is greater than or equal to 1 (m_insights ≥ 1); (b–d): Mean amount of learning experiences per month is less than 1 (m_insights < 1). The participants categorized in (a) made information explicit using the measurement instrument throughout the measurement period. The participants categorized in (b) did not make information explicit using the measurement instrument towards the end of the measurement period (time-series with long 0-value tails), those in (c) had time-series in which 0 (no information made explicit using the measurement instrument on a day) prevailed, and those in (d) stopped participating prematurely. Consequently, the participants categorized in (a) are considered to have more affinity for our measurement instruments, whereas the participants categorized in (b, c and d) are considered to have less affinity for them. See Table 11.1 for participants’ measures in each group

Table 11.1 Descriptives and measures of each participant

In order to uphold motivation, the first author offered individual coaching sessions to the participants. These sessions took place once every month, lasted about 45 min, and were conducted over the telephone. In general, during a session, the information a participant had reported in the log was summarized, and the participant was asked to respond to that. Towards the end of the conversation, the first author categorized some of the information in the diaries and labelled this summary, after which there was opportunity for the participant to reflect upon the labelling of the information. Each conversation ended with the first author asking feedback on the instrument and the conversation. These calls were not intended as part of the measurement of the study and have therefore not been recorded.

11.3.3 Analysis Strategy

The aim of the analyses was to assess in which way the overall level and the routine of the intensity of making information explicit relates to the overall intensity of making insights explicit. We calculated one measure for making insights explicit: each participant’s mean of moments of reflection on learning experiences over the measurement period per month participated (overall insight intensity). This measure is based on the monthly log data. The mean per month was calculated to correct for differences between participants in the duration that they participated.

Crucially, this measure was also used to assess whether participants had affinity for the measurement instruments, that is, whether teachers disengaged from the immediacy of their work to make time to ‘interact’ with our measurement instruments. In line with our request to fill in at least one log entry per measurement occasion, we set a mean of 1 or more reflections on learning experiences per month as the criterion of affinity. Using the monthly log data to categorize participants into groups thus allowed us to differentiate between participants with regard to the validity of administering logs to them. Moreover, it allowed us to contrast group patterns of dynamics of reflection on the environment, which helps to interpret the results.

For reflection on the environment, we calculated three measures. These measures were based on the daily log data. The first measure was the mean of the intensity of making information explicit in the measurement period per working day (overall information intensity). The mean per working day was calculated to correct for differences between participants in working days.

To assess teachers’ routine (or within-person variability) in making information explicit, we applied categorical RQA on each participant’s time-series of intensities of reflections on the environment per day. The time-series only include those days, on which participants received invitations to fill in daily logs (working days). Other days, such as weekends or holidays, or days of the week on which participants were not employed or were employed by another employer, are not part of the time-series. These ‘non-working days’ were cut out to create an uninterrupted time-series. Consequently, the time-series vary in length. The categorical RQA was conducted in MATLAB, using Marwan’s toolbox (Marwan et al., 2007; Marwan, Wessel, Meyerfeldt, Schirdewan, & Kurths, 2002). As measures of routine, we used Recurrence Rate as a measure of the overall regularity of the intensity of the reflection process over time, and Determinism as a measure of teachers’ persistence in sequences of intensities of reflection. The relations between these four variables were established through visual inspection of scatterplots.

11.4 Results

First, we calculated each measure for each participant. To give an idea of how the trajectories of the intensity of making information explicit (information intensity) correspond to their auto-recurrence plots and their measures, four examples thereof are given in Fig. 11.3.

Fig. 11.3

Four examples trajectories of intensities of reflection on the environment represented by time-series and recurrence plots

Note: (a, b, c and d) are the trajectories of intensities of reflection on the environment of 4 participants (based on daily log data). Each trajectory is represented by a time-series graph and a recurrence plot (top and bottom image, respectively). For descriptions of the time-series, see Fig. 11.2 and Table 11.1. t_days = the amount of working days during the measurement period. Σ_infos = the sum of intensity of making information explicit in the measurement period. m_infos = the mean intensity of making information explicit per working day (overall information intensity). %REC = the Recurrence Rate of intensities of making information explicit (as a percentage). %DET = the Determinism of intensities of making information explicit (as a percentage). For detailed descriptions of the the Recurrence Rate and Determinism, see Fig. 11.1 and the Recurrence Quantification Analysis-section. Additionally, the three measures that are based on the monthly logs are also presented: t_months = the number of months on which monthly log invitations were sent. Σ_insights = sum of intensity of making insights explicit in the measurement period. m_insights = the mean amount of making insights explicit per month (overall insight intensity). For the extent to which overall insights intensity, overall information intensity, Recurrence Rate, and Determinism correlate, see Fig. 11.4 and the Results-section

Second, we assessed the participants’ affinity for the measurement instruments. Seven participants had an overall insight intensity (mean of reflections on learning experiences per month) that was greater than or equal to 1, and thus showed more affinity for the measurement instruments. The other ten participants had an overall insight intensity that was less than 1, and thus showed less affinity for the measurement instruments. Splitting the sample into two groups based on overall insight intensity uncovered striking differences in the temporal patterns of making information explicit. Consider Fig. 11.2. The participants categorized in (a) made information explicit using the measurement instrument throughout the measurement period, whereas that seems to falter or cease with the participants in (b, c, and d). The participants categorized in (b) did not make information explicit using the measurement instrument towards the end of the measurement period (time-series with long 0-value tails), those in (c) had time-series in which 0 (no information made explicit using the measurement instrument on a day) prevailed, and those in (d) stopped participating prematurely. Consequently, the participants categorized in (a) are considered to have, for whatever reason, more affinity for our measurement instruments in the measurement period, whereas the participants categorized in (b, c, and d) are considered to have less affinity for them. Due to the difference between the groups in the fit of the measurement instruments to the participants, administering daily and monthly logs seems to be more valid for the participants in (a) than for the others. See Table 11.1 for the participants’ measures and descriptives in each group. A comparison of the descriptives of the two groups suggests a connection between affinity for the measurement instruments and the amount of working days and/or the amount of invitations that failed to be sent.

Third, we explored how overall insight intensity related to the overall information intensity (mean of reflections on the environment per day), and how both related to the Recurrence Rate of information intensity, and Determinism of information intensity. Consider Fig. 11.4. Figure 11.4 Plot (a) suggests a positive correlation between overall information intensity and overall insight intensity within the whole sample, and also within each affinity group separately. More moments of making information explicit co-occurred with more moments of making insights explicit.

Fig. 11.4

Scatterplots with correlations between overall insight intensity, overall information intensity, Recurrence Rate, and Determinism

Note: Squares represent the group of participants that had more affinity for the measurement instruments (see Fig. 11.2 and Table 11.1). Diamonds, triangles, and crosses represent the group of participants that had less affinity for the measurement instruments. Diamonds represent participants that stopped participating prematurely. Triangles represent participants that did not make information explicit using the measurement instrument towards the end of the measurement period (time-series with long 0-value tails). Crosses represent participants, in whose time-series 0 (no information made explicit using the measurement instrument on a day) prevailed. Numbers indicate the participants. Overall insight intensity = the mean amount of making insights explicit (reflection on learning experiences) per month, overall information intensity = the mean amount of making information explicit (reflection on the environment) per day, Recurrence Rate = Recurrence Rate of information intensities, Determinism = Determinism of information intensities. The means of overall insight intensity (per month) and overall information intensity (per day) for each particpant are used to correct for differences between participants in working days and the duration that participants participated. As such, the axis-scales of these two variables go from the minimum (0) to the maximum (3) per measurement occasion. See the text of the Results-section for descriptions of the correlations

Figure 11.4 Plot (b) suggests a negative correlation between overall information intensity and Recurrence Rate within the sample, and also within each affinity group separately. More moments of making information explicit co-occurred with less regularity in doing that. This relation might be explained by the increasing difficulty of having an additional reflection moment beyond the previous one on any given day. Note that none of the participants had both a high level of overall information intensity and a high Recurrence Rate: a highly regular high level of information intensity did not occur

Figure 11.4 Plot (c) suggests a negative correlation between Recurrence Rate and overall insight intensity in the sample. However, within each affinity group separately, there is no clear relation between Recurrence Rate and overall insight intensity. The group of participants that had more affinity for the measurement instruments made more insights explicit and had less regularity in information intensity during the measurement period than the group of participants that had less affinity for the measurement instruments. The level of making insights explicit seems unrelated to the level of regularity of making information explicit when taking affinity for the measurement instruments into account

Figure 11.4 Plot (d) suggests a negative correlation between overall information intensity and Determinism within the sample, and also within the group of participants that showed more affinity for the measurement instruments. However, within the group of participants that had less affinity for the measurement instruments, there is no clear relation between overall information intensity and Determinism. Note that in this group, nearly all of the information intensity values were 0 or 1 and that both a re-occurrence of 0 as of 1 creates a recurrence point. Due to this small set of low values, this group of participants had a low level of overall information intensity and a high level of Determinism, which was similar for those participants, whose time-series consisted of more 0’s as for those, whose time-series consisted of more 1’s. For the group of participants that had more affinity for the measurement instruments, more moments of making information explicit co-occurred with less persistent (periodically recurring) engagement in any of the levels of intensity of making information explicit (or sequences thereof). However, this relation can be explained by the difficulty of maintaining a high level of information intensity over time. Indeed, a highly persistent high level of information intensity did not occur. The correlations from both groups thus highlight the weaknesses of using the response rates of daily logs with several entries as a measurement instrument for the application of RQA

Figure 11.4 Plot (e) suggests a negative correlation between Determinism and overall insight intensity in the sample, and also within the group of participants that had more affinity for the measurement instruments. For this group, more moments of making insights explicit co-occurred with less persistent engagement in any of the levels of intensity of making information explicit (or sequences thereof). However, within the group of participants that had less affinity for the measurement instrument, there is no clear relation between Determinism and overall insight intensity. For this group, the level of making insights explicit seems unrelated to the level of persistence of engagement in any of the levels of intensity of making information explicit (or sequences thereof). Following the argumentation given for the relations in plot (d), it seems likely that those participants that manage to make information explicit whenever an opportunity occurs are also the ones that are able to make the most insights explicit. Note that, on the one hand, P04 seems to have organized these opportunities as one per day and thereby to be able to make insights explicit, as inferred from a highly persistent moderate level of information intensity as well as a high level of overall insight intensity. On the other hand, P17 seems to have strived to have as many of these opportunities as possible on each day and thereby to be able to make insights explicit, as inferred from a lowly persistent high level of information intensity as well as a high level of overall insight intensity

In sum, these results point towards a trend that higher levels of overall information intensity and overall insight intensity concur within a certain period of time. On top of that, no clear pattern was found relating the level of overall insight intensity to with which routine participants made information explicit during a certain period of time.

11.5 Discussion

To summarize, in this study, we explored teacher learning through reflection as a situated and dynamic process using logs as the measurement instruments and RQA as the analysis technique. More specifically, the study focussed on the routine with which teachers engage in making information explicit (reflection on the working environment), and how that, in comparison to the overall levels thereof, relates to making new insights explicit (reflection on learning experiences). We also explored the validity of the measurement instruments and measurement intervals for the application of RQA. Seventeen VET teachers filled in daily and monthly logs over a period of 5 months. From the responses to the daily logs, we generated time-series of the intensity of making information explicit (information intensity) for each participant and applied categorical auto-RQA to each time-series. As measures of the routine of information intensity, Recurrence Rate (regularity) and Determinism (persistence) were used. In addition, we calculated a measure for overall information intensity (the mean amount of information intensity per day in the measurement period) and a measure for overall insight intensity (the mean amount of making insights explicit per month in the measurement period). Relations between the four variables were established through inspection of scatterplots. We found that the sample could be divided into two groups: One that had more and one that had less affinity for the measurement instruments. Moreover, inspection of the scatterplots indicated that higher levels of overall information intensity related to higher levels of overall insight intensity. However, the regularity and the persistence of the intensity with which participants made information explicit had no clear relation with the level of overall insight intensity when taking affinity for the measurement instruments into consideration. In this section we will elaborate on these findings.

That the sample could be divided into one group that had more and another group that had less affinity for the measurement instruments (both daily and monthly logs), may be due to several related reasons. One reason might be related to the difference between the groups in the amount of invitations that failed to be sent. The participants in the less affinity group did not receive an invitation about twice as often as the participants in the more affinity group when correcting for the amount of working days. Increasingly, undependability may have led teachers to falter or cease using our measurement instruments. One of the challenges in conducting this study was to send personalized logs with personalized intervals using an online instrument that was not designed for that, but rather for large-scale, cross-sectional surveys. The developments in digital technology, such as smartphone applications, will have made this problem obsolete for future studies, however.

A second reason might be related to the difference between the groups in the number of days per week they worked. The participants in the less affinity group worked roughly a day more than the participants in the more affinity group, and may simply have been too busy to disengage from the immediacy of their work to make time to reflect by using logs.

A third reason might be related to the dynamics of the reflection process itself. As experience grows, people become less responsive to new information in their environment, and the new information is not further corroborated into experience (Schöner & Dineva, 2007). In this study, the daily logs served as impulses to become aware of information in the environment that some participants might otherwise not have made explicit. Consequently, as experience with this initially attended-to information grew, participants may have felt a need to consolidate acting upon that information first, rather than attending to even more information and deciding how to act upon that. This reason seems particularly fitting for the participants, who did not make information explicit using the measurement instrument towards the end of the measurement period. Nevertheless, whereas administering logs seems to be less valid for these particular participants, the dense time-series the logs generated did point towards an interesting dynamic that future research may explore further.

This third reason relates to that teachers need time to learn (and can attend to teaching less), and also need time to teach (and can attend to learning less) (Mulford, 2010), which points towards the fourth reason: Despite the fact that all teachers volunteered to participate, it could have been that the participants in the more affinity group had a period in which they could attend to learning more, whereas the participants in the less affinity group had a period in which they had to attend to teaching. This fourth reason might complement the second reason.

One final reason may be that the participants did develop and adapt their teaching practices, but not through reflection on the working environment and learning experiences at a later point. Rather, they may have engaged in experimentation with new teaching methods or keeping up to date with the latest literature (Oude Groote Beverborg, Sleegers, & van Veen, 2015c). Despite their initial willingness to participate, they may have found that making information and insights explicit by using logs did not befit them. Future studies could investigate for whom what knowledge content is discovered with what additional learning activities or other forms of reflection. All in all, using daily and monthly logs with open questions to study learning through reflection fitted better to some participants than to others.

For the discussion about the findings related to how the extent to which the overall level and the routine of the intensity of making information explicit co-occurs with the overall intensity of making insights explicit, we focus on the group of participants that was considered to have more affinity for the measurement instruments. We found that levels of overall reflection on the working environment positively related to levels of overall reflection on learning experiences. In this regard, it is relevant that information to be made explicit is always present in the working environment. Insights, on the contrary, can only be made explicit when learning experiences occurred. As such, the situated manner in which we assessed teacher learning through reflection corroborates findings from large-scale survey studies, which showed that engaging in learning activities more goes together with having more learning results (Oude Groote Beverborg et al., 2015a; Sleegers et al., 2014).

Furthermore, we found no clear relation between the measures of the routine with which teachers reflect on the working environment and their overall reflection on learning experiences. The regularity of making information explicit was unrelated to the overall level of making insights explicit. The persistence of making information explicit could be seen as negatively correlated to the overall level of making insights explicit, but the dispersion was high. To illustrate, from the top three participants in making insights explicit, one had the least and one had the most persistence in the intensity of making information explicit. Thus, the answer to the question about whether learning can be facilitated through reflecting very constantly or in bursts, is: both. The application of RQA thereby extends research on sequences of (multiple) learning activities (Endedijk, Hoekman, & Sleegers, 2014; Zwart et al., 2008). Moreover, these RQA-based findings suggest that constancy in reflection-intensity is not necessarily beneficial to school improvement and educational change (see also Mulford, 2010; Weick, 1996). Such constancy may, again, fit better to some than to others. Consequently, teachers cannot be discharged from the responsibility of finding out what manner of learning befits them personally, and colleagues can only seduce them to do so. Studies with additional measures and in additional contexts are needed to validate our findings concerning the constancy of everyday teacher learning.

How, then, to support teachers in sustaining levels of reflection without enforcing high constancy thereof (see also Giles & Hargreaves, 2006; Timperley & Alton-Lee, 2008)? An answer thereon may not be based on focussing on the routine of engagement in the learning activity itself, but by also taking the situated nature of the process in consideration (Barab et al., 1999). Our findings suggest that those participants that are able to make the most insights explicit are also the ones that manage to make information explicit whenever an opportunity occurs, which could be done by organizing such opportunities (i.e. moments of disengagement from the work flow, the use of evaluation instruments or logs, classroom observations, meetings, or appraisal interviews) with determined intervals, but also by being keen to have as many such moments as the working environment may provide each day, or a combination of both. Either way, the working environment would have to provide ample information that is salient and interesting enough to further think about and to distil a new way of acting from, whenever teachers have an opportunity to do so (Lohman & Woolf, 2001). In this respect, critically reflecting colleagues and transformational school leaders, who inspire, support, and stimulate, are crucial in helping to see the workplace in a new light and in providing examples of how one can synchronize one’s practice with newly found information (Hoekstra & Korthagen, 2011; Oude Groote Beverborg et al., 2015c; van Woerkom, 2010). Future research could investigate the development and dynamics of coordination of team members in creating such an interesting environment by engaging in knowledge sharing with the aim to co-construct shared meaning and to facilitate school improvement and educational change (see also Zoethout, Wesselink, Runhaar, & Mulder, 2017).

In sum, the findings of this study indicate that teachers who make more information from their working environment explicit are also able to make more new insights explicit. This suggests that higher levels of engagement in reflection are beneficial to teachers’ developments, and, by extension, to educational change and school improvement. The routine with which teachers make information explicit was found to be mostly unrelated to making new insights explicit. Of importance seems to be to reflect upon the working environment whenever an opportunity arises. Crucial seems to be that this (social) environment provides information that is salient and interesting enough to distil a new way of acting and attending from. Teachers might additionally benefit sometimes from organizing opportunities to become aware of information in the environment with a certain constancy. In this regard, the use of daily and monthly logs seems to fit better to some participants than to others.

This study is a first step in understanding teacher learning through reflection in the workplace as an everyday and ongoing process. The use of measurement instruments that generate dense time-series and the application of RQA to assess stability and flexibility over time shows that longitudinal research can concentrate on more than just on growth or couplings between variables over time (e.g. Hallinger & Heck, 2011; Heck & Hallinger, 2009; Heck & Hallinger, 2010; Oude Groote Beverborg et al., 2015a; Sleegers et al., 2014; Smylie & Wenzel, 2003; Thoonen, Sleegers, Oort, Peetsma, & Geijsel, 2011). Moreover, the study provides an example of how novel methodology, such as RQA, can be adopted to tap into professional learning as a dynamic and situated process in support of school improvement and educational change.

11.5.1 Limitations & Future Directions

The initial idea of the study was to dive deeper into the reflection process than presented here, by measuring what specific types of information teachers attended to using the daily logs, by measuring the contents of learning experiences using the monthly logs, by analysing the dynamics of attending to those types of information using categorical auto-RQA, and by establishing a relation between for instance the persistence in one type of information and the occurrence of a learning experience with a corresponding content. With this aim, we coded the daily and monthly log entries. However, the time-series generated per code-category were not dense enough for the application of RQA. Moreover, we assumed that setting a fixed time for reporting learning experiences would help generate a higher response rate. However, not knowing when learning experiences took place during the months made it very difficult to relate it to the information reported in the daily logs. Thus, the design failed to generate the timing information that would have been needed to be able to model the learning experiences’ occurrences. Having participants fill in learning experiences at (or very soon after) the moment they have them, would therefore have been a better approach. Additionally, our choice of measurement interval was a compromise between the expected rate of change with which salient information would be made explicit and the practical consideration of not wanting to burden the participating teachers too much. Our measurement intervals were therefore too crude for our initial purposes. In sum, measurement methods with a higher sampling rate, such as observations that happen in real-time, are needed to model how information in the working environment affords development and adaptation more accurately (Granic & Patterson, 2006; Lewis et al., 1999; Lichtwarck-Aschoff et al., 2012). Nevertheless, qualitative analyses on the data generated by the logs used in this study can be used to relate the contents that teachers reflected upon with the contents of what they learnt. This would still contribute to understanding more about the role of affordances in teacher learning, but the aim would no longer lie on finding systematic relationships (Barab & Roth, 2006; Greeno, 1994; Little, 2003; Maitlis, 2005).

We would like to stress that RQA’s derive their power from frequent measurements – and not from a large sample size. Whereas using small samples could constrain generalizability, studies assessing for instance the temporal pattern of teacher interactions in only one team in real-time, might provide important, new insights into the process of how teachers collaborate to make sense of the challenges they face and how that culminates in the generation of new knowledge or a shared meaning (e.g. Fullan, 2007). Additionally, such studies might prove very valuable for researchers, who are interested in the systematics of change processes and seek to combine the results of various studies in simulation studies (Clarke & Hollingsworth, 2002), rather than meta-analyses (see also Richter, Dawson, & West, 2011; Sun & Leithwood, 2012; Witziers, Bosker, & Krüger, 2003). By building on the current study, future research could contribute to a bottom-up understanding of how learning communities, but also change capacity of schools, emerge and continue to evolve (Hopkins, Harris, Stoll, & Mackay, 2010; Stoll, 2009). Another benefit of the proposed measurement methods and analyses, due to their focus on the circumstances and periodicities of individuals, is that they allow for tailored advice to individual teachers (or teams of teachers). Consequently, this approach to investigating professional learning would allow teachers and policy makers alike to formulate situated expectations about the pace of adaptation, the rate of innovations within a certain time, and delays in proficiency. An interesting follow-up question nevertheless concerns the extent to which diaries served as an intervention for fostering reflective learning and thus influenced the learning occurrences accordingly (Hoekstra & Korthagen, 2011). A new study with an experimental design and additional dependent measures would be needed to investigate this (Maag Merki, 2014).

Despite its limitations, this study does provide a first enquiry into studying teacher learning as a situated and dynamic process through the use of logs and RQA. In future research, the methodology could have utility in studying aspects of the dynamics of teacher learning such as, on an individual level, shifts in appreciation of the importance of certain classroom practices or differentiation in perception, or on an organizational level, alternations of periods of tight versus loose couplings between teachers, teams, or departments (see also Korthagen, 2010; Kunnen & Bosma, 2000; Mulford, 2010; Nonaka, 1994; Orton & Weick, 1990). The methodology could also help policy makers in balancing top-down and bottom-up processes in shaping the organization of the school (e.g. Feldhoff, Huber, & Rolff, 2010; Hopkins et al., 2010; Spillane et al., 2002; van der Vegt & van de Vliert, 2002). Moreover, by studying the temporal pattern of sensemaking processes in schools (see also Coburn, 2001; Feldhoff & Wurster, 2017; Spillane et al., 2002), more can be understood about the development of professional learning communities and the inner workings of the change capacities of schools. Consequently, in line with trends in accountability to focus on learning of organizations rather than fulfilment of inspection criteria, Inspectorates of Education could use the methodology to tap into a developmental process rather than only the results thereof in order to support the sensemaking processes in schools (Feldhoff & Wurster, 2017).

Appendices

Appendices

11.1.1 Appendix A

11.1.1.1 Daily Log 1(2)

11.1.1.1.1 Information

This question is about informal learning from colleagues in the workplace.

Informal learning can be seen as the daily discovery of information.

Information can be known or new, it can be positive or negative, and it can be something from the educational praxis or something from a conversation.

More concretely, you can think of information as something a colleague said; something that was recommended to you; something you experienced; the manner in which you did something; the feedback you gave someone; something you did not do; etc.

This question is about which information struck you the most today. Below you see four answer categories.

Below, you see four answer categories.

Choose one of the answer categories.

Later, you can choose a new answer category.

After you have clicked on one of the options, you will be presented with questions about the nature of the information that struck you.

After you have answered the questions about the nature of the information, you can choose one of the four answer categories again.

You can choose an answer category maximally three times, thereafter the diary entry of today will stop.

Try to use no more than 5 min for filling in today’s diary entries.

11.1.1.2 Daily Log 2(2)

11.1.1.2.1 Information

Where “colleague” is stated, you can also read “colleagues”.

You stated that you agreed with something a colleague said or did today.⁶

The following questions elaborate on that.

Try to answer the open questions in no more than three sentences.

11.1.2 Appendix B

11.1.2.1 Monthly Log

11.1.2.1.1 Learning Experience

Learning can occur everywhere and always. Learning can be planned and spontaneous. You become conscious of having learned something when you have had a learning experience.

You can think for example of a learning experience as having found a new way to prepare a task with your colleagues, or as having had an insight about how you can transfer something to your students after having had a conversation with a colleague.

The questions in the monthly log are about learning experiences that you have had in the past month. We kindly ask you to report three learning experiences.⁷ Each entry is about one learning experience. This is the entry of learning experience 1⁸.

Try to answer the questions in no more than three sentences.