1 Introduction

The need for discussing scaling up has been articulated by Adler, Ball, Krainer, Lin, and Novotná (2005) in the following way:

We know little about what happens when programs spread to multiple sites. We have also done less of studying what it means to scale up or what it means to extend a program that has worked in one setting to another setting—what works, what goes wrong, what do designers need to know and think about. (Adler et al., 2005, p. 376).

Indicators of an increased research interest are the ZDM special issue Evidence-based CPD: Scaling up sustainable interventions (see, e.g., the introduction by Roesken-Winter, Hoyles, & Blömeke, 2015) and the conference Educating the Educators (Maaß et al., 2015) focusing on international approaches to scaling up professional development in mathematics and science education.

There is knowledge about factors fostering the scaling up of educational innovations. For example Cobb and Smith (2008) highlight networks, shared vision and mutual accountability. Factors fostering the scaling up of educational innovations depend on the level where innovations are expected—at the individual level of teachers, local level of schools, or at the regional or national level. Regarding large initiatives aiming at innovations at some or even all of these levels, there is a lack of knowledge. This paper synthesises research results within the context of a multi-layered nationwide initiative in Austria, called IMST (Innovations Make Schools Top), with the purpose to deepen our knowledge regarding factors which foster scaling up educational innovations as well as challenges.

The overall goal of IMST is to establish a culture of innovative MINDT teaching (Mathematics, Informatik/computer science, Natural sciences, Deutsch/German language, Technology). Culture of innovation means starting from teachers’ strengths, understanding teachers and schools as owners of their innovations and regarding innovations as continuous processes that lead to a further development of practice, as opposed to singular events that replace an ineffective practice (see, e.g. Krainer, 2003). The establishment of a culture of innovation requires that activities gain impact at the individual level as well as at the local level as prerequisite for scaling up processes. In this paper, we synthesise empirical findings regarding the Austrian IMST initiative to analyse and identify relevant factors for scaling up. Since these factors depend on the level where innovations are expected, we scrutinised scaling up processes on the individual, local, regional and national levels. IMST operates on each of these levels. Due to this complexity, we synthesise different research approaches and results that have been conducted in and about this initiative to identify relevant factors. Within this meta-study, we aim at gaining knowledge that cannot be learned from a single research study. The theoretical framework used in this paper is based on diffusion of innovation (Rogers, 2003) and self-determination theory (Deci & Ryan, 2008), because self-determination theory (SDT) highlights the impact of a supportive environment as IMST represents it for developing stable self-determined motivation.

1.1 IMST—a nationwide learning and teaching initiative

As a reaction to Austria’s disappointing TIMSS 1995 results at the upper secondary level, the Austrian Ministry of Education (in the following, in brief called ministry) commissioned a research team working out suggestions for improving the situation. This gave birth to the research project IMST (1998–1999; see, e.g., Krainer, 2003). Reasons for the unsatisfactory results were multi-layered: among others, challenges refer to learning outcomes (e.g., poor achievements regarding higher levels of thinking), to teaching processes (e.g., fewer students in Austria than in most other countries were well involved in reasoning tasks in mathematics lessons), to the status of mathematics and science education (e.g., only at some universities there were professors and doctoral students within this disciplines and none at other teacher education institutions), to the situation of teacher education (e.g., weak exchange between primary and secondary teacher education) and (therefore) to the lack of a systematic support system for teachers and schools. The main message of the research results was that not only teachers need to learn but also the whole educational system, including principals, teacher educators, researchers, administrators, policy makers and their corresponding institutions such as schools, teacher education and research institutions, administration and policy bodies including the ministry, as well as the society itself. Thus, the idea of a “learning system” (e.g., Krainer et al., 2002) was launched.

IMST was further commissioned by the ministry for the period 2000–2004 with the aim to foster innovations at secondary schools and to propose a long-term plan for improving the situation. The proposal was to establish (starting with the school year 2004/2005) a nationwide support system for mathematics and science teaching in Austria, consisting of seven measures to be taken, whereas in the period 2000–2004, only mathematics and science at the secondary level had been in focus; since then, IMST was gradually enlarged to all school levels and some additional subjects. IMST was prolonged several times, experienced several adaptions, but is still running.Footnote 1 The seven measures (M1–M7; see Table 1) were allocated at the local, regional and national level as described below (see, e.g., Krainer & Zehetmeier, 2013).

Table 1 Status and years of implementation of measures M1–M7 as proposed by IMST
Full size table

Local level

An important measure is the establishment of an Austrian-wide fund where MINDT teachers can submit projects and get support to try out and document innovations (M6). The fund is structured into several thematic programmes which cover topics chosen (by a scientific jury) because of their importance for educational policy, practice and scientific community. Corresponding with the amount of budget, 80 to 200 innovations by MINDT teachers are supported by experienced teams of university staff and teacher experts each year. With the help of these facilitators, teachers participating in IMST thematic programmes write down their plans, experiences and evaluations in “innovation reports”, which are accessible on the wiki-Website of IMST (https://www.imst.ac.at/imst-wiki/index.php/Hauptseite). Another important measure, namely the establishment of a content-related middle management at all schools, has not been realised so far (M1).

Regional level

To scale up IMST, we began to establish regional networks (M4) in each federal state in 2003 (e.g., Rauch, 2013). Until 2006, IMST had signed contracts with all nine federal states, and the activities by each regional network were co-financed by regional institutions (school boards, teacher education institutes, economy, etc.). In four federal states, dedicated regional education experts founded smaller networks at the district level, thus contributing to further broaden IMST activities. To strengthen human resources in the field of mathematics and science education, IMST promoted (starting in 2006) the establishment of regional competence centres for didactics in several subjects (M3). The centres received starting money from IMST, but the long-term personal and financial basis had to be laid by regional institutions (universities, university teacher colleges, school boards, etc.). Due to the increasing interest of the federal states in such centres and the growing human resources in didactics, IMST started promoting the launch of Regional Educational Competence Centres (RECCs) in 2013. In the years 2014 and 2015, applications by institutions were examined whether they met the requirements for a RECC label (decided by a scientific jury); finally, 20 centres (five of them specialised in or including mathematics) were accredited as RECC by the ministry. Since the accreditation would have ended in 2018, a new—and even more rigorous—application and accreditation process was organised, leading to 14 institutions awarded the RECC label by the ministry in May 2018. Another measure aimed at establishing a content-related middle management (Krainer & Müller, 2007) in all nine federal states of Austria.

National level

An important measure at the national level is the establishment of Austrian Educational Competence Centres (AECCs) as lighthouses for didactical research and innovation. Since 2004, six AECCs have been established: one centre for instructional and school development (M7) and five for subject didactics, including mathematics, German language, biology, chemistry and physics (M5). The AECCs are, among others, engaged in planning and evaluating innovative research and development projects; they provide support for classroom teaching and monitor national developments and international programmes. The AECCs and the RECCs are partners of IMST in the support system to promote MINDT teaching in Austria.

There is networking between the measures at the local, regional and national levels. Teachers running small-scale projects in the regional networks might successfully submit a larger project in one of the thematic programmes; vice versa, experiences from such larger projects are presented in regional network meetings to encourage teachers to start small-scale projects. This helps the centres to build bridges to practice, to other academic institutions and fields and to become stronger partners of IMST and the educational system. This increases the intended deep interconnection between policy, research and practice.

IMST regards teachers as the crucial group contributing to the further development of learning and teaching. It is assumed that it is not possible to have a direct “transmission” of (general) knowledge from administrators, teacher educators and researchers to teachers; rather, the (specific) knowledge needs to be constructed by the teachers themselves (supported by colleagues, teacher educators, etc.). Indicating the limits of the classical research-development-dissemination (RDD) model of innovation, Schön (1983) introduced the term technical rationality into the educational discourse. This follows three basic assumptions: (i) There are general solutions to practical problems; (ii) these solutions can be developed outside practical situations (in research or administrative centres); and (iii) the solutions can be translated into practitioners’ actions by means of publications, training, administrative orders, etc. Reflective rationality, in contrast to technical rationality, follows (see, e.g. Altrichter, Feldman, Posch, & Somekh, 2008) quite different assumptions: (i) Complex practical problems require particular solutions; (ii) these solutions can only be developed inside the context in which the problem arises and in which the practitioner is a crucial and determining element; and (iii) the solutions can only rarely be successfully applied to other contexts, but they can be made accessible to other practitioners as hypotheses to be tested in practice. Therefore, the teacher is a key stakeholder in innovation and research. Policy regulations (e.g. educational standards) or research results play an important role in IMST; however, these top-down elements of educational steering need to be balanced with bottom-up based innovations by teachers. Thus, IMST regards teachers as experts who investigate their own teaching in a systematic and self-critical way (among others, writing innovation reports in the context of action research, see, e.g., Altrichter et al., 2008).

In this paper, we focus on two research questions, reflecting and summarising IMST findings of several studies at a meta-level (see Table 2):

  1. 1)

    Which factors fostered scaling up in the IMST initiative?

  2. 2)

    Which challenges regarding scaling up arose in the IMST initiative?

Table 2 Overview regarding IMST studies used in the meta-study
Full size table

Since these questions can hardly be answered experimentally at this numerical and time scale, we combine a series of empirical studies and theoretical frameworks for our meta-study. We use both to construct an argument for identifying fostering factors and challenges. Thus, to answer the research questions, we synthesise findings from empirical studies which were conducted about IMST with the help of two theories as lenses to identify these factors and challenges: the diffusion of innovation (DOI; Rogers, 2003) theory and the self-determination theory (SDT; Deci & Ryan, 2008). DOI focuses on conditions for successful innovations, whereas SDT explains the development of intrinsic motivation and the requirements needed for such a development to occur.

2 Theoretical framework

This meta-study uses two frameworks to identify relevant fostering factors and challenges for scaling up: diffusion of innovation theory and self-determination theory.

2.1 DOI

This framework supports explaining and predicting the adoption or rejection of new ideas and practices. Rogers (2003) explicitly highlights the “exciting potential contribution […] made by the education research tradition, stemming from the fact that organisations are involved, in one way or another, in the adoption of educational innovations” (p. 63). According to Rogers (2003), an innovation is defined as an idea, practice or object that is perceived as new by an individual or other unit of adoption within a social system. Rogers (2003) defines diffusion as a process by which an innovation is communicated through certain channels over time among the members of a social system (p. 5). He describes the innovation-decision through which an individual passes from (a) the antecedents to (b) the process further to (c) the consequences. One of the innovation’s antecedents are specific receiver variables, for example personality variables or the perceived need for the innovation (Rogers, 2003).

The characteristic of an innovation determine its rate of adoption, defined as “the relative speed with which an innovation is adopted by members of a social system” (Rogers, 2003, p. 221). The rate of adoption of an innovation can be explained by five characteristics: relative advantage, compatibility, complexity, trialability and observability. Since the adoption of IMST innovations is in the focus of this meta-study’s findings, these characteristics are described in detail:

Relative advantage includes the advantage one perceives through the innovation, thus the degree to which an innovation is perceived by potential adopters as better than the idea it supersedes. This also includes delineating what is to be gained from adapting the innovation. An innovation with greater relative advantage will be adopted more rapidly. Compatibility denotes the degree to which an innovation is perceived as consistent with the potential adopters’ needs, existing values and past experiences. The more compatible it is, the more rapidly it will be adopted. Complexity includes the potential adopters’ estimation of how difficult the innovation is to understand and how complex the application for the own field is. More complex innovations are adopted rather slowly. Trialability denotes the possibility of potential adopters to test an innovation (at least on a limited basis). Innovations that can be tested in small steps provoke less uncertainty and will be adopted as a whole more rapidly. Observability points to the claim that innovations, which are visible to other people and institutions are more likely to be accepted and adopted. The easier it is for individuals to see the results, and the more opportunities they have to observe its potential impact, the more likely they are to adopt the innovation.

2.2 SDT

The success of initiatives in the sense of a movement like IMST depends to a high extent on the motivation and self-initiative of the teachers involved. If they are willing to innovate their work in schools, IMST can support them. If they are not motivated to change their instructional strategies, IMST is not able to coax them. IMST only functions properly if the involved teachers show self-initiative. We learned that teacher’s autonomous motivation is a crucial factor for IMST. Autonomously motivated teachers are predestined to act as “up scalers” of innovative ideas because they do not depend on external motivating factors when working on and spreading their ideas. The fact that IMST only works with motivated teachers led us to investigate the conditions that teachers need for working with stable self-determined motivation. SDT (Deci & Ryan, 2008) with its focus on supportive conditions and environments seems to be predestinated as theoretical background.

For Deci and Ryan, it was not sufficient to distinguish only between intrinsic (motivated by the task itself) and extrinsic (motivated by an external source). They emanate from different regulatory styles of motivation that are influenced by the degree of perceived self-determination versus perceived external control. A low level of perceived self-determination leads to external regulation that conforms to the classical definition of extrinsic motivation, for example to get rewards or prevent punishments. The highest form of perceived self-determination is associated with the classical understanding of intrinsic regulation that represents behavior primarily motivated by the task itself. In literature, intrinsic motivation is often used as a synonym for interest, where the task itself provokes motivation. In contrast, the more a person feels controlled by their environment, the more their motivation is regulated externally.

The degree of perceived self-determination is influenced by different aspects. One of the most important factors is the fulfilment of the three psychological basic needs autonomy, competence and social relatedness (e.g., Deci & Vansteenkiste, 2004). If we do not perceive ourselves autonomous in, for example choosing an activity or choosing how to act, we feel externally controlled and cannot establish self-determined motivation. The feeling of autonomy may occur not only when a person can choose what they want to do but also when or how a specific task can be handled. Furthermore, if we do not feel that our teachers are supportive for achieving competence, we also have a problem in developing self-determined motivation. The need for competence is fulfilled when a student is confronted with achievement situations that can be achieved successfully. This enhances student’s self-confidence and leads to a higher willingness to accept the next challenge. Teachers bear responsibility that their students are confronted with learning situations in which they feel challenged, not threatened. Lastly, the need for social relatedness means that we need supportive groups and functioning social networks. For the development of self-determined motivation, we depend on our social environment. A positive classroom climate perceived as socially supporting by students is a crucial factor for the development and stabilisation of sustainable intrinsic motivation. On the basis of SDT, Pelletier, Séguin-Lévesque, and Legault (2002) found that teacher and student motivation is associated positively. Similar findings about the general positive connection between teacher and student motivation are reported by Schiefele and Schaffner (2015) as well as by Kunter et al. (2013).

2.3 Combining DOI and SDT

In this meta-study, we combine both DOI and SDT with the aim to identify relevant fostering factors and challenges for scaling up. SDT describes aspects of personality and—contrasting stable trait approaches—personality development. When individuals have developed a mature personality with an elaborated ability of self-regulation, they feel self-determined in living their lives according to their goals and identity, in choosing and doing activities (see, e.g., Ryan, Kuhl, & Deci, 1997). As described above, Rogers (2003) emphasises that personality aspects are in the forefront whether an individual rather tends to adopt an innovation or not. The perceived fulfilment of the three basic needs leads to a motivated and self-regulated personality.

SDT describes which characteristics foster motivation on the individual level. Those characteristics normally come from the social environment: The fulfilment of the basic needs depends on other people or a social system. Most of the time, innovations are brought to a person’s attention over other people or a social system. If one or more of an innovation’s characteristic meet a person’s basic needs, we can speak of a fit between the person’s needs and the innovation’s characteristic. In more detail, Rogers’s innovational characteristic relative advantage can meet all three psychological basic needs: A person can perceive a relative advantage through a better fulfilment of the need for autonomy, competence or social relatedness. Further, the innovation’s characteristic compatibility is associated with the need for autonomy: If the innovation’s aspects are in line with the attributes of a person’s work and interests, the person will perceive no constraints through the innovation regarding their autonomy. If an innovation is not too complex and can be tried out, the need for competence is met: A person does not feel overwhelmed and is motivated to try out the innovation. Rogers’ fifth innovation’s characteristic observability can lead through higher visibility to a perceived fulfilment of social relatedness: What is observable has the potential to lead to a corporate feeling.

3 Methods

As announced in the introduction, our meta-study synthesises studies about and within the IMST initiative. In general, research within IMST is a conglomeration of approaches: It is conducted not only by researchers from different fields (e.g., pedagogy, psychology or subject didactics) but also by teachers themselves (action research). Three books (in German), many research papers and other kinds of publications have been produced in the context of IMST, however, none of them directly focusing on factors fostering the scaling up of educational innovations.

In particular, our criterion for selecting a subset of studies to be used in this meta-study was twofold: (a) Each study provided data that helped to answer the meta-study’s research questions, and (b) each study referred to a different level (individual, local, regional and national). This leads to the selection of studies A, B and D (see Table 2).

Moreover, to gain further insights through new relevant data, and thus being better able to answer the research questions, we conducted two small additional studies (C and E; see Table 2): Study C complements the results at the local level by investigating existing innovation reports, in particular examining teachers’ dissemination efforts; study E complements the results at the regional/national level, in particular exploring selected leaders’ views on fostering factors and challenges for scaling up.

In detail:

  1. A.

    At the individual level, focusing on teachers’ and students’ motivation, we report on some already published studies (see references in Section 4) taking a traditional evaluation design (mostly without control group), primarily basing on self-report questionnaires. We investigated students’ interest, anxiety and self-concept, basing on the Austrian PISA Study (Haider & Reiter, 2003). Furthermore, we asked students to fill out a self-determination questionnaire (adapted translation of a questionnaire constructed by Ryan & Connell, 1989) and a learning environment questionnaire regarding support of autonomy, competence, social relatedness, teacher’s engagement and relevance of content. On the teacher level, we surveyed teachers’ self-determination (motivation to teach; using an adapted translation of a questionnaire constructed by Otis, Grouzet, & Pelletier, 2005) as well as the teachers’ perceived student motivation. Teachers were also asked to complete a work environment questionnaire with the following dimensions: school government, colleagues, parents, administration and public (using an adapted translation of a questionnaire constructed by Pelletier et al., 2002). Data were analysed statistically. To find connections between the different dimensions, we conducted correlational analyses, path analyses as well as structural equation modelling on the basis of partial least squares and maximum likelihood algorithms.

  2. B.

    At the local level, we refer to already published research (see results section) where data collection was done during 2001 and 2010 and contained documents and archival records (including teachers’ innovation reports) as well as ex-post interviews with teachers, teachers’ colleagues, principal and former project facilitators. Data was analysed by qualitative content analysis (Mayring, 2014) to identify common topics, elaborate emerging categories and gain deeper insight into teachers’ professional growth over time.

  3. C.

    To generate insights into IMST teachers’ efforts to disseminate innovations that go beyond a particular case of a teacher, we conducted a new study. We analysed teachers’ innovation reports (more than a thousand can be downloaded from the IMST-Website). Selection criteria for these reports were a clear focus on mathematics and their realisation in primary and secondary schools. This selection of innovation projects was differentiated into two status levels, follow-up projects and new projects. To examine dissemination efforts, qualitative content analysis was applied, in particular, the procedure of inductive category formation (Mayring, 2014). Finally, each innovation project was allocated to dissemination levels. This should allow us to analyse to which extent follow-up projects and new projects lead to different dissemination levels (a higher level indicating a better scaling-up of innovations).

  4. D.

    At the regional and national level, an already published paper (using a questionnaire and interviews) on the status and role of subject didactics in Austria (see results section) is reported, with an emphasis on findings related to the scaling up issue.

  5. E.

    To explore selected centre or network leaders’ views on fostering factors for scaling up, we conducted a small new study, based on semi-structured interviews. Our selection criteria for the representatives included (a) implementation of mathematics-featured activities, (b) regional distribution all over Austria and (c) status of the represented group (two centres, two networks): One leader was a representative of the AECC mathematics and another leader a representative of a RECC mathematics (see section about IMST); two other leaders were representatives from two regional networks (all stemming from different federal states). Some days before the interview, the four interviewees got short descriptions of the eight characteristics. The interviews were audio-recorded and transcribed. A clean read or smooth verbatim transcript was constructed and analysed by using an Open Access Qualitative Content Analysis software (Mayring, 2014). Both deductive (using Rogers’ DOI and Deci & Ryan’s SDT as theoretical background) and inductive category formation methods were applied. In study E, we confine ourselves to present a descriptive statistics, counting how often the four chosen leaders used the five DOI and the three SDT characteristics in describing fostering factors for scaling up innovations (related to projects in their centres or networks).Footnote 2 This allowed quantifying leaders’ usage of fostering factors, providing a rough picture of which factors leaders ascribe relevance to promote scaling up. A second, however minor, focus was on identifying categories regarding challenges raised by the leaders. Since the interviews did not explicitly ask for identifying challenges, only a few cases of challenges were expected. Thus, two authors jointly identified the most prominent challenge for each leader.

4 Results

As mentioned earlier, this paper reports a meta-study. The results are written as an argumentative story: Two theoretical frameworks (DOI and SDT) are used as lenses to synthesise empirical findings and identify fostering factors and challenges for scaling up.

4.1 Individual level

Findings from A

The fulfilment of the students’ basic needs autonomy, competence and social relatedness by teachers and a positive correlation between teacher and student motivation are essential for spreading a positive attitude towards learning and for scaling up. These connections are even more crucial for mathematics and science subjects where students tend to be overwhelmed or demotivated. Overstraining and demotivation lead to less willingness to deal with a difficult subject because of growing anxiety (e.g., Beilock & Maloney, 2015). Against the background that the satisfaction of the basic needs leads to more enthusiasm to learn, we were interested to what extent basic needs satisfaction within IMST projects takes place. Since 2006, more than 4500 students and their teachers from different school types participated in our surveys. On the basis of these data, we conducted different analyses, which we summarise in the following.

The fulfilment of the basic needs is a prerequisite for the development of self-determined motivation and in this vein a potential factor of fostering scaling up processes. Within IMST evaluation, Müller (2010) showed that students perceive a high fulfilment of the basic needs through one school year. This led us to the assumption that teachers who conduct an IMST project are competent in fulfilling the basic needs of their students. We investigated to what extent insufficient need satisfaction during adolescence affected the decline of intrinsic motivation (Gnambs & Hanfstingl, 2015). Analysis showed that after control of the basic needs as covariates, intrinsic motivation stayed stable, whereas the control of the need for competence made intrinsic motivation even slightly increase. The stabilisation of intrinsic motivation was influenced by the level of basic needs satisfaction.

Hanfstingl, Andreitz, Müller, and Thomas (2010) focused on the mediating role of teachers’ person-related variables between the perceived three psychological basic needs and intrinsic motivation to teach. As expected, also on a teacher level, the perceived fulfilment of the three basic needs is highly correlated with intrinsic motivation. However, within IMST and through our experience as teacher educators, we realised that some people have—from an objective point of view—a fulfilment of basic needs, but they are not really intrinsically motivated for their work as teacher. In contrast, there are some teachers who seem to be highly intrinsically motivated with a low fulfilment of the basic needs. This evidence seems to speak against theoretical assumptions. Thus, we investigated whether specific personality aspects are co-responsible for the level of intrinsic motivation. We found that not only the fulfilment of basic needs is required for the development of intrinsic motivation but also a specific personality factor (responsible for identity development but also the development of self-congruent goals). This result goes in line with findings in personality research (Kazén, Kuhl, & Leicht, 2014) but also with the DOI related assumption that there are relevant personality factors which influence the need for innovations, whereas Rogers (2003) cited more traditional concepts of personality than self-regulation and self-control. When we conducted the study, we emanated from a SDT and a personality view. However, what we found confirms Rogers’ thesis that teachers’ personality does matter for the willingness of conducting innovations.

To sum up, we found in our analyses that IMST especially attracts and supports teachers who are highly motivated to invest additional efforts into their teaching. Within IMST, we can speak of a preselected group who is motivated to improve their instructional work and their relationship with students. Furthermore, teachers also pass their motivation to students through a high fulfilment of students’ basic needs (Müller, 2010; Müller, Hanfstingl, & Andreitz, 2009). Regarding our first research question, we can say that within IMST, teachers join with higher motivation, they feel motivated by the many supporting opportunities IMST offers and they pass this positive movement to their students. This characterises a matrix for scaling up opportunities. Due to the transfer of a motivating working style from teachers to teachers and from teachers to students a positive relationship to work, learning and especially innovations can grow. Regarding our second research question, IMST is especially challenged by providing support for those teachers who are not (yet) motivated to work on instructional improvements. A supporting approach does not work only on the individual teacher’s level but has to reach organisational level. Here, the targeted inclusion of school principals who have leading responsibilities could be a practicable strategy. Principals have a high impact on teachers’ working style (e.g., Pashiardis & Brauckmann, 2014; Tschannen-Moran, 2009).

4.2 Local level

Findings from B

In the following, we present a case study which allowed for the acquisition of more data about one particular project and thus a deeper analysis of one specific case in which the interplay between the various factors present in DOI and the basic needs of SDT are discussed in more detail with regard to sustainability of impact (see, e.g., Zehetmeier & Krainer, 2013).

Barbara (pseudonym; a secondary mathematics teacher) took part in IMST from 2000 to 2002. She was revisited by a researcher in 2005 and 2010 to gather data for a case study regarding the sustainable impact of her project. A comparison of the 2005 results with data from 2010 allows discussing the factors which fostered the sustainability of impact.

Barbara’s main objective was to support students’ inquiry-based learning (IBL) by integrating open learning environments (e.g. students are working autonomously in learner-centred classroom settings) into her mathematics classes. After this project, she planned to put even more effort into establishing open learning settings. Moreover, she wanted to get feedback from her colleagues regarding these teaching practices. The central idea of all these activities was to create IBL environments for students; in particular, independent and autonomous learning processes as well as individual goals and working schedules were fostered.

Concerning the case of Barbara, the following exemplary results (regarding the sustainable impact of Barbara’s project) were found (Zehetmeier & Krainer, 2013): Barbara’s beliefs regarding IBL and open learning environments changed since she could see their positive effects on students’ content knowledge, as well as on their self-confidence. This impact was sustained: Barbara still held these beliefs in 2005 and 2010. This enabled her to continue carrying out innovative teaching methods. She developed an inquiry stance towards her teaching mirrored by her belief about the value of feedback: In 2005 as well as 2010, Barbara was still convinced of the importance of critically evaluating her teaching. Even after the programme’s termination, she continued to actively facilitate her students’ discussion about her teaching practices. Her activities also inspired other teachers in the school develop an inquiry stance in their classrooms.

One of the central factors fostering the sustainability of impact was the engagement of the school’s principal. The school also had an efficiently organised management and school development structure. Additionally, Barbara experienced personal benefit, which helped the impact persist after the programme’s termination. Yet another fostering factor was represented by the IMST workshops and seminars, in which teachers could share their experiences. This supported Barbara to serve as a model for other teachers at other schools.

Concerning this paper’s focus on scaling up, impact on the local level was fostered by SDT’s basic needs competence and social relatedness (Deci & Vansteenkiste, 2004), as well as DOI’s relative advantage, compatibility, trialability and observability (Rogers, 2003). Barbara gained new insights into open learning environments and established IBL activities within her classroom. She developed an inquiry stance towards her own teaching and could share her expertise with other teachers during IMST seminars. Thus, her increasing competence was a central fostering factor. Furthermore, Barbara’s activities were embedded into a wider social community. Within her school, a team of colleagues also participated in IMST. Additionally, she got feedback from other (non-participating) colleagues concerning her teaching practices. She could also discuss her experiences with other schools’ teachers during IMST seminars. In sum, her efforts were fostered by social relatedness.

The case study highlights that Barbara experienced personal benefit which was perceived as an important relative advantage. She was able to integrate IBL strategies into her daily classroom activities. This compatibility fostered the impact of her efforts. Barbara’s school had an efficiently organised school development structure. This productive and constructive interplay between teacher and institutional level allowed her and the other school’s teachers to experiment with innovative teaching practices in small steps, which—in turn—supported the innovations to scale up. This trialability provoked less uncertainty and, thus, fostered the adoption of new ideas. Impact was also fostered by observability, since Barbara’s activities were visible both within her school and beyond: Barbara’s principal and her colleagues supported her efforts, whereas during IMST workshops and seminars, colleagues from other schools were informed about and inspired by her activities. On the one hand, this case study seems to show primarily a shining example for scaling up innovations through the support of IMST; on the other hand, looking more deeply into the case, it also shows the limitations of IMST (or of teacher education initiatives, in general): Without an efficiently organised school development structure and a strong principal, it would have been unlikely for Barbara’s project to be so successful.

Findings from C

In the following, we describe the findings of a study we conducted especially for this paper to generate knowledge that goes beyond a particular case of a teacher. The study focused on the following research question: Do innovations disseminate more often beyond a teachers’ respective classroom or school in projects that are applied more than once? To answer this question, a selection of innovation reports was analysed regarding teacher’s dissemination efforts.

Of all the projects submitted between the start of IMST in the school year 2004/2005 and the latest finished period in 2014/2015, n = 124 projects with a clear focus on mathematics were analysed. Of these, 52 projects were follow-up projects and 72 new projects. The descriptive analysis leads to several levelsFootnote 3 of dissemination efforts, which two of the authors merged into three levelsFootnote 4: Dissemination on level 1 contains no spreading of a teacher’s innovative ideas beyond his/her own classroom. Dissemination on level 2 signifies that teachers managed to disseminate beyond their own classroom, but their dissemination efforts ended within their respective schools (e.g., collaboration with colleagues or presentation of projects during open days or in the annual school report); level 3 includes dissemination efforts that go beyond a teacher’s own school, for example holding seminars in other schools and/or tertiary institutions or giving talks at conferences. For each teacher, the highest effort of dissemination was identified.

After applying these coding guidelines, the data were analysed quantitatively. The cross-tabulation below (Table 3) indicates that teachers carrying out follow-up projects show higher levels of dissemination (reaching beyond a teacher’s class and school).

Table 3 Crosstabulation of dissemination within IMST projects (school years 2004/2005–2014/2015)
Full size table

According to Rogers (2003), having substantial financial resources available and being able to apply complex knowledge are important prerequisites for innovators. Thus, by providing professional and financial support,Footnote 5 IMST might not only have served as a steppingstone for the development and refinement of their teaching ideas but might also have encouraged innovative individuals to disseminate.

Seeing a benefit in an activity (relative advantage) is surely a motive to do it (trying out an innovation) once more. Also, achievement and success enhance self-confidence and lead to a higher willingness to accept the next challenge (fulfilling the need for competence) and to try out another innovation.

To give meaning to an innovation, an individual should have the possibility to see how an innovation works “under one’s own conditions” (Rogers, 2003, p. 258). Since MINDT teachers are supported by IMST facilitators in starting teaching projects, they can personally try out innovations (trialability), which helps to “dispel uncertainty about a new idea” (p. 258). Thus, in addition to relative advantage and need for competence, trialability can be seen as an important characteristic in disseminating teaching ideas at the local level.

The following example briefly sketches a teacher’s attempts in disseminating her ideas beyond her own classroom and school. Supported by IMST facilitators,Footnote 6 a mathematics teacher at an upper secondary school introduced a remedial course for all first year students to enhance their knowledge of mathematics found to be inadequate in some students transitioning from lower secondary schools. During the first run, the teacher could encourage four colleagues from her school to support her; during the second run, she managed to win over the whole school. In addition to that, a digital mathematics schoolbook was developed with examples created by students. During the third run, additional chapters regarding the new mathematics MaturaFootnote 7 in Austria were added. Due to receiving several awards for her work, other schools had come to notice her efforts and also started to use e-learning schoolbooks with examples developed by students. During the fourth and last run, a Moodle course was added in which all vocational schools in Vienna were invited to upload exercises relevant for the new Matura.

Again here, the dynamics of bridging between a teacher’s learning and a whole school’s learning is evident. There are good examples within IMST where such bridges were built rather quickly: Even regarding new projects, nearly half of them (47%) achieves a higher level of dissemination; however, in the majority of new projects, the scaling up remains a challenge, at least in the first year. In follow-up projects, the rate of projects achieving a higher level of dissemination increases to more than three fourths (77%). On the one hand, this shows progress; on the other hand, it indicates that scaling up is a challenging process, combining individual and organisational learning.

4.3 Regional and National Level

Findings from D

In the following, we briefly summarise the findings of the already published study on the role of subject didactics in Austria (Krainer et al., 2012). The study involved experts in subject didactics and administration at Austrian teacher education institutions. It shows that considerable impulses for subject didactics occurred at universities and university colleges in which subject didactic-related centres and professorships had been established. Twenty regional centres were accredited as RECC in Austria by 2015, five of them specialised in or including mathematics. In 2018, again five RECCs related to mathematics (out of 14) were accredited. In addition, in all nine regional networks of IMST, subject-related activities are launched (see, e.g., Rauch, 2013). Thus, there is a broader support system for teachers at the regional and national level, which implies a higher likelihood that innovations by teachers are observed, supported, disseminated, etc. This can be regarded as fostering the DOI characteristic trialability. In mathematics, AECC mathematics was an important stakeholder, in particular by supporting the establishment of the mathematics Matura and the educational standards for mathematics in Austria (e.g., Kröpfl & Schneider, 2012), by networking Austrian experts in mathematics didactics (e.g., organising the subject didactics day at the yearly IMST conferences) and by providing many professional development initiatives nationwide.

By supporting the creation and further development of nationwide networks and centres, IMST contributes to the development of human resources and infrastructure in the field of MINDT subject didactics, which in turn increases the number of experts being able to support MINDT teachers and schools carrying out innovations. This can be interpreted as a contribution to fulfil the need for competence of teacher educators in subject didactics and teachers in teaching. Despite some progress, there is still a considerable lack of personal resources in MINDT education in Austria, in particular regarding experts with PhD and habilitation in primary education.Footnote 8 This challenge has consequences not only for subject didactics as a scientific field but also for teaching practice and educational policy.

Findings from E

As a complement to study D’s focus on the organisational level, insights into the individual level seemed necessary. Therefore, we conducted a small exploratory study especially for this paper, investigating centre and network leaders’ views on important factors and challenges concerning scaling up. The study comprised interviews with four experts (the leader of AECC mathematics, the leader of one RECC mathematics, and two leaders of regional networks). All four interviewees reported on activities to improve mathematics teaching in their region or (in the case of AECC mathematics) nationwide, by reflecting on the importance of the five DOI and the three SDT characteristics as possible fostering factors for scaling up. The attribution of all text components in the transcripts to one of the eight characteristics showed the need for competence and relative advantage as the most frequent occurring factors (19–20 counts), followed by need for social relatedness, trialability, observability and need for autonomy (6–11 counts); complexity and compatibility occurred only a few times (1–3 counts).

Most challenges regarding scaling up innovations that leaders articulated related to students’ or teachers’ competences. This mirrors the decision of the ministry to focus on “competence-oriented teaching” (e.g., introducing educational standards and a partly centralised Matura). One leader of a regional network stated that the introduction of the new mathematics Matura in Austria caused a lot of pressure on many teachers. It was very helpful that these teachers received timely support through professional development provided by the IMST network (and thus fulfilling the need for competence). A leader of a regional mathematics education centre indicated the pressure on teachers to improve their mathematical knowledge, in particular regarding stochastics which was not part of the earlier teacher education curricula. Also in this case, the network supported teachers to cope with the new Matura. Communication was considered as a crucial means to both decrease anxiety and make clear that there is no other choice than to participate in professional development. The leader of the national centre for mathematics education, which was co-responsible for the conceptualisation of the new mathematics Matura, indicated an additional view on this innovation: She stressed its importance as a necessary paradigm shift that puts students’ basic competences into focus (raising another issue of fulfilling the need for competence): Teachers need to understand that only focusing on their input—as it was mostly in the past—is not sufficient; it is important to focus on both teachers’ input and students’ output, their competences. The second leader of a regional network also focused on students’ competences, but in the context of a mathematics competition, the network organised. The leader highlighted on the one hand the importance of individual interest in and the joy of doing mathematics (cognition and emotion) and on the other the social dimension of the activity (working as a team, having fun, strengthening class community).

5 Discussion

The importance of innovations and teachers’ (intrinsic) motivation to create, try out and investigate these innovations led us to use DOI and SDT as frameworks for reflecting on the scaling up of IMST. Since IMST is not about (teachers) merely adopting innovations (from research and/or policy), the question needs to be raised whether the idea of adopting (innovations) needs to be adapted; in particular, it does not make sense to frame the work of IMST as transmission of something prefabricated (by researchers and/or policy makers) that teachers only have to put into practice. By giving teachers the opportunity to build on their specific context, to go their own ways (autonomy), to do it critically and jointly (social relatedness), to regard them as experts of change (competence) and to give them a chance to see the benefit of doing it (relative advantage), IMST encourages their self-determined role as change agents. In this sense, we understand teachers’ adoption of innovations as an adoption based on their autonomous and (at the same time) networked actions, and their self-critical reflections, accompanied by research-based support by external experts (experienced teachers or researchers coming from outside). However, how can an initiative working with highly motivated teachers scale up to many more or even all teachers?

This leads to the two research questions of this paper:

  • Which factors fostered scaling up in the IMST initiative?

  • Which challenges regarding scaling up arose in the IMST initiative?

In the following, we discuss the results regarding these two questions at the individual level, local level and the regional and national level; this is followed by a discussion bringing together all three levels.

Individual level (studies A)

Regarding the first research question, we found (with the lense of the SDT framework) that teachers pass their motivation to students through a high fulfilment of students’ basic needs. It is essential that innovations reach students. DOI does not focus directly on this impact, but relative advantage and observability help explaining this issue: Teachers see a relative advantage (only) if—through an innovation—their students learn (better) and their learning becomes (more) visible. The evidence that teachers feel motivated by the many supporting opportunities IMST offers can be regarded as confirming the importance of relative advantage. Since IMST teachers to a high extent define their innovations and steps of implementation, the factors compatibility and trialability are apparently crucial.

The second research question refers to the challenges of IMST. Although our analyses and the reports by teachers show that they in general estimate their participation very positively and they grow professionally, in particular, one challenge of scaling up remains: Does participating teachers’ growth have an impact on other teachers’ growth or even the whole school? To scale up the initiative, more elements of community building (e.g., working with the whole group of mathematics teachers) and/or organisational development (e.g., making good teaching as a leitmotif for the whole school) seems to be a viable option to be considered. Teacher leaders and principals are valuable stakeholders to support teachers’ innovations and their scaling up.

Local level (studies B and C)

Regarding the first research question, we found in study B (referring to one teacher) that SDT’s basic needs competence and social relatedness, as well as DOI’s characteristics relative advantage, compatibility, trialability and observability, were crucial in fostering impact. Study C showed that innovative teaching ideas disseminate more often beyond a teachers’ respective classroom or school when applied more than once. Repeated trying out of an innovation underlines the special importance of the DOI characteristic trialability, providing the learning opportunity to focus on an idea over time and thus go into more depth. Going deeper (quality) and showing continuous effort over a longer period (time) increase other teachers’ interest: they might realise that working on a specific innovation must be beneficial for this colleague and maybe for oneself and others, too.

The second research question relates to challenges. Study B indicates that the successful development of an efficiently organised school development structure depends on a strong principal. Study C showed that innovative teaching ideas disseminate stronger in follow-up projects as compared to new projects. However, even follow-up projects struggle with reaching higher levels of dissemination. This shows that scaling up combining individual and organisational learning is a challenging process. An individual’s wish to promote is one side of the coin, the other is a school’s (or a system’s) readiness to change.

Regional and national level (studies D and E)

Regarding the first research question, we found in study D that centres and networks are crucial elements of initiating impulses for the subject didactics in Austria. These centres and networks seem to fulfil the need for competence both for teacher educators as experts in subject didactics and for teachers. In turn, the existence of active (regional) centres and networks, near the working place of teachers and teacher educators, increases the likelihood of sharing experiences and of disseminating innovations. This nearness can be regarded as a relative advantage to foster innovations in teaching, both at schools and in teacher education. Study E shows that need for competence and relative advantage were the most frequent factors that centre and network leaders used regarding the issue of fostering the scaling up of innovations.

The second research question addresses a challenge that refers to the status of MINDT didactics in Austria. Study D indicates a considerable lack of personal resources in MINDT education in Austria, in particular in primary education. Study E shows that centres and networks are helpful in coping with new national affordances (e.g., like the new Matura); however, these centres and networks need to be stabilised, which in turn requires qualified personnel.

6 Meta-view and outlook

In this final section, we discuss our results regarding the two research questions from a meta-level, overlooking all three levels, and we sketch ideas how to further develop the design of IMST.

Related to the first research question, we found that all factors (five DOI and three SDT) are to some extent relevant; in particular, when using these theoretical lenses, most often we saw evidence of need for competence, relative advantage and trialability.

Regarding the relevance of DOI and SDT, a high relative advantage for adopters was identified as the most important attribute. This is in line with the information Rogers (2003) provides about relative advantage, namely that diffusion scholars have identified this attribute “to be one of the strongest predictors of an innovation’s rate of adoption” (p. 233). The most important relative advantage was that teachers could gain a more profound knowledge of the subject (indicating a link to SDT’s need for competence) and felt thus more secure in dealing with students’ questions and that students felt more confident in applying their knowledge gained and were thus more motivated during the lesson. High trialability and high observability also played a crucial role in the successful scaling up of projects, whereas low complexity was less relevant. The characteristic compatibility represented a special case since inconsistency with past values or experiences did not necessarily mean that the innovation was met with more resistance. In the context of educational innovations, it seems that fulfilling the needs of potential adopters has more influence on their rate of adoption than compatibility with past values and experiences. As far as the basic needs are concerned, fulfilling the need for competence was seen as most relevant.

Related to the second research question, we found three major challenges regarding scaling up that arose in the IMST initiative.

  1. a)

    A first important challenge is to optimise the interconnection between individual teachers’ learning and subject-related school development. Here, a subject-related education management (see, e.g., Krainer & Müller, 2007) at schools (e.g., department heads at secondary schools who further develop the quality of teaching together with their colleagues) and principals (who realise the importance of subject-didactics and support these departments) could play an essential role. There is a high (but so far not optimally used) potential that principals foster educational innovations and subject-related collaboration among teachers (Krainer & Peter-Koop, 2003); however, they need a kind of “leadership content knowledge” (Cobb & Smith, 2008). The organisational context is often underestimated in (mathematics) teacher education, like the genuine role of the subject is often underestimated in school development and corresponding research.

  2. b)

    A second important challenge is to combine bottom-up based innovations by teachers with top-down elements of (research- and policy-based) educational steering. This means not only to continuously build on teachers’ autonomy and motivation but also to build on schools’ responsibility to use professional development offers that bring new elements from outside to schools (policy-relevant innovations like educational standards or new findings in subject didactics). Like in many other countries, in Austria, more school autonomy (introduced by law in June 2017) goes hand in hand with more accountability. This includes writing “development reports” by schools. The reflection on our studies supports the idea of having subject-related chapters in these development reports. This could stimulate joint discussions among teachers (of mathematics and of other subjects) about the quality of teaching, about how to improve it and how to evaluate the success. In particular, looking at the new Matura in Austria, the paper indicates the following: A top-down-introduced initiative like the Matura benefits from networks as bottom-up initiatives being able to flexibly cope with new challenges.

  3. c)

    A third important challenge is the considerable lack of personal resources in MINDT education in Austria, in particular in primary education. On the one hand, it shows the importance of IMST in helping to establish centres and networks; on the other hand, it shows that much more needs to be invested in fostering MINDT education.

These findings were to a large extent generated from reflecting on the scaling up of educational innovations. Discussing these challenges leads to valuable starting points to critically reflect the current design of IMST. Thus, the meta-analysis study not only brought insights regarding the two research questions but also generated ideas for further developing the initiative after 2018, with schools and principals having higher autonomy and more responsibility.