1 Introduction

Digital technologies in the mathematics classroom have the potential to support mathematical learning by means of new representations, actions and pedagogies (Hegedus et al. 2017; Ball et al. 2018). But despite great progress in the development of teaching ideas and theoretical lenses, digital technology in the classroom remains underused and only rarely takes advantage of the potential suggested by research and policy (Bretscher 2014; Drijvers 2019). The reasons for this “quantitative and qualitative gap” (Bretscher 2014, p. 43) are multifaceted. Besides factors such as school culture, departmental support, curriculum and national policies, it has become clear that the most important factor is the teacher, who has to orchestrate technology use in a systematic way and guide the manifold processes of instrumental genesis in the classroom (Drijvers 2019; Hegedus et al. 2017). In light of this recognition, it is acknowledged that teachers need specific competencies in order to integrate digital tools successfully. For example, the framework of “pedagogical technology knowledge” (PTK; Thomas and Palmer 2014) highlights that teachers not only need a special kind of knowledge for technology implementation, but that beliefs play a crucial role since they frame, guide and filter situations, actions and intentions (Fives and Buehl 2012; Pierce and Ball 2009; Thomas and Palmer 2014). These beliefs comprise, in particular, (i) beliefs about teaching with technology, (ii) self-efficacy beliefs and (iii) beliefs about the nature of mathematics and mathematics learning (Ertmer and Ottenbreit-Leftwich 2010; Thomas and Palmer 2014).

In light of these aspects, professional development programs that directly address teachers’ competencies are regarded as important in supporting the development of knowledge, beliefs and practices (Hegedus et al. 2017; Clark-Wilson et al. 2014; Sztajn et al. 2017). While it has become clear that professional development programs must go beyond basic instruction concerned with handling the technology, little is known about the efficacy of technology-related mathematics professional development programs (Driskell et al. 2015; Grugeon et al. 2010; Hegedus et al. 2017). Past research on professional development programs for teaching mathematics with technology was often either qualitative or focused on how teachers’ practices evolve while taking part in professional development programs (e.g. Clark-Wilson and Hoyles 2019). While this research is very valuable in order to highlight individual learning trajectories of teachers, identify design principles and advance the knowledge of certain professional development activities, the question of causal effects remains blurred. Causality is usually established through experimental or quasi-experimental designs using control or comparison groups, but this type of research is still rare in this field (Sztajn et al. 2017). In fact, with respect to teaching mathematics with technology, the study by Jiang et al. (2013) seems to be one of the rare exceptions that uses an experimental design in order to establish effectiveness of a professional development program. While the study by Jiang et al. (2013) focused on dynamic geometry and proved effects of professional development on teachers’ mathematical content knowledge and teaching strategies, our study addresses a professional development program related to multi-representational tools and takes teacher beliefs and frequency of technology use as outcome measures. The professional development program at hand was designed and carried out by the German Center for Mathematics Teacher Education (DZLM) according to state-of-the-art design principles identified in the literature (Goldsmith et al. 2014). We pursued the goals of investigating whether the professional development program had an effect on teacher beliefs and frequency of technology use, and of deriving recommendations for the design of professional development programs related to teaching mathematics with technology.

2 Theoretical background

2.1 Teaching and learning with multi-representational tools

Multi-representational tools (also called “mathematics analysis software”; Pierce and Stacey 2010) combine the capabilities of scientific calculators, function plotters, spreadsheets, statistic and geometry packages and computer-algebra systems. Nowadays, these tools are most widespread in the form of handhelds due to their permanent availability, personal ownership and the easy integration into assessment. The potentials of these tools are multifaceted, including, for example, the support of multiple representations and more student-centered learning (Ball et al. 2018; Hegedus et al. 2017; Pierce and Stacey 2010). For example, technology provides easy access to different forms of representation, which supports the learning of mathematical concepts by transforming, linking, and carrying out translations between representations. It is also emphasized that technology can support more constructivist approaches, allowing students to explore, discover, and develop mathematical concepts by investigating regularity and variation (Barzel and Möller 2001; Pierce and Stacey 2010).

2.2 Teacher beliefs and belief change

Research has consistently shown that teaching with technology is not a straightforward task, but requires the teacher to guide a subtle development of students’ mathematical understanding and tool techniques (e.g. Drijvers 2019). In light of this, the framework of “pedagogical technology knowledge” (PTK; Thomas and Palmer 2014) highlights that teacher knowledge and beliefs are key competencies for teaching with technology. In particular, Thomas and Palmer (2014) argued that besides knowledge, more attention has to be given to beliefs since they are especially proximal to practice as they shape the way in which we see the world and direct us how to act in a given situation. In this sense, beliefs act as filters and frames, provide an orienting and guiding function, and serve as a connection between knowledge and action (Fives and Buehl 2012). For teaching with technology, there are three facets of teacher beliefs that have been shown to be of high importance (e.g., Ertmer and Ottenbreit-Leftwich 2010): (i) beliefs about teaching with technology, (ii) self-efficacy beliefs and (iii) epistemological beliefs. We now describe these in more detail.

(i) Beliefs about teaching mathematics with technology

Teachers’ commitment to technology use can be related to “recognizing the educational value and believing in the transformative potential of the technology” (Hennessy et al. 2005, p. 185; Pierce and Ball 2009). For example, Goos and Bennison (2008) showed that teachers with more positive beliefs towards teaching mathematics with technology use technology more frequently. Positive beliefs comprise, for example, the beliefs that technology supports deeper understanding of mathematics or aids in changing between different forms of representations (Pierce and Ball 2009). However, it is stressed that beliefs about detrimental effects of technology use have to be taken into account as well (Pierce and Ball 2009; Thomas and Palmer 2014). These beliefs include, for example, that the use of technology is too time-consuming, that technology contributes to a loss of by-hand skills, that pupils become dependent on calculators, and that technology use impedes understanding (Pierce and Ball 2009).

(ii) Self-efficacy beliefsFootnote 1

Self-efficacy beliefs relate to a person’s belief in one’s capabilities (Bandura 1997), which has been shown to affect motivation, decision-making, instructional behavior and persistence (Bandura 1997; Tschannen-Moran and Hoy 2001). In particular, domain-specific self-efficacy for teaching mathematics with technology can be linked to a more frequent use of technology and to stronger positive beliefs about teaching mathematics with technology (Thurm and Barzel 2019; Thomas and Palmer 2014). Clark-Wilson and Hoyles (2019) identify self-efficacy in interacting with technology in front of the class as a likely reason for the gap between planned and real enactment of technology-based lessons. Ertmer and Ottenbreit-Leftwich (2010, p. 261) came to the conclusion, that “self-efficacy beliefs may be more important than skills and knowledge” in teaching with technology, which might be particularly true because the use of technology confronts teachers with more destabilizing situations and questions teachers’ authority and their role as sole source of knowledge (Dunham and Dick 1994; Thurm and Barzel 2019).

(iii) Epistemological beliefs

Epistemological beliefs capture beliefs concerning the nature of knowledge and learning. Beliefs about the nature of mathematics can be differentiated into dynamic and static perspectives, in which mathematics is seen, respectively, as a process of inquiry or as a set of rules and procedures (Blömeke et al. 2008). Beliefs about learning are often differentiated into constructivist and instructivist views (Blömeke et al. 2008). While one has to be cautious with respect to oversimplifying complex relationships, research indicates that teachers with stronger constructivist beliefs and a dynamic view of mathematics tend to be more active technology users, implement technology in a more student-centered way, and view technology not only as a computational, but rather as an instructional tool (Ertmer and Ottenbreit-Leftwich 2010; Tharp et al. 1997). Furthermore, epistemological beliefs are also important because beliefs about technology are often grounded in and derived from epistemological beliefs (Dewey et al. 2009).

Belief development

The aforementioned three facets of teacher beliefs are closely connected with each other. Therefore, none of them can be addressed solely if one aims at supporting teachers in changing their beliefs (Tharp et al. 1997; Thomas and Palmer 2014).Footnote 2 However, supporting teachers in changing their beliefs is not an easy task. As noted earlier, beliefs act as filters and influence how a person interprets new information and experiences. Teachers filter new information gained in professional development programs through their existing belief systems, and tend to turn conflicting evidence into support for their existing beliefs (Ertmer and Ottenbreit-Leftwich 2010; Fives and Buehl 2012). Because of this complexity, it is necessary to give teachers the opportunity to reflect on their beliefs and practice, to witness different teaching approaches, to share implementation experiences and to take an evolutionary approach in which beliefs and practice develop in tandem (Philipp 2007; Ertmer and Ottenbreit-Leftwich 2010). Furthermore, personal successful experience (mastery experience) is regarded as particularly important for strengthening self-efficacy beliefs (Bandura 1997; Ertmer and Ottenbreit-Leftwich 2010).

2.3 Professional development for teaching with technology

While teacher beliefs and knowledge have been identified as important factors for teaching mathematics with technology, little is known about whether professional development programs are effective in developing these competencies. In general, there have been only a few studies investigating professional development programs for teaching mathematics with technology. For example, Driskell et al. (2015) analyzed 1210 mathematics educational technology papers spanning the years 1968–2009 and found only 21 studies addressing professional development. In addition, they highlighted that measures of effectiveness were often omitted and descriptions of the professional development programs were missing. The ICME topical survey of Hegedus et al. (2017) concluded that only a few studies include an evaluation of the professional development program for teaching mathematics with technology. These observations are in line with the review by Sztajn et al. (2017), who pointed out that mathematics education research in general rarely investigates causal links between participation in mathematics professional development programs and teacher or student outcomes. They concluded that “there is a need for more effectiveness studies […], particularly if mathematics professional development researchers want to participate in policy discussions about the professional development offered to mathematics teachers” (Sztajn et al. 2017, p. 816).

The existing studies that focus on the effectiveness of professional development programs for teaching mathematics with technology often report a disappointment with the outcomes (Hegedus et al. 2017). Drijvers et al. (2018) noted that this is particularly disconcerting since in most cases the researchers themselves are strongly involved in designing and providing the professional development programs. However, the study of Jiang et al. (2013), which is exceptional due to its inclusion of a control group, reports positive results on teaching strategies and even student outcomes. Studies accompanying the “Cornerstone Maths Project” also highlight positive development of the quality of lesson plans (Clark-Wilson and Hoyles 2018), but these studies do not provide causal links to the professional development sessions in an experimental design. With respect to studies taking teacher beliefs as outcome measure into account, the studies of Tharp et al. (1997) and Schmidt (1999) show that changes in beliefs are not easy to accomplish. Schmidt (1999) investigated a sample of n = 31 in-service teachers and did not identify any belief changes. The author noted that epistemological beliefs constrained a change of beliefs about teaching with technology. The professional development program investigated by Tharp et al. (1997) placed a high value on reflection concerning teachers’ beliefs and their instructional practice. In this study, positive effects on technology-related beliefs were reported; however, epistemological beliefs did not change over time and rule-based teachers did not benefit from the professional development program.

In total, little is known about causal effects of professional development programs for teaching mathematics with technology on teacher knowledge and belief. In particular, there is a lack of experimental and quasi-experimental studies. Thus, despite some advances, the statement of Grugeon et al. (2010, p. 329), who noted that “research in the field of teacher development courses in mathematics and technology is still in its infancy”, still holds true in many respects.

3 Design of the professional development program

An often-voiced criticism of professional development research is that publications only insufficiently describe the design and content of the professional development programs at hand (Driskell et al. 2015; Sztajn et al. 2017; Goldsmith et al. 2014). Therefore, this section gives a more detailed description of the professional development program researched in this study. The design of the professional development program was, on the one hand, driven by features identified in recent years that characterize high-quality professional development programs (Goldsmith et al. 2014). The DZLM has summarized these key characteristics in six design principles which are compulsory guidelines for designing and implementing its professional development courses. These principles comprise competence orientation, participant orientation, stimulating cooperation, case-relatedness, the use of various instructional formats, and fostering reflection (Barzel and Biehler 2019). On the other hand, we drew on the existing literature for teaching mathematics with technology (e.g., Clark-Wilson et al. 2014). The program addressed the first year of upper secondary school in Germany and was also meant for teachers who are novices in teaching mathematics with technology. In this school year, the content focus is mainly on basic features of functions, derivatives and basic probability theory. In the following, we outline the basic content and course of events of each module. Additional information regarding the design and content of the professional development modules can be found in publications by Thurm et al. (2015) and Thurm (2019).

The professional development program consisted of four one-day face-to-face modules and alternated input, try-out and reflection phases. Each module was held as a 6-h face-to-face meeting; therefore, the overall face-to-face time amounted to approximately 24 h. Teachers were provided with access to a web-based platform where they could access all materials from the face-to-face meetings. Between modules, the teachers had approximately two months of time in which they were expected to try out and implement the approaches they learned in the face-to-face modules. During the face-to-face modules, representations of practice (i.e. students’ solutions) were used and participants were urged to bring their own case-examples from their classrooms in order to facilitate further case-based learning. All modules were developed in cooperation between a group of researchers from the University of Duisburg-Essen and the University of Münster. In addition, practitioners and facilitators who had a long history of teaching with technology and providing professional development courses were part of the team. The team included the authors of this paper. While the first author was exclusively responsible for the research study, the second author was also involved in carrying out the professional development program.

Module 1: Introduction to teaching with multi-representational tools

At first, the didactical background of using technology (see Sect. 2.1) was provided by highlighting how technology supports students’ concept images of functions, for example by using multiple linked representations. Subsequently, the module focused on appropriate tasks for teaching with technology, as tasks are one of the key elements driving technology integration (Leung and Baccaglini-Frank 2016). At first, teacher activities centered around the “power flower” task (Barzel and Möller 2001), in which students are asked to draw a given picture of graphs of power functions by using a function plotter. For this task, students have to identify how the functions connect. Teachers solved this rich task and reflected on the potential and risks of technology use with this task. Subsequently, teachers were introduced to other types of tasks, chose a task according to their individual interests and skill levels, solved the task themselves, reflected on the potentials and pitfalls of using technology with this task and discussed similarities and differences between the tasks. Finally, teachers planned to fit one of the tasks into one of their upcoming lessons.

Module 2: Modelling tasks with multi-representational tools

After reflecting the try-out phase and discussing the representations of practices that were brought by the teachers to the face-to-face meeting, the aspect of task design and selection was deepened with respect to mathematical modelling. It was stressed that technology is particularly powerful “in the conceptualization of the model rather than simply as a tool which is used to solve a mathematical problem after it has been abstracted” (Geiger et al. 2010, p. 64). Teachers were asked to solve a modelling task predicting oil production based on historical data, and reflected on the use of technology in different phases of the modelling cycle (Greefrath 2011). In the second part of the day, the focus was on modelling stochastic phenomena. Teachers solved and discussed a task in which technology can be used for modelling situations pertaining to the Galton Board. In particular, teachers reflected on potentials and pitfalls of technology use in this task. Afterwards, a set of modelling tasks tuned to the curriculum of the first year of upper secondary school was provided for teachers as a basis for planning their own try-out phase.

Module 3: Classroom organization

After reflecting on the try-out phase, Module 3 focused on classroom design and implementation. Since teachers often believe that technology should be used only after the mathematics is understood without technology (Pierce and Ball 2009), we discussed the black-box/white-box approach and the white-box/black-box approach (Zbiek et al. 2007). Teachers compared two hypothetical lesson plans that utilize the power flower from the first module. Each respective lesson plan followed one of the approaches. In the first lesson, the task was meant to introduce new content while the other lesson was dedicated to practicing skills. Teachers discussed which version they preferred for their teaching and reflected on advantages and drawbacks of each approach.

The second part of the module focused on the different teaching phases. We distinguished technology use for learning mathematics and for doing mathematics (Drijvers et al. 2011), as well as technology use during the different phases of introduction, deepening, and practicing of mathematical content. It was discussed which approaches work for which end and in which phases. In particular, it was stressed that in the practicing phases, technology should be used not only for developing skills, but that technology can support rich tasks for deepening concepts, procedural knowledge, and skills at the same time. Afterwards, teachers solved and discussed sample tasks for each purpose and developed their own tasks for the upcoming try-out phase.

Module 4: Assessment and Documentation

After a reflection on the try-out phase, the last module focused on how examinations and documentation are affected by the introduction of technology. Teachers used the classification scheme of Brown (2003) to analyze sample final examination tasks, discussed criteria for technology-supported examination questions, and modified questions to fit these criteria. Afterwards, the module focused on the issues of language and documentation when teaching mathematics with technology, as students tend to use shorter, tool-oriented language when working with technology (Ball and Stacey 2003). Based on students’ sample solutions, teachers discussed which language and documentation are expected and helpful in examinations and in learning situations. The module finished with a reflection on the whole professional development program.

4 Research questions and hypotheses

The objective of the present study was to scrutinize to what extent the professional development program described in the last section had an effect on participants’ beliefs and the frequency of technology use. In particular, we pursued the following research questions (RQ):

  • RQ1: Do teachers who participate in the professional development program develop more positive beliefs about (i) teaching with technology than teachers who did not participate?

  • RQ2: Do teachers who participate in the professional development program develop stronger (ii) self-efficacy beliefs regarding teaching with technology than teachers who did not participate?

  • RQ3: Do teachers who participate in the professional development program develop more supportive (iii) epistemological beliefs (i.e. more constructivist and dynamic beliefs) than teachers who did not participate?

  • RQ4: Do teachers who participate in the professional development program increase (iv) frequency of technology use more than teachers who did not participate?

In the following, we outline the hypotheses according to each research question.

  • H1: For RQ1, we hypothesized that professional development participants develop more positive beliefs about teaching with technology than the control group. In all modules, teachers were exposed to activities that highlighted the potentials of technology and intensively reflected on how technology can support student learning. In addition, we expected teachers to witness the benefits of technology use during the try-out phases, which should also contribute to belief change.

  • H2: For RQ2, we hypothesized that professional development participants develop stronger self-efficacy beliefs than the control group. During the face-to-face modules, teachers had multiple opportunities to experience the use of technology, solve and design tasks and discuss and plan technology supported lessons. In addition, we expected that the try-out phases would provide mastery experience, where teachers experience that they are capable of integrating technology into the classroom.

  • H3: For RQ3, we hypothesized that professional development participants develop stronger constructivist and dynamic beliefs than the control group. Even though constructivism was not explicitly the focus of the professional development program, the design was clearly grounded in constructivist beliefs and a dynamic view of the nature of mathematics. However, epistemological beliefs are deeply ingrained in the beliefs system of a teacher, which could obstruct large effects (Tharp et al. 1997).

  • H4: For RQ4, we hypothesized that professional development participants manifest more frequent use of technology than the control group. On the one hand, the development of more positive beliefs, stronger self-efficacy, stronger constructivist and dynamic beliefs, as well as additional knowledge gained in the professional development program, should support technology uptake. On the other hand, technology integration takes time, as it often takes place in small steps in which teachers rethink classroom routines and adapt to new roles (Ertmer and Ottenbreit-Leftwich 2010), which might impede large effects on frequency of technology use.

5 Method

5.1 Design

The professional development program was carried out in the federal state of North Rhine-Westphalia at the beginning of the school year 2014/2015. In this state, the use of multi-representational tools in upper secondary school became compulsory in that school year and therefore, most teachers in this state had not used these tools in their teaching before. Teachers volunteered to sign up for the professional development program. The program was carried out at three sites throughout the state. We compared data from an experimental group of teachers taking part in the professional development program with data from teachers not participating in the professional development program. After the first professional development module paper-based pre-test, questionnaires covering (i) beliefs about teaching with technology, (ii) self-efficacy beliefs, (iii) epistemological beliefs, and (iv) frequency of technology use were distributed to the participants. The post-test questionnaire was distributed at the end of the last professional development module. N=39 teachers completed the pre- and post-tests. The control group consisted of n=88 teachers who were enlisted by means of a circular letter to all upper secondary schools in the German federal state of North Rhine-Westphalia. Paper-based questionnaires were provided to the control group at the same time as those for the experimental group.

5.2 Instruments

To measure teachers’ beliefs and frequency of technology use, we used quantitative questionnaires as described in this section.


(i) Beliefs about teaching with technology

To measure technology-related beliefs, we used a questionnaire that covered different aspects of technology integration. The items were refined in cycles of interviews with teachers and experts and were validated with a total sample of n = 199 in-service teachers (Thurm 2017). Sample items and reliability are reported in Table 1 (all items can be found in Thurm 2017). Responses were given from 1 = “strongly disagree” to 5 = “strongly agree”.

Table 1 Scales for measuring (i) beliefs about teaching with technology (multi-representational tools (MRT))
Full size table

(ii) Self - efficacy beliefs regarding teaching with technology

For measuring self-efficacy, we followed Banduras “guide for constructing self-efficacy scales” (Bandura 2006), which points out that self-efficacy scales should be tailored to the domain of functioning of interest. We identified task design and task selection (Leung and Baccaglini-Frank 2016) as well as lesson design and implementation (Drijvers 2019) as important domains. Items were generated based on the recommendations of Bandura (2006) and previous research (e.g., Thomas and Palmer 2014) and were refined in cycles of interviews with teachers and experts. Sample items and reliability are reported in Table 2. Following Bandura (2006), teachers rated the strength of their belief in their ability on a scale from 0–100.Footnote 3

Table 2 Scales for measuring (ii) self-efficacy beliefs
Full size table

(iii) Epistemological beliefs

To measure epistemological beliefs, we used the questionnaires from the international TEDS-M study (Blömeke et al. 2008). The scales distinguish between a static perspective (E1, mathematics as rules and procedures) and a dynamic perspective (E2, mathematics as process of inquiry) and between learning by teacher direction (E3) and by active learning (E4). A sample item from the scale capturing the static perspective on mathematics was as follows: “Mathematics is a collection of rules and procedures that prescribe how to solve a problem.” Response format was a 6-point Likert scale ranging from “1 = strongly disagree” to “6 = strongly agree”. Reliability of the scales ranged from 0.6–0.76.


(iv) Frequency of technology use

Frequency of technology use was captured using the Likert scale questionnaire described by Thurm (2018). Teachers reported their frequency of technology use in the following areas: (F1) use of technology for discovery learning, (F2) use of technology for linking multiple representations, (F3) use of technology when practicing, (F4) use of technology to support individual learning. In addition, category (F5) captured how often the use of technology was subject to critical reflection. Response categories for scales (F2), (F4) and (F5) were “almost never”, “once or twice a quarter”, “once or twice a month”, “once a week”, “almost every lesson”. Scales (F1) and (F3) were rated with respect to the proportion of technology use on a 5-point Likert scale. Reliability of the scales ranged from 0.78–0.89.


Teachers’ appraisal of the professional development program

Satisfaction with the professional development program is regarded as a necessary prerequisite to effects on teacher competencies, as satisfaction with activities, materials and instructors renders change more likely (Guskey 2000). Therefore, we evaluated the reactions of the participants to the professional development program with regard to content, structure, and activities. For example, teachers rated how the program supported their pedagogical content knowledge and how much the professional development materials (handouts, worksheets, examples) were relevant for their teaching at school.

5.3 Propensity score-matching and data analysis

To reduce the risk of selection bias, which occurs when differences between experimental and control group influence outcome measures, we used the method of propensity score matching (Austin 2011), which is nowadays widely used to estimate average treatment effects in quasi-experimental studies. The propensity score method uses a one-dimensional score, namely the probability of receiving treatment given the observed variables, to match experimental and control group units. It can be shown that under certain conditions such as “no unmeasured confounders”, matching on the propensity score mimics a randomized control trial and gives unbiased treatment effects.

In our study, we used nearest-neighbor matching to match each treatment unit to two comparison units (matching with replacement) whose propensity scores were most similar to the treated unit in question. The variables that were included in the matching process were age, gender, prior years of experience with teaching mathematics with technology, self-efficacy beliefs, and beliefs about teaching with technology. In total, n = 33 different control group units were matched to the n = 39 experimental group teachers. With regard to matching with replacement, the control group units were assigned weights, the sum of weights equaling n = 38 treated units. After matching took place, the unmatched comparison units were discarded. After matching, there were no significant differences on baseline characteristics between experimental and matched control group (see Tables 3 and 4). To obtain measures of efficacy, we used the matched sample to conduct Analysis of Variance on the difference between pre-test and post-test scores and calculated the effect size for each outcome measure. Since effect sizes for professional development programs average around d = 0.6 for effects on students learning (Hattie 2009) and effects on teacher learning are naturally higher, we expected effects on teachers’ beliefs and practice to be above d = 0.65. The required sample size for an effect of d = 0.65 equals n = 38 if alpha level is set at the 0.05 level and power is set to the common choice of 80% (for an effect of d = 0.7 a sample size of n = 33 would be required).

Table 3 Results of the analysis of variance on the difference between pre-test and post-test scores regarding (i) beliefs about teaching with technology (*p < 0.05; **p < 0.01; ***p < 0.001)
Full size table
Table 4 Results of the analysis of variance on the difference between pre-test and post-test scores regarding teachers (ii) self-efficacy beliefs
Full size table

6 Results

6.1 Baseline characteristics for the matched sample

Most of the teachers in the experimental and control group were novices with regard to teaching mathematics with technology, which is reflected in an average teaching experience of 1.05 years in the experimental group and 1.18 in the matched control group. In particular, 28 teachers (72%) in the experimental group had no previous experience in teaching with technology at all. Average age in the sample was of the order of the average age of upper secondary school teachers in North Rhine-Westphalia (45 years). With regard to (i) teachers’ beliefs about teaching with technology, teachers held positive beliefs about the potential of technology to support multiple representations and discovery learning. However, at the same time, teachers were also wary about the possible loss of by-hand skills, and considered the idea that teaching with technology leads to mindless working. Thus, teachers were neither overly enthusiastic nor hostile towards technology use. Teachers’ (ii) self-efficacy beliefs were quite low. The scales capturing (iii) epistemological beliefs show that teachers strongly favored active learning and held a dynamic view about mathematics. Finally, (iv) frequency of technology use was rather low, in particular, technology was only rarely used to link multiple representations.

6.2 Effects of the professional development program

As satisfaction with the professional development program is a prerequisite for effects, it is important to note that teachers were overall satisfied with content, structure and relevance of the professional development program. Teacher rated these aspects between 4.19 and 4.78 on a six-point Likert scale, where higher values correspond with higher satisfaction.

6.2.1 Effects on beliefs about teaching with technology

Beliefs regarding teaching with technology developed more favorably in the experimental group (see Table 3). For scales (T1) (discovery learning), (T2) (multiple representation), (T3) (time consuming) and (T6) (procedures must be mastered first), the control group teachers developed technology-hostile beliefs whereas those of the experimental group remained unchanged. Effect sizes for these scales ranged from d = 0.24 (T1) to d=0.45 (T2). With regard to scale (T5) (technology leads to mindless working), the experimental group developed more technology-friendly beliefs, whereas those of the control group teachers remained almost unchanged (d = 0.41, p = 0.006). With respect to (T4) (loss of by-hand skills), both groups were still strongly convinced that technology will lead to erosion of by-hand skills, even though there was a slight decrease in both groups. In total, our first hypothesis that beliefs towards teaching with technology will develop favorably was supported by the data, except concerning the loss of by-hand skills.

6.2.2 Effects on self-efficacy beliefs

With respect to self-efficacy beliefs, those of experimental and control group teachers both increased significantly from pre- to post-test (Table 4). It can be seen that self-efficacy increased from a low level in the pre-test to moderate levels in the post-test in both groups. Even though the gains in the experimental group were higher for both scales, which may indicate a positive effect of the professional development program, the gains were not sufficiently larger than in the control group to yield a significant effect of the professional development program. Hence, our second hypothesis (“professional development will lead to stronger self-efficacy beliefs”) is not supported by the data.

6.2.3 Effects on epistemological beliefs

With regard to teachers’ epistemological beliefs (Table 5), there were no substantial changes in beliefs about the nature of mathematics or beliefs about the teaching and learning of mathematics. Teachers were still strongly privileging constructivist beliefs (scale E4) and seeing mathematics as a process of inquiry (scale E2). Seeing mathematics as a set of rules and procedures (scale E1) remained on a moderate level. However, it has to be noted that teachers in our study already held strong dynamic and constructivist beliefs, which rendered further changes less likely. Hence, our third hypothesis (“professional development will support more constructivist beliefs”) is not supported by the data.

Table 5 Results of the Analysis of Variance on the difference between pre-test and post-test scores regarding (iii) teachers’ epistemological beliefs
Full size table

6.2.4 Effects on frequency of technology use

Finally, when looking at frequency of technology use (Table 6), no substantial changes were observed for most scales. However, frequency of using technology to support multiple representations (F2) and frequency of reflecting technology use (F5) picked up more in the experimental group compared to the control group (F2: d = 0.34, p = 0.13; F5: d = 0.29, p = 0.12). Thus, our fourth hypothesis (“increased use of technology”) is partly supported by the data.

Table 6 Results of the analysis of variance on the difference between pre-test and post-test scores regarding (iv) frequency of technology use
Full size table

7 Discussion and conclusions

We have examined a half-year professional development program for teaching mathematics with multi-representational tools that was based on design principles regarded as crucial in the literature. Participants in the study were mostly novice teachers with regard to teaching with technology. The research questions scrutinized effects of the professional development program on teachers’ beliefs about teaching with technology, self-efficacy beliefs, epistemological beliefs and frequency of technology use.

Of course, the results of this study are subject to limitations in the study design. Unobserved confounders may still have biased the results, classroom implementation was not triangulated with classroom observation, and not all participants of the professional development program participated in our study. In particular, the incorporation of interviews and lesson reports may have provided further insights about how other variables such as school culture and organizational support might have influenced the effects of the professional development program, and which activities during the face-to-face meetings were perceived as particularly supportive by the participants for their classroom implementation. In addition, we propose to extend this study to different contexts, to longer periods of time and by including measures of student achievement. While being aware of the aforementioned limitations of the study, this research contributes by adding quasi-experimental evidence to the underresearched field of effectiveness of professional development programs for teaching mathematics with technology, which will now be discussed in detail.


Importance of supporting novice-teachers using technology

The most significant effect was found on (i) beliefs about teaching with technology. Remarkably, in most cases, the effect did not result from the development of more technology-friendly beliefs in the experimental group, but from control group teachers developing more technology-hostile beliefs. Since beliefs are largely derived from classroom experiences (Guskey 2000), this development in the control group might be attributed to the so-called “implementation dip” (Fullan et al. 2005), which describes how teachers often experience problems during the early stages of implementation. Ideally, these problems are seen as part of the natural process of professionalization, but control group teachers seemed to attribute these problems to technology itself. Hence, our study draws attention to the necessity of supporting teachers at early stages of adoption, since technology-hostile beliefs are rarely sufficiently revised and are likely to become deeply ingrained and highly resistant to change (Ertmer and Ottenbreit-Leftwich 2010). It appears that the professional development program either reduced the experience of a severe implementation dip or helped to frame their negative experience as a natural part of the implementation process. In addition, our study shows that particular attention has to be given to teachers’ beliefs about the loss of by-hand skills, which remained highly pronounced in both experimental and control group. This belief seems particularly resistant to change, possibly due to its close link to the fundamental belief of many teachers that “one must be able to do the mathematics by hand in order to understand and use it” (Dewey et al. 2009, p. 391).


Necessity of explicitly fostering self-efficacy

The second important finding of our study shows the necessity of making self-efficacy a stronger focal point of professional development programs for teaching with technology. While (ii) self-efficacy increased slightly in both groups (which can be attributed to the fact that the rate of improvement is particularly strong at the beginning of implementing new approaches; Bandura 1997), the question arises why teachers taking part in the professional development program did not gain more self-efficacy than control group teachers. This might be explained by the study of Tschannen-Moran and McMaster (2009), who showed that the combination of mastery-experience with extensive follow-up coaching is crucial for strengthening self-efficacy. However, such individual coaching was missing in the professional development program (and many other professional development programs reported in the literature), usually because such coaching is resource intensive and hard to scale up. In light of this, it is worth investigating approaches such as micro-teaching or approximations of practice (Grossman et al. 2009), which might offer less resource-intensive approaches to enabling mastery experience.

These approaches might in turn influence (iv) frequency of technology use and (iii) epistemological beliefs, which were only marginally affected by the professional development program at hand.

Summarizing these results, we can state that professional development programs may be effective with regard to beliefs about teaching with technology. Self-efficacy beliefs and epistemological beliefs as well as measures of actual classroom practice, such as frequency of technology use, seem to require additional effort and time to change.