1 Introduction

Mathematics teachers face challenges in modifying their teaching to incorporate effective pedagogical practices, technology tools, and new curricula resources. They also face challenges in making changes to address updated standards and expectations for mathematics and statistics learning at the middle school, high school, and early college levels. Teachers often have limited resources to support professional development to learn how to make these changes. Many teachers are seeking out online professional development opportunities by participating in massive online open-access courses (MOOCs).

The development and wide reach of MOOCs has opened up the possibilities for teachers to engage in a variety of learning opportunities. This may include teachers enrolling in specific disciplinary content MOOCs intended for a wide audience (Seaton, Coleman, Daries, & Chuang, 2015), but also includes MOOCs specifically targeted for K-12 teachers (e.g., Avineri, Lee, Tran, Lovett, & Gibson, 2018; Borba et al., 2016; Ferdig, Pytash, Merchant, & Nigh, 2014; Lee & Stangl, 2017). Over the past five years, we have designed and offered three Massive Open Online Courses for Educators (MOOC-Eds) specifically developed and advertised for teachers of mathematics and statistics:

  1. 1.

    Teaching Statistics through Data Investigations (Data Investigations),

  2. 2.

    Teaching Statistics and Inferential Reasoning (Inferential Reasoning), and

  3. 3.

    Teaching Mathematics with Technology (Mathematics Technology).

These courses were designed based on principles of effective online professional development and informed by the Interconnected Model of Professional Growth. In this paper, we examine the following questions:

  1. 1.

    How are design principles enacted in the development of the MOOC-Eds?

  2. 2.

    How does the enactment of these principles influence participants’ engagement and provide opportunities for them to develop their knowledge of, beliefs about, and attitudes towards teaching mathematics and statistics?

2 Theoretical framework

Clarke and Hollingsworth (2002) theorized an Interconnected Model of Professional Growth (Fig. 1) that suggests teacher change occurs through the process of enacting and reflecting on practice. This takes place in four different contexts experienced by teachers: the personal domain (knowledge, beliefs, attitudes), the domain of practice (what teachers implement in classrooms), the domain of consequence (outcomes), and the external domain (sources of information).

Fig. 1
figure 1

The interconnected model of professional growth (Clarke and Hollingsworth, 2002, p. 951)

Full size image

A MOOC-Ed exists within the external domain while the knowledge, beliefs, and practices developed while participating are situated within a teacher’s professional practice. The path through these different domains is non-linear and influenced by the change context within which teachers are learning and working.

There have been a number of studies that have investigated features of the external domain and made recommendations about the design of online professional development to support teachers. For the current study we considered how participants in the MOOC-Eds enacted their knowledge, beliefs, and attitudes (personal domain) in their work and in the discussion forum posts as they engaged with and reflected upon elements from the external domain (e.g., frameworks, readings, mathematics and statistics tasks). Thus, it was important for our analysis to operationalize elements from the Interconnected Model of Professional Growth to examine the influence of design principles on the personal domain of MOOC-Ed participants.

3 Research-based design: Review of related literature

Evidence suggests online professional development (PD) that is accessible, meaningful, collaborative, and addresses varied needs and abilities of participants can lead to changes in teachers’ instructional practices (e.g., Luebeck, et al., 2017; Renninger et al., 2011; Vrasidas & Zembylas, 2004). In particular, Herrington et al. (2009) found that teachers succeeded in implementing new pedagogical strategies when they felt supported by an online community.

Findings from Qian et al. (2018) led to three recommendations for designing online PD: (1) use activities that match teachers’ background knowledge and experiences, (2) align activities with curricula, and (3) use motivational design to enhance teachers’ engagement. Furthermore, Powell and Bodur (2019) found six features to be critical for teachers’ online engagement: relevancy, authenticity, usefulness, collaboration and interaction, reflection, and context. They also emphasized that online professional development should be job-embedded. That is, teachers should be able to use materials from a professional development course in their job, and aspects of a teacher’s job (e.g., understanding content they need to teach, planning lessons, making sense of students’ work, implementing tasks and reflecting on learners’ experiences) should be included in professional development activities. In the case of Qian et al., they examined participation of 33 computer science teachers in an online professional development experience and from that made inferences about important features to inform instructional design. Powell and Bodur (2019) studied how six features of an online professional development experience impacted six high school social studies teachers. In both cases, the designers of the professional development were familiar with the teachers. Teachers in Qian et al’s study attended a two-day face-to-face professional development and teachers in Powell and Bodur’s study were from the same school district. We were curious to examine how the implementation of the design principles abstracted from these prior studies would affect the participation of mathematics and statistics teachers’ in a MOOC-Ed that included teachers from across the globe who were unfamiliar with each other and the designers of the online professional development.

Assessment of design principles used to guide the development of MOOCs for teachers was conducted by Aldon, Arzarello, Panero, Robutti, Taranto & Trgalová (2019). They examined how instructors’ practices influenced collaboration and participation in MOOCs implemented in France (eFan Maths MOOC) and Italy (UniTo: Geometria MOOC and Numeri MOOC). The MOOCs from both countries supplemented discussion forums with the use of other collaborative tools (e.g., Padlet, social networks, collaborative project spaces). There were differences noted in how the instructors facilitated collaboration. The instructors in the French MOOCs focused on fostering local collaboration while the Italian MOOCs encouraged collaboration among all participants within the MOOC. This study points to the importance of examining not just the design of a MOOC for teachers, but also how such MOOCs are enacted and experienced by participants.

Over the past seven years, a collection of MOOC-Eds has been created at the Friday Institute for Educational Innovation at NC State University (http://place.fi.ncsu.edu). All courses are developed using research-based design principles of effective professional development and online learning (e.g., Garet et al., 2001, Darling-Hammond et al, 2009) that have been synthesized to focus on: (a) self-directed learning, (b) learning from multiple voices (c) job-connected learning, and (d) peer-supported learning (Kleiman, Wolf, & Frye, 2015). Not only did we design our three courses using research-based practices for professional development and online learning, but we also used best practices specific to mathematics and statistics teacher education. Considering these design features in the context of teaching and learning mathematics and statistics is unique to this study. In particular, the design of the mathematics and statistics MOOC-Eds includes:

  1. (a)

    frameworks that can assist teachers in applying newly learned content and strategies to their own instructional practices and focus on students’ learning (Franke, Carpenter, Levi, Fennema, 2001; Boston & Smith, 2011; Stein & Smith, 1998).

  2. (b)

    opportunities for teachers to engage in mathematics/statistical tasks as learners and reflect on those experiences from the perspective of a teacher (e.g., Conference Board of Mathematical Sciences, 2012; Franklin et al, 2015; Stein & Smith, 1998);

  3. (c)

    samples of student work and video recordings of students and teachers to provide participants opportunities to notice student thinking and learn from practice (e.g., Wilson, Lee, & Hollebrands, 2011; Sherin & Van Es, 2005);

  4. (d)

    opportunities for teachers to share their perspectives and discuss their teaching practices (e.g., Borko, 2004).

As we describe the design of the courses according to the primary MOOC-Ed design principles, we will illustrate course elements that are also informed by professional development recommendations for teachers of mathematics and statistics and synthesize data collected for prior studies to discuss the influence of these features from the external domain of the MOOC-Eds on the personal domain of participants.

4 Methods

4.1 Course contexts

This paper reports on three MOOC-Ed courses. Teaching Statistics through Data Investigations (Data Investigations) was first piloted in Spring 2015 with 797 participants, using a Google Coursebuilder platform. Since Fall 2015, Data Investigations has been offered, with only very minor editorial changes, using a Moodle-based platform. Teaching Mathematics with Technology (Mathematics Technology) launched in Fall 2016 and Teaching Statistics Through Inferential Reasoning (Inferential Reasoning) was first offered in Fall 2017. This paper reports on data from 14 course offerings between September 2015 and May 2019 (7 Data Investigations, 4 Mathematics Technology, 3 Inferential Reasoning).

Courses were typically open in Fall or Spring. Fall courses opened in September and remained open until December. Spring courses opened in January or March and remained open until May. Only one course (Data Investigations) was offered in summer (June–August) in 2016. Each course contained an orientation unit and five core units. Each core unit included resources to read and watch as well as several technology-enhanced activities to complete and discuss. The MOOC-Ed courses were open for new enrollments for approximately 10 weeks. That is participants could begin the course anytime during the first 10 weeks. The courses remained open for approximately 12–15 weeks. For the first 5 weeks of enrollment, a new unit was available for participants, and once opened a unit was always accessible. There were several times when Data Investigations was offered that the entire course opened at one time so that all units were accessible. For all courses, either weekly or bi-weekly announcements were emailed to registered participants to discuss course activity and remind participants to engage in materials. Related to the self-directed principle, participants could decide which units to complete and whether to complete some or all course materials.

Requirements for obtaining a certificate of completion for 20 h of professional development credit typically included accessing resources in the “Essential” material section, completing core activities, and participating in discussion forums, in each unit. They were also required to complete brief quizzes and surveys to earn a certificate of completion. After a course was officially closed, discussion forums became “read-only,” but participants were able to enter the course site and access all material. In this way, the course became a resource to return to if desired.

4.2 Participants

Registration data was used to identify and describe course participants. Across the 14 course offerings, there were 4429 unique enrollees, for a total of 5767 registrations across the courses. This means that 1338 educators enrolled in more than one offering of the three courses. All states in the US (n = 3842) and 106 countries were represented in the enrollment, with New Zealand (n = 117), Canada (n = 78), India (n = 74), Australia (n = 67), and the United Kingdom (n = 56) having the highest non-US enrollments.

Of those who enrolled in the 14 offerings, over half (61.9%) reported they were classroom teachers, 70.9% were female, and 72.8% had advanced degrees (masters or doctoral degrees). Participants were asked to report their primary reason for enrolling in a MOOC-Ed (not required question), across all 14 offerings, of those that responded (n = 2794), the two most common reasons were a desire to deepen their understanding of course content (44.8%) and an interest in collecting resources and tools to support their practice (35.1%). Several participants reported they planned to collaborate with peers while enrolled in a MOOC-Ed. A summary of participants by course is included in Table 1.

Table 1 Description of MOOC-Ed participants by course
Full size table

Across the three courses we note similarities in that more than half of the participants are female, have Master’s degrees, and most consider their primary role as a classroom teacher. The distribution of participants’ experience in education was similar across the courses. However, in the two Teaching Statistics MOOC-Eds a much larger percentage of participants have doctoral degrees (19.3% and 24.1%) compared to 7.2% in the Mathematics Technology MOOC-Ed. This is reflected in the number of participants who enrolled in Data Investigations and Inferential Reasoning who teach statistics at the college and university levels (35.8% and 43.3%). It is also interesting to note the high percentage of participants (51.7%) in the Mathematics Technology MOOC-Ed who indicated plans to collaborate with peers while completing the course in Fall 2016.

4.3 Data sources and analysis

Using the Interconnected Model of Professional Growth, we considered how elements from the external domain of the MOOC-Ed course influenced the knowledge, beliefs, and attitudes of participants’ personal domain. Elements of the external domain included resources and activities. Readings, videos, frameworks, and tasks were considered resources, while discussion forum posts and participants’ work on the tasks were considered activities. Participants’ knowledge, beliefs and attitudes considered through the processes of enactment and reflection on the resources were assessed by examining registration questions, click logs, discussion forum posts, end-of-course surveys, follow-up surveys sent 6–12 months after a course, and post-course interviews with a few active participants from each course. These data were collected over a number of years as part of project evaluations and research (e.g., Hollebrands et al., 2018; Lee et al., 2017). The alignment of the data to the research questions examined in this paper are presented in Fig. 2.

Fig. 2
figure 2

Alignment of research questions with data collections and methods of analysis

Full size image

Content analysis was conducted to respond to the first research question. Data for this question were all resources included across all three MOOC-Eds. A resource was the unit of analysis. These resources were coded using the four design principles (e.g., self directed learning, learning from multiple voices, job-connected learning, and peer-supported learning). Researchers coded and discussed the resources and arbitrated codes about which there was disagreement.

To respond to the second research question, course activity was tracked through click logs that allowed us to examine trends in participants’ engagement with materials. These click logs were uploaded to Tableau. We were able to visualize all participants’ engagement over time through a dashboard in Tableau, that provided both descriptive statistics and visual representations. Our analysis included identifying how many enrollees participated across each unit and in discussion forums. We also examined trends in the number of resources accessed by a participant and the number of days they had logged into a course. The qualitative data from coding of discussion forum posts, end-of-course surveys, follow-up surveys, and interviews were used to assess how the enactment of the design principles in the external domain provided opportunities for participants’ to develop their knowledge of, beliefs about, and attitudes (personal domain) towards the teaching and learning of mathematics and statistics. A forum post was the unit of analysis and these were coded using an open coding process guided by research questions related to participants’ knowledge, beliefs, and attitudes. These codes were analyzed to identify themes. Surveys, interviews, and registration data were analyzed to gather additional evidence to support or refute the identified themes. Detailed descriptions of the analytical techniques have been reported elsewhere (Hollebrands et al., 2018; Lee et al, 2017).

5 How design principles are enacted and the learning opportunities they supported

The design of the three MOOC-Eds were guided by four design principles (a) self-directed learning, (b) learning from multiple voices (c) job-connected learning, and (d) peer-supported learning (Kleiman, Wolf, & Frye, 2015). This section is divided into four parts to address each of the design principles. Elements from the MOOC-Eds (external domain) that were designed using each of these principles will be illustrated (research question 1) and then this is followed by a description and analysis of data that highlights how the enactment of the design principles provided opportunities to influence the personal domain (knowledge, beliefs, and attitudes) of the participants (research question 2).

5.1 Self-directed learning

Several aspects of the courses support self-directed learning. These aspects include choices in when participants can access the course and what activities within the course they complete. Others have found that educators appreciate this flexibility and often learn about new ideas and strategies that impact their classrooms even when they only engage with a small number of resources in a MOOC (e.g., Ferdig, et al., 2014; Jacobsen, 2019).

5.1.1 Enactment of the self-directed learning design principle

Participants in our MOOC-Eds are able to register and start a course anytime during the timeframe it is open. The courses are asynchronous, which allows educators opportunities to log in when they have time and provides flexibility for when they complete any part of the course. Once a unit is open, it remains open, and participants can traverse to any material in the unit that is of interest. Many of the videos produced for the course include time-stamped bookmarks for easy navigation to particular topics in a video, and transcripts and podcasts are available for participants to download and read or listen to on a mobile device. Although requirements for a certificate of completion requires completing certain elements (e.g., posting in discussion forums), if participants are not interested in a certificate, they can select to view any resource and can do so at their own pace. In anticipation that many educators would only have the time to engage with a few resources and activities, several critical experiences related to the content and focus in each MOOC-Ed were included in Units 1–2 of the courses. We hypothesized that several of these course resources or activities had the potential to influence perspectives on important aspects of teaching mathematics and statistics.

In our MOOC-Eds, a variety of materials are provided to appeal to participants who teach at different grade levels (middle, high school, or college) or participants who teach different mathematics topics (algebra, geometry, statistics). Teachers can strengthen their content understanding and develop pedagogical strategies by engaging as learners in mathematics and statistics tasks. When teachers experience a task as a learner, it can provide an opportunity for them to develop a different perspective about what it means to learn mathematics and statistics meaningfully (Conference Board of Mathematical Sciences, 2012; Stein & Smith, 1998). Each unit also includes extra resources in an “Extend Your Learning” section that allows participants to explore topics, strategies, or tools further. Not only can participants choose to select topics and materials most relevant to their needs and interests, but they can return to the materials even after the course has ended.

In addition to allowing participants the option to choose which resources they wish to examine, there are also opportunities for participants to check their understanding and demonstrate what they have learned. All of the MOOC-Eds include short quizzes or Likert-scale surveys that can help participants assess their learning. For example, in the Data Investigations course, participants are given the option to take the Self Efficacy for Teaching Statistics survey (Harrell-Williams et al., 2019) in the Orientation unit and again in the final unit (Unit 5). This survey helps them set goals at the beginning of the course, and assess and reflect on changes in confidence to teach statistics at the end of the course.

5.1.2 Learning opportunities supported by the self-directed learning design principle

The self-directed learning design principle allowed participants to visit as few or as many resources as desired when the course opened and beyond its closing. Of the 4429 unique participants who enrolled in the 14 offerings of these three MOOC-Eds, 2963 accessed a course (66.9%), 2815 viewed at least one resource (63.6%), 1545 posted to a forum (39.2%), and 501 (11.3%) received at least one certificate of completion. Certificates are awarded for 20 h of work within a MOOC-Ed course, which often counts as professional development credit for K-12 teachers in the US. The low number of certificates is not surprising with so many collegiate-level teachers in two of the courses where tracking of professional development hours is typically not required. In addition to examining who earned a certificate of completion, one can also examine which units participants engaged with and which resources participants viewed within a MOOC-Ed.

Similar to findings about trends in participation and typical drop-off rates in MOOCs (Eriksson, Adawi, & Stöhr, 2017; Onah, Sinclair, & Boyatt, 2014), not all participants who registered for the Data Investigations, Mathematics Technology, and Inferential Reasoning MOOC-Eds completed all five units (Fig. 8). We consider the drop-off between the Orientation Unit and Unit 1 to indicate those registrants who realized the course was not of interest to them, or those who determined they did not have the time to engage further. What we observe, however, is those who completed the third unit tended to finish a MOOC-Ed. Across all offerings of the three MOOC-Eds, about 37.9% of participants that engaged in Unit 1 stayed with the course and completed through Unit 5 (n = 964). This completion rate is much higher than the typical completion rate of 5–20% of learners who show up in a beginning unit in most MOOCs (Perna, et al., 2014) (Fig. 3).

Fig. 3
figure 3

Participation in 14 offerings of the three MOOC-Eds across units (TSIR Inferential Reasoning which was offered 3 times, TMT Mathematics Technology which was offered 4 times, and TSDI Data Investigations which was offered 7 times)

Full size image

We also consider engagement in the courses from a different perspective. Figure 4 shows that most participants visited a course on 10 or less days (many access a course 5 or less days) and access resources 100 or fewer times (large cluster in bottom left of graph). However, a number of participants accessed a course and viewed resources on many different days, often after a course was completed but still accessible to them. This provides evidence that many participants valued the resources by returning to them even after completion of a MOOC-Ed.

Fig. 4
figure 4

Scatterplot showing trends in number of resources viewed and number of days participants visited a course (each course offering, n = 14, is a different color)

Full size image

In the end-of-course survey completed by 131 participants in the Mathematics Technology MOOC-Ed across three administrations the average rating was 4.4 indicating agreement with the statement “this course enabled me to personalize my learning through differentiated resources and activities” (on a scale of 1–5; 5 strongly agree). This suggests participants appreciated the different resources and activities that were available from which they could select the ones most appropriate for their goals to personalize their learning experience. This sentiment was echoed by a participant from the Data Investigations MOOC who said,

Some all day workshops can be painful and provide little benefit. I think teachers who have given up instructional time and been burned on a poorly designed workshop become increasingly resistant to later PD opportunities. This course has been just the opposite. I can engage with it on my own schedule, rather than losing class time, and I’m coming away with lots of new ideas, resources, and activities. I feel grateful for this opportunity and look forward to finding more like it (TSDI participant).

5.2 Learning from multiple voices

Another element of the external domain in the design of the MOOC-Eds are opportunities for participants to hear multiple perspectives and learn from others who have experiences similar and different from their own.

5.2.1 Enactment of the learning from multiple voices design principle

Rather than a MOOC-Ed course presenting the singular view of a lead instructor, our courses are designed based on research in mathematics and statistics education, recommendations from professional organizations, and practical experiences from educators and students in mathematics and statistics education. One way these perspectives and experiences are enacted is through video recordings of conversations with teachers, teacher educators and researchers in one-on-one settings or as a panel to discuss important issues related to the teaching and learning of mathematics and statistics (See Fig. 5).

Fig. 5
figure 5

Learning from expert panel discussions, interviews with experts, and conversations with teachers. (Links to videos and other resources available in the Appendix)

Full size image

Participants are also provided links to other resources which include open-access readings, activities, and technology tools created by others to support their learning and investigations. This allows opportunities for participants to learn from various sources that can assist them in making informed decisions about what they do in mathematics and statistics classrooms.

Video recordings and animations of students solving mathematics and statistics tasks and teachers implementing effective teaching practices are also included in the courses. Examples of different types of videos are in Fig. 6. These videos provide opportunities for participants to learn from students and teachers and to connect what they are learning in the course to activities they do as classroom teachers (job-connected). More features related to the job-connected principle are provided in a later section.

Fig. 6
figure 6

Example videos of students working on a mathematics task, an animation of students’ reasoning about a statistics task, and a teacher launching a statistical investigation (links for videos included in the Appendix)

Full size image

5.2.2 Learning opportunities supported by the multiple voices design principle

In all three of the MOOC-Eds, participants had opportunities to learn from the instructor, experts, teachers, students, and other resources. In Data Investigations, the expert panel video discussions also supported teachers in reconsidering their own prior experiences in learning and teaching statistics. In a study examining shifts in teachers’ perspectives and practices in teaching statistics, Lee, Lovett, and Mojica (2017) identified the expert panel videos as a primary trigger for reflection and change. For example, in a discussion forum post from Data Investigations Unit 2, a teacher shared her reflection about her own teaching of statistics as it related to the ways in which members of the expert panel discussed statistics teaching. She wrote:

I had a “lightbulb moment”. Although I have been teaching HS [high school] math for 24 years, I have never actually taught “statistics” as defined by the members of the expert panel. I have taught units that I THOUGHT were statistics, but I was merely providing students with a few mathematical tools that statisiticians [sic] can use (e.g. finding a mean, making a histogram, calculating a standard deviation, etc.)…

This post spurred a long discussion thread with 12 participants joining in and discussing why they may have taught statistics in a certain way and their realization and commitment to make changes in their practices. In another discussion forum in Unit 3 of Data Investigations, several teachers referred to a video where one expert (Chris Franklin, the American Statistical Association Ambassador for K-12 Statistics Education—see link to video in Appendix) illustrated how to develop the concept of mean through tasks at different levels of sophistication.

Wow—that whole idea around how to introduce the idea of variability as seen in the ‘Number in your family activity’ at level A through to C is fantastic. Loved the video of Chris and Hollylynne. I can see what an advantage it is when they get to high school level to have been introduced to the concept [of mean] in this way.

In all the courses, videos of students working on mathematics and statistics tasks are included. These were also found to be a source of critical reflection on how to change practices in teaching statistics in the Lee et al. (2017) study. In the Mathematics Technology course, many videos of students working on technology-based tasks are included in each unit, and these often served as important voices for participants from which to learn (Hollebrands, Mojica, & Outlaw, 2018). For example, a Mathematics Technology participant posted,

When I was doing the task, I was trying to ask myself the same questions the boys asked in the video … What are the points doing, are they constant, do they grow, does it reflect or rotate around something. This is a great way to get students to think about the concept without actually involving a graph or a shape.

In all three MOOC-Eds, samples of student work and video recordings of students and teachers were provided to allow opportunities for participants to notice student thinking and learn from practice (Sherin & Van Es, 2005; Wilson et al., 2011). Hollebrands, Mojica, and Outlaw (2018) examined the ways participants analyzed students’ mathematical thinking in the Mathematics Technology course by coding 179 discussion forum posts made by 35 highly engaged participants. Coding considered what participants were attending to (e.g., technology, students, mathematics) and how they were analyzing student thinking if the focus was on students. While during the first two units participants seemed to focus primarily on the use of technology, as they progressed through the course posts showed increased attention to student thinking. When participants focused on student thinking, they made more inferences about students and related what they viewed in the video in the course to their own students and teaching. Participants appreciated learning from students, the instructor, experts, and teachers. They also interacted with their peers in discussion forums.

5.3 Job-connected learning

Another important feature of the external domain of the MOOC-Ed are resources that connect to the work participants are doing in classrooms. The job-connected learning principle is enacted through the use of tasks and lessons that are ready to be implemented in participants’ classrooms.

5.3.1 Enactment of the job-connected learning design principle

Within each MOOC-Ed, participants are provided activities that are classroom-ready and also provided artifacts to examine that include samples of student work and videos or animations of teachers and students engaged in mathematics and statistics learning and teaching. In the discussion forums, participants post reflections about what they are learning in the MOOC-Ed and how it applies to their job experiences.

Each course also provides a framework, informed by research, to support teachers in connecting research to their teaching practice and to assist them in putting together all of the ideas presented in the course (see Figs. 7 and 8).

Fig. 7
figure 7

Framework used in the teaching mathematics with technology MOOC-Ed

Full size image
Fig. 8
figure 8

Framework used in the data investigations and inferential reasoning MOOC-Eds

Full size image

The Teaching Mathematics with Technology framework begins with four key pedagogical activities that take place within the didactic triangle that involve teachers, students, and mathematics (Brousseau, 1997; Freudenthal, 1991; Steinbring, 2005). These activities include the selection and implementation of tasks, posing questions, facilitating discourse, and assessing student learning. Building from the triangle we add technology as a vertex to construct the didactic tetrahedron (Olive et al., 2009, Ruthven, 2012). When technology is added to these four pedagogical activities, a teacher needs to consider how technology is being used (e.g., convey information or perform mathematical actions), what types of representations the technology offers (Kaput 1992), and whether the technology tools have mathematical fidelity (Dick, 2008). The teacher also needs to consider if technology is being used to amplify or reorganize students’ work and thinking (Pea, 1985, 1987). Finally, the type of feedback technology can provide students is another important feature teachers need to consider when choosing to use technology to teach mathematics. This framework is introduced in Unit 1 of the course and different aspects of the framework are used to organize the content and learning materials in each of the Mathematics Technology units (See Appendix for link to video explanation of framework). Teachers have an opportunity to use the framework in the course to consider the pedagogical strategies of designing and choosing tasks, posing questions, facilitating discourse, and assessing students’ reasoning when using technology.

There were also frameworks developed for the Data Investigations and Inferential Reasoning courses to support teachers’ learning of statistics and making connections to their teaching practice (See Appendix). For example, the Data Investigations development team built upon an existing framework (Guidelines for Assessment and Instruction in Statistics Education [GAISE], Franklin et al., 2007) by incorporating research on students’ development of statistical thinking (e.g., Ben-Zvi & Garfield, 2004; Shaughnessy, 2007; and productive statistical habits of mind (e.g., Burrill & Biehler, 2011; Wild & Pfannkuch, 1999). The new framework, Students’ Approaches to Statistical Investigations [SASI, Fig. 8] situates a statistical investigation cycle (e.g., Franklin et al. 2007; Friel, O’Connor, & Mamer, 2006)—posing a question, collecting data, analyzing data, and interpreting results—at the core. While these phases often occur in that order, they can also be non-linear and cyclic in nature. Productive statistical habits of mind are interwoven throughout the SASI framework. A habit of mind is developed when a person approaches situations in similar ways so that a more general heuristic is developed over time (Cuoco, Goldenberg, & Mark, 1996). The SASI framework focuses on specific habits of mind that are productive for engaging in statistics and describes growth in statistical sophistication, from level A to C, which do not necessarily correspond to grade levels. As students are beginning to learn to conduct investigations, regardless of age or grade level, they should have experiences that allow them to grow in their statistical sophistication. In the Data Investigations course, teachers are introduced to the investigative cycle and statistical habits of mind in Units 1 and 2 and have opportunities to use these aspects of the framework to analyze tasks to consider if they engage students in different phases of the cycle and have the potential (or not) to promote students’ development of statistical habits of mind. In Unit 3 and 4 teachers are introduced to the levels of sophistication (e.g., how an important concept like the mean can be developed instructionally at different levels). They have an opportunity to use these levels to analyze students’ work on tasks. Teachers can also apply the SASI framework to their own contexts by using it to design instruction and assessment, and consider students’ work within a statistics investigation.

5.3.2 Learning opportunities supported by the job-connected learning design principle

Several participants referred to the usefulness of the frameworks that were provided in the three courses. For example, in Unit 5 of Data Investigations, some participants reflected on ways in which the SASI framework supported their thinking about statistics. A teacher posted,

The most important point that I got from this course is being able to develop habits of mind (described explicitly in the SASI framework) that will help students to build conceptual frameworks for statistics. … We should be interested in the students’ reasonings (as opposed to the result).

Engaging with the SASI framework in the Data Investigations course not only led to teachers expressing a different perception of statistics, it supported them in imagining ways to change their practice.

Participants also expressed appreciation in forum posts and surveys for how resources included in the MOOC-Eds could inform their teaching practice and be used directly with their own students. Lee, Mojica, Azmy and Barker (2019) found that across all offerings of Data Investigations, two of the main experiences that led to changes in perspectives and practices was the plethora of resources available and the ease of which these resources could be directly used in their classrooms. Even teachers who did not complete the course reported that using GapMinder, an online multivariate visualization tool, and video of students’ thinking with the tool introduced in Data Investigations Unit 1, was one of the most impactful resources they learned about and now use in their classroom.

In end of course surveys across all implementations, 85% of respondents reported the course was effective or very effective in meeting their professional goals and 89% of respondents reported the course was effective or very effective in preparing to make changes in their practice. While we do not have observation evidence of ways the MOOC-Ed materials influenced teachers’ job-connected practices after they took the course, we do have reports from follow-up surveys (Kellogg & Kleiman, 2018). For example, one teacher who took Mathematics Technology in Spring 2017 reported:

…I have been much more aware of what math tasks would be improved with the use of technology and have added CODAP, Desmos, and Math Playground to my lessons, none of which I had ever heard of before. (Kellogg & Kleiman, 2018, p. 12)

One interviewee had taken the Data Investigations course in Fall 2015 and then returned 2 years later to engage in Inferential Reasoning in Fall 2017. She was highly active in discussion forums in both courses. After Data Investigations, she reported using many resources from the course as informal introductions to concepts in her high school Advanced Placement Statistics class [a high school course that can be taken by students and they receive college credit]. When engaged in Inferential Reasoning, she was impressed by the emphasis on using simulations and real data. In the interview, she discussed how one particular video of an inferential reasoning task in a 5th grade classroom inspired her to adapt it for her high school Advanced Placement Statistics students (Kellogg & Kleiman, 2018).

5.4 Peer-supported learning

The fourth design feature focuses on peer-supported learning. It is enacted in the MOOC-Eds by providing discussion forums where participants can communicate with each other about what they are learning in the course and share additional information and resources.

5.4.1 Enactment of peer-supported learning design principle

The first discussion forum in the Orientation unit allows participants to introduce themselves and to get to know their peers, personally and/or professionally as participants share whatever they feel is important. Each user has a profile with basic demographic information populated from the registration survey, and they have an option to upload a picture or additional information.

Units 1–5 contain two different discussion forums where participants are encouraged to talk to each other about materials in the course, share ideas, and resources, and start discussions about issues of interest to them related to the course. These discussion forums are mostly unfacilitated. Lead instructors do post and interact with participants, but the frequency varied by course, instructor, and offering.

In recruitment and advertising materials, teachers are encouraged to take a MOOC-Ed with a local colleague or to consider forming a small group or professional learning team.

Professional Learning Team facilitation guides (See example in Fig. 9) were developed and made available to support small groups of participants who decide to learn together while completing the MOOC-Eds. These guides provide suggestions for organizing a small group meeting for each unit in a course. Recommendations include what to read before a group meeting, what activities can be done together, and prompts that can be used to foster discussions.

Fig. 9
figure 9

Portion of a facilitation guide from the mathematics technology MOOC-Ed

Full size image

5.4.2 Learning opportunities supported by the peer-supported learning design principle

Participants in the MOOC-Eds actively engaged with others in discussion forums. In examining the discussion forum data for courses through Fall 2018 (12 offerings), there were 1538 participants (53.1% of those who engaged in at least the Orientation Unit) who posted at least once in a forum. With 4017 discussion threads and 11,487 total posts, there was an average of between 7 and 8 posts per participant. However, the distribution of number of posts is heavily skewed right, with some participants emerging as “super-posters” who are highly active in discussion forums. An in-depth look at one semester of Data Investigations showed the influence of these super-posters on conversations (Bonafini, 2018). Analysis of the Spring 2018 offering of the Mathematics Technology course found that of the 89 participants who posted in all five units only five participants posted and did not receive a response from others in the course and over 71% of the posts expressed positive sentiments (Barker, Mojica, Hollebrands, & Smiling, under review). Barker and Lee (2018) showed how highly active forum participants in Data Investigations returned to take Inferential Reasoning in a future semester and also emerged as leaders in forums. As illustrated below, some posts were effective in starting lengthy discussion threads with many participants.

Of the many discussion threads across the offerings of courses, the number of posts within a thread certainly varied. There were indications that many participants would create a post that would not generate further discussion (thread size of 1), or minimal discussion (thread size 2-3). These posts may be merely stating a brief response to an investigation or opinion. However, many threads where the initial post included a participant explicitly discussing classroom practices were ones that sparked popular and extensive discussions with many participants. For example, in Fall Data Investigations 2015 a participant made a post in a Unit 2 forum where he shared something from his own practice with a title of “Classroom Experiments.” There were 47 other posts in this thread (across an 8-week period) where participants uploaded and shared activities, discussed activities that were shared, and also generated conversations about the struggles and successes of doing experiments in classes where students collect their own data.

On end-of-course surveys and follow-up surveys sent six months after the close of a course, many participants expressed that one of the most beneficial aspects of their MOOC-Ed experience was engaging in discussion forums with peers. As Mojica, Lee, and Lovett (2018) discuss, Data Investigations participants expressed appreciation for the opportunity to engage with other teachers across geographic boundaries. Some participants indicated they lacked support for improving their teaching within their physical brick and mortar school. For example:

I have no opportunity for collaboration other than on forums such as this one but at the same time I have autonomy to try and do what I want in my class. This course has given me so many resources to continue to improve my teaching without the support of others at my school. (Data Investigations, Spring 2017)

Providing opportunities for participants to interact with their peers through discussion forum posts showed created space for them to engage in supportive conversations to support the exchange of ideas and resources that could impact their knowledge, beliefs, and attitudes related to mathematics and statistics teaching.

5.5 Summary of features in MOOC-Eds

The four design principles (i.e., self-directed, peer-supported learning from multiple voices and job connected) were used in the creation of all three of our MOOC-Eds for mathematics teachers. In Fig. 10 we provide a summary of the key features that were incorporated in our courses that align with these four design principles.

Fig. 10
figure 10

Summary of the four design principles and examples

Full size image

6 Conclusion

The three MOOC-Eds focused on mathematics and statistics were designed using research-based practices used in face-to-face mathematics teacher education as well as four design principles for online PD: self-directed, learning from multiple voices, job connected, and peer-supported. The goal was to examine how the courses, which exist in the participants’ external domain, provide opportunities to influence their personal domain (Clarke & Hollingsworth, 2002). One limitation of the research is the reliance on self-report data from participants on changes they are making to their practice. We acknowledge that future research should follow up with participants and conduct in-depth interviews or observations to better ascertain what teachers have learned in the MOOC-Eds and are applying in their classroom. However, our design, implementation, and research does provide interesting findings and implications for online teacher professional development.

Across the 14 offerings of the MOOC-Eds, there were 5767 enrollments by educators in all US states and 106 countries, with 2543 educators accessing materials in Unit 1 of a course. Over the 3 year period, 964 educators completed an entire MOOC-Ed course and from those who responded to end-of-course or follow-up surveys, were highly satisfied with their experiences and saw direct ways they could enhance their classroom practices. Not all educators who begin an online self-directed PD intend to engage in an entire course. However, aligned with findings from Jacobsen (2019), materials from Unit 1 in a course could provide educators with ideas and resources they could implement in their classrooms. When we noticed that the typical MOOC drop-off trend by units (Perna, et al., 2014) was very similar in MOOC-Ed courses also designed for teachers, we purposely made sure that critical learning opportunities were present in Unit 1. These learning materials not only set the stage for material in later units, but also gave educators concrete ideas and resources they could immediately implement. Designers of online courses for teachers should consider time constraints on teachers and the benefits of self-directed learning, and create early learning experiences that have the potential to have lasting impacts.

Analysis of data logs, course surveys, interviews, and discussion forum posts provide evidence for how participants responded to the design features and impacts they had on their engagement in the online course and learning. These design principles were abstracted from research in face-to-face professional development and we have evidence to support their effectiveness in an online environment. For example, most research focused on teachers’ noticing of students’ mathematical thinking has taken place in face-to-face professional development. Related to the learning from multiple voices design principle, we provide evidence that teachers who are geographically dispersed are able to notice students’ mathematical thinking from videos and discuss it within asynchronous forums.

Each MOOC-Ed utilized expert panel videos to incorporate multiple voices in which the course instructor interacted with experts and master teachers of mathematics and statistics. These video conversations often sparked deep discussions among participants and made lasting impacts on participants’ perspectives about critical issues related to teaching mathematics and statistics (i.e., personal domain). Multiple voices that can be effective in impacting teachers’ beliefs, perspectives, and practices include: the instructor, other experts in mathematics and statistics education, master practicing teachers, K-12 students, and course participants themselves (i.e., peer-supported). The evidence we provide is encouraging to those who are scaling up efforts to support teachers across schools, districts, states, and countries.

Because technology is not a tool that teachers are required to use and statistics is often a topic not all teachers are expected to teach, there was a wide range of prior experiences of those who enrolled in the courses. The opportunities for participants to differentiate their learning opportunities based on their experiences and interests was important for personalizing the course and making the material relevant and meaningful. All three courses engaged teachers with technology tools to explore tasks in mathematics and statistics. We were able to successfully provide content-focused experiences using easy-to-use free online technology tools (e.g., GeoGebra, Desmos, Google Sheets, CODAP, Gapminder) that engaged teachers in doing mathematics and statistics that was likely different from tasks used in their classroom practices. Teachers discussed learning about these new tools with others in the course and reported using these tools and tasks in their classroom with their own students. Thus, these course activities were highly job-connected and showed promise for sustained impact on teachers’ domain of practice.

We also found evidence of the effectiveness of frameworks to guide course development and to support participants in applying what they learned to their own contexts (i.e., job-connected). The framework for using technology (Fig. 7) was designed to go beyond the specifics of a particular tool to consider general features teachers may want to consider when choosing and using technology in mathematics classrooms. For teachers of statistics, the SASI framework (Fig. 8) was useful to them when considering tasks and the ways students were reasoning about those tasks when implemented.

Overall, the three MOOC-Eds were successful in allowing two experienced mathematics teacher educators to design engaging experiences for teachers that have shown to have positive impacts on their beliefs, perspectives and practices in teaching mathematics and statistics. Scaling up professional development for teachers requires much more than simply transforming typical in-person experiences into online videos and readings. By grounding our design in an interconnected model of professional growth that has informed the field of teacher learning for almost two decades, and using best practices from mathematics teacher education and design principles for online teacher engagement, we established a large-scale professional development that engaged and impacted teachers from around the world.