Introduction and background

Professional development (PD) for mathematics teachers in China, especially in Shanghai, has received growing international attention over the last three decades (e.g., Chen, 2020; Paine & Ma, 1993; Sargent & Hannum, 2009; Tucker, 2014; Zhao & Fan, 2022). This is arguably related to the fact that Chinese students, particularly Shanghai students, have demonstrated consistently outstanding performance in large-scale international mathematics assessments such as PISA (e.g., Organisation for Economic Co-operation and Development [OECD], 2016, 2019a), and teachers play an essential role in the teaching and learning of mathematics and, moreover, teachers’ professional development is a key to improving teachers’ teaching and effecting changes in students’ achievement (Adler, 2000; Fan, 2014).

Concerning mathematics teachers’ professional development, researchers have generally agreed that China has established a unique, hierarchal and institutionalized PD system for in-service teachers, in which Teaching Research Groups (TRGs) at the school level and Teaching Research Offices (TROs) within the government education bureaus at the county, district, city and provincial levels play a crucial role (e.g., Fan et al., 2015; Huang et al., 2017). The mathematics TRG is in a sense similar to the “Department of Mathematics” in many other countries, though the TRGs in China focus more on the classroom teaching, e.g., organizing teachers for activities such as preparing lessons together, observing each other’s lessons, reflecting and commenting on observations collectively, whereas the TROs are responsible for guiding teaching research activities, overseeing teaching quality, and providing consultation and PD programs for teachers (e.g., Fan et al., 2015).

So far, most available studies about PDs in China have focused on successful sides of the PD practices, such as school-based teaching research activities by the TRGs (Yang & Ricks, 2012), exemplary lesson (keli in Chinese) development (Huang & Bao, 2006), lesson explaining (Peng, 2007), teaching contests (Li & Li, 2009), lesson observation and critique (Sargent & Hannum, 2009) and master teacher studios or work stations (Li et al., 2011). On the other hand, some researchers have pointed out some weaknesses or problems of specific PD practices in China, such as lack of innovative content, overuse of lecture-based form (Cai & Zhang, 2012), poor logistics services (Xue & Chen, 2012), inadequate connection to teaching practice (Wang, 2013) and short duration (Zhang et al., 2016). Nevertheless, there has been no empirical research focusing on less successful PD practices in China. The present study contributes to filling this gap in research.

By “successful PD”, we refer to the PD practices that meet the participating teachers’ expectations and contribute to their professional growth effectively from their perspectives, and in contrast, we use “less successful PD” to refer to the PD practices that do not meet teachers’ expectations as perceived by the teachers due to various problems or inadequacies. Focusing on less successful sides from teachers’ perspectives, the study examines the PD practices that offer a variety of organized learning experiences to in-service teachers in Shanghai in a relatively comprehensive manner. Those PD practices include in-service training programs, seminars, workshops, conferences and other forms of professional learning activities organized by specific organizations, including but not limited to TROs or TRGs, at different levels for mathematics teachers (Fan, 2014; Sztajn et al., 2017). The study is guided by the following research questions:

  1. 1.

    What aspects are manifested in less successful professional development practices as perceived by mathematics teachers in Shanghai, China?

  2. 2.

    What are the underlying reasons for less successful professional development practices as perceived by mathematics teachers in Shanghai, China?

Concerning teachers’ PD, we believe it is important to understand their perceptions of the PD practices they attended, since they are not only the receivers of PD, but in a large sense they are also ultimate judgers of the effectiveness of PD. There is no doubt that how teachers perceive of the PDs they receive can largely determine teachers’ learning motivation, behaviors and outcomes, which has critical importance for improving the effectiveness of PD. Hence the value of studying the related issues about teachers’ PD from their perspectives is easy to see.

In relation to the research questions, the study also intends to examine whether and how different factors influence teachers’ perceptions of less successful PD practices in terms of (1) teachers’ demographic features such as gender, educational background and length of teaching experience, and (2) school characteristics such as performance level, school type and geographical location. By addressing these questions, we hope that the study can provide research evidence to help understand key aspects and reasons behind less successful PD practices and hence enhance the quality of PD practices in China and, particularly, Shanghai. Furthermore, due to the commonality of the needs and challenges of mathematics teachers for continuously improving their teaching across different countries, we hope the findings from the study can also shed light on issues concerning organizing more effective professional development activities for mathematics teachers beyond the Chinese educational setting.

Literature review and conceptual framework

Although the in-service teacher PD system is not as mature as the pre-service teacher education system, researchers have pointed out that teachers’ in-service experiences are more important than their pre-service experiences for their professional development (Fan, 2014; Sztajn et al., 2017). Some researchers have evidenced that PD can lead to improvements in teachers’ instructional practice and students’ achievement (e.g., Borko, 2004; Fan, 2002; Garet et al., 2001). The growing attention to in-service teachers’ PD and its increasing connection to educational reforms and policy agendas, have promoted research on this topic, especially in the subject of mathematics (e.g., Borko, 2004). Thus, there is a growing body of scholarly publications about PD for in-service mathematics teachers (Sztajn et al., 2017). For convenience, unless otherwise indicated, hereafter we refer to organized PD for in-service teachers simply as PD.

Aspects conceived to evaluate PD

The quality or effectiveness of PD is one of the thematic areas emphasized in teacher PD research (e.g., Avalos, 2011). Researchers have developed several frameworks to evaluate the quality of PD or depict features of effective PD. For example, Guskey (2000) proposed a model of content characteristics, process variables and context characteristics as guidelines for evaluating PD. In the model, content characteristics refer to the “what” of PD, concerning new knowledge, skills and understandings, as well as the magnitude, scope, credibility and practicality of PD; process variables refer to the “how” of PD, concerning the types and forms of PD activities as well as the ways in which these activities are planned, organized, carried out and followed up (i.e., ways to participate); and context characteristics refer to the organization, system and culture in which PD takes place and new understanding is developed. Similarly, other researchers have evaluated PD in terms of the following three dimensions: “What does PD cover?”, “How is PD delivered?” and “Who are involved in PD?”.

What does PD cover? Earlier researchers have identified a number of components or features deemed important for effective PD, among which content focus and coherence were commonly emphasized as core features of PD (e.g., Desimone, 2009; Garet et al., 2001). Desimone (2009) pointed out that content focus, the extent to which PD activities focus on the subject matter content, “may be the most influential feature” (p. 184) to determine the effectiveness of PD. Also, based on a review of research on mathematics PD, Sztajn (2011) identified the content focus on mathematics, student thinking, or curriculum materials as one of the key standards for examining the effectiveness of PD.

Considering coherence, Garet et al. (2001) argued that PD is more likely to be effective if it forms a coherent part of a wider set of learning opportunities for teachers, and they specified the connections with the program goals and other earlier activities or follow-up activities as one way to assess coherence. In addition, other researchers argued that the clarity and properness of the objective of PD, that is, whether the objectives are clearly presented and whether they reflect the mathematics subject (e.g., Sowder, 2007; Sztajn, 2011), the intellectually challenging level of PD (Knapp, 2003), and the practicality of PD contents, i.e., the extent to which they can be connected to classroom practice, should be taken into account for evaluating PD (e.g., OECD, 2019b; Pedder et al., 2008).

How is PD delivered? There are different forms of PD activities, including lectures, seminars, lesson study, workshops, mentoring and study groups (e.g., Garet et al., 2001; Xue & Chen, 2012). Garet et al. (2001) pointed out that the forms of activities may set the context for the substance (content) of the PD activities. Furthermore, several researchers considered the percentage of various forms of PD activities, from which the diversity of forms of activities and the portion of traditional or novel forms of activities can be examined (e.g., OECD, 2019b). The Teaching and Learning International Survey (TALIS) 2018 revealed that the most attended forms are courses or seminars attended in person, with 76% of the participating teachers (OECD, 2019b). In a survey of 9,026 Chinese elementary and secondary school teachers conducted in 2010 (Xue & Chen, 2012), the most attended forms are lectures (73.8% of the participants), followed by lesson study (65.0%). Moyer-Packenham et al. (2011) found that courses and workshops were the dominant forms of PD for mathematics and science teachers in the U.S. In recent years, there are more novel forms of PD, for example, e-learning and practical try-out phases (Barbel & Biehler, 2020).

Darling-Hammond et al. (2017) reviewed 35 methodologically rigorous studies on PD and identified three features concerning how effective PD was organized: active learning, collaboration, and sustained duration. Active learning, as opposed to passive learning that often occurred in the form of lectures, is a feature of effective PD frequently stressed by researchers (e.g., Desimone, 2009; Knapp, 2003; Sowder, 2007). However, teachers’ active or passive participation in PD was not necessarily determined by the forms of PD activities. In fact, how teachers participate in such activities, i.e., attending lectures, observing lessons, collective participation, etc. and whether they participate online or onsite matter to a considerable extent. Considering PD as a specific type of adult learning, the richness of ways for teachers to participate in PD can meet the various needs for teachers’ active construction of knowledge and skills, thus influencing the motivation of teachers (Xue & Chen, 2012).

The duration of PD was also used as an indicator by many researchers when evaluating PD (e.g., Desimone, 2009; Garet et al., 2001; Sztajn, 2011). Specifically, Sztajn (2011) looked into the contact hours of PD sessions, arguing that “a small number of hours spread over a short amount of time” would potentially cause mathematics PD to be less successful. Finally, researchers emphasized the need for assessment for PD participants (e.g., Darling-Hammond et al., 2017; Zhang et al., 2016), among whom Cai and Zhang (2012) further identified several principles, including assessing from various perspectives, transforming summative assessment to formative assessment and integrating these assessments with follow-up evaluations. Based on a review of three synthesis articles, Sowder (2007) also stressed the need for assessment that provides teachers with feedback they need to grow.

Who are involved in PD? Another major aspect that researchers have looked into when examining the effectiveness of PD concerns the providers/organizers of PD. For example, Xue and Chen (2012) looked into teachers’ satisfaction with organizers in terms of the PD management, as well as logistic issues such as costs, accommodations and food. Some researchers called for a formative assessment of organizers, either from the participants (Cai & Zhang, 2012) or from third-party agencies to conduct evaluation on the organizers (Zhang et al., 2016). The coordination between PD organizers from different levels in educational systems, such as those in different regions of China, was also considered important (Wang & Hu, 2020).

The quality of trainers (or instructors) is another factor leading to successful or less successful PD. In a qualitative study on teachers’ perspectives on effective PD, Bayar (2014) found that teachers considered the quality of trainers as a component that influenced the effectiveness of PD, where one specific indicator mentioned by one participant in that study was trainers’ preparation for PD. Cai and Zhang (2012) maintained that appropriate PD trainers should have a solid theoretical basis in education as well as rich practical knowledge (also see Darling-Hammond et al., 2017).

Finally, it should be pointed out that, even with increased recognition about the importance of teacher PD, as researchers have noticed, a significant number of publications were small-case studies with a small number of teachers, and very often, were on successful PD practices with positive and “significant” results, which were possibly due to the publication bias (Sztajn et al., 2017). Moreover, as Tirosh et al. (2015) recommended, researchers need to follow up on less successful results in mathematics PD in order to gain insight on why a specific PD program fails to have a lasting impact; nevertheless, to date, there has been no specific investigation of less successful PD practices in a systematic way, particularly in China. This paper begins to address this gap.

Problems and inadequacies concerning less successful PD

Much of the available literature concerning less successful PD points to some disadvantages or problematic aspects of specific kinds of PD. According to the TALIS 2018 study, which is a large-scale international survey covering about 260,000 teachers from 48 countries and economies (regions) in Europe, Asia, North and South America, and Oceania, the two major obstacles for lower-secondary teachers to participate in PD were the time conflicts (with teachers’ work schedule) and lack of incentives (OECD, 2019b). In a large-scale survey of a national probability sample of 1,027 mathematics and science teachers in the U.S., Garet et al. (2001) reported that many PDs did not have high-quality features (e.g., emphasis on mathematics content knowledge, opportunities for active learning, coherence, enhancement of teachers’ knowledge and skills) and indicated that providing PD with multiple high-quality features required a substantial amount of lead time and planning, which schools and districts might not always have. In another large-scale survey of a national random sample of 1,126 teachers in England, Pedder et al. (2008) found that many PDs not only lacked coherent focuses, collaborative approaches, consistent approaches within and between schools and connections to classroom practice, but also were rarely evaluated in relation to planned outcomes. Lindvall (2017) reported two large-scale PDs for mathematics teachers in Sweden, one PD placed higher demands on changes in instruction (more familiar for primary-level teachers) and the other PD emphasized mathematical content and competencies (more familiar for secondary-level teachers), and showed that the PD program resulted in different effects for primary and secondary levels, which was probably due to teachers’ different familiarity with the advocated teaching practices, and thus she argued that the PD content should be adjusted to cater for the specific needs of teachers at each grade level. In a qualitative study of 50 in-service mathematics teachers in South Africa regarding their views on a PD, Chigonga and Mutodi (2019) reported that the teachers considered the workshops of PD were disconnected from classroom practices, and the connection is a crucial measure of the success of PD.

Considering the Chinese context, there is a remarkable scarcity of large-scale quantitative research into the effectiveness of teacher PD (Ke et al., 2019). An earlier related large-scale national survey of more than 9,000 elementary and secondary teachers in China was conducted in 2010. The study reported that some in-service teacher training programs lacked a pre-analysis of teacher needs, connection to classroom practice, different forms of activities, active learning opportunities, external supervision and evaluation, administrative and logistics support and good trainers (Xue & Chen, 2012). Furthermore, Wang and Hu (2020) suggested there was a lack of proper coordination and smooth communication among TROs as key PD organizers at the district, city and provincial levels.

Some researchers have also examined elementary and secondary teachers’ perspectives of PD in China, including (1) the suitability of national/provincial/district-level in-service teacher training programs with 188 teachers in Zhejiang (Cai & Zhang, 2012) and with 259 teachers in Xinjiang (Zhang et al., 2016), and (2) school-level PD practices with a sample of 324 teachers: 175 from Shanghai and 149 from Mianyang (Zhang & Pang, 2016). Some inadequacies were identified for different levels of PD: firstly at the national-level PD (e.g., Zhang et al., 2016), the issues of repetitive contents and a lack of good trainers providing necessary feedback, follow-up support and proper supervision were identified; secondly, at the provincial/district-level PD (e.g., Cai & Zhang, 2012), there were the issues of out-of-date forms, inappropriate timing or short duration of the programs, and a lack of the following essential or desirable qualities including innovative content, practical content, diverse forms, and teachers’ need analysis; and thirdly, at the school-level PD, the lack of support for collaborative learning was found among Shanghai teachers in comparison with Mianyang teachers (Zhang & Pang, 2016). Besides, among various levels of PD, researchers almost uniformly pointed out that there was a lack of necessary and reasonable assessment for teachers. Nevertheless, it should be noted that the aforementioned studies focused more on general school teachers, not particularly on mathematics teachers.

Focusing on Chinese mathematics teachers, two regional survey studies, one with 169 leading teachers in Guizhou (Wang, 2013) and the other with 148 teachers in Qingdao (Li, 2017), revealed four major problems of PD: (1) inadequate connection to teaching practice, where the PD contents were not in accordance with local curriculum materials; (2) lack of individualized and innovative content, such as contents catering for the needs for teachers with different educational backgrounds, professional levels/titles,Footnote 1 genders or from different ethnic groups; (3) over-emphasis of theoretical content at the expense of practical content, where some PD mainly offered teachers theories of teaching and learning instead of showing how they could actually be used in classrooms; and (4) time conflicts, where teachers found the duration of PD too long or in conflict with their work schedule.

Partly related to the outstanding performance of Shanghai students in PISA, extensive attention has been given to the mechanism and system of teacher PD in Shanghai (e.g., Huang & Bao, 2006; Yang & Ricks, 2012). Shanghai, as a pioneer of educational reforms in many areas (e.g., curriculum reform) in China, has developed a series of strategies and mechanisms related to teacher PD, such as building three levels (school-level, district-level and city-level) of teaching-research networks, and implementing school-based teaching research (Yang et al., 2020). Teachers in Shanghai, compared with other cities in China or other countries, tend to have more opportunities to attend diverse PD practices, such as attending lectures by university professors and training programs offered by school-level TRG or district-level TRO (e.g., Zhang & Pang, 2016). As shown in the TALIS 2018 result (OECD, 2019b), 94.0% of Shanghai teachers had participated in school-level PD activities, showing an impressive difference compared with all TALIS countries with an average of 50.8% (also see Opfer, 2016).

However, most available studies have focused on successful PD cases (e.g., Yang et al., 2020), and as mentioned earlier, there have been no studies that focused on less successful PD practices in China, let alone from teachers’ perspectives.

Conceptual framework

Drawing on the literature reviewed above as well as our own professional experiences in Shanghai, in this study we established a conceptual framework to investigate Shanghai teachers’ perceptions about less successful PD practices. The framework looks at teachers’ perceptions from six aspects of less successful PD. These six aspects are: (1) objective (the “why” dimension), (2) content (the “what” dimension), (3) organization, (4) way to participate, (5) time, duration and frequency, and (6) assessment and management (the “how” dimension). For each aspect, possible problems or inadequacies were further identified for investigation, as shown in Table 1.

Table 1 A conceptual framework about different aspects of less successfully organized PD
Full size table

It should be pointed out that in the table, a few of the problems or inadequacies which are not mentioned in the literature reviewed earlier are added based on our own experience working with Shanghai school teachers, such as (1) lack of substance (the “content” aspect), as some organized PD activities might merely be a formality and they failed in offering substantial knowledge for teachers, (2) scale too large (the “organization” aspect), thus making organized PD hard to cater for teachers’ individualized needs, and (3) limited or inconvenient ways to participate (the “way to participate” aspect), for example, some onsite organized PD activities that required teachers to travel for long hours or those exclusively for specific groups of teachers.

Furthermore, we explored the underlying reasons for less successful PD from the dimension of “who”: (1) organizers: possible reasons included their plan, coordination of PD and the inappropriate evaluation for them, and (2) trainers: possible reasons included their understanding of teachers’ needs, practical experience, theoretical knowledge, training experience, preparation and creativity.

In addition, we looked at how teachers’ demographic features and school characteristics may bear an influence on teachers’ perceptions of less successful PD programs since they may experience different types of PD (Garet et al., 2001), for example, Mahmoudi and Özkan (2015) explored experienced and novice teachers’ perceptions about PD and revealed differences in their preferred types of PD activities. However, much less is known about how different teachers’ demographic features and school characteristics bear an influence on mathematics teachers’ perceptions of less successful PD practices. Moreover, the high portion and frequency of teachers’ participation in school-level organized PD in China, mostly by TRGs, as revealed by TALIS 2018 (OECD, 2019b) and other studies (e.g., Wei et al., 2019), also called for the need to look into the differences between different levels of organized PD practices in China.

Methods and procedures

The data in the study were collected through a questionnaire survey from a stratified random sample of 132 teachers from 9 secondary schools in Shanghai and follow-up interviews with 18 teachers with 2 from each school. After the interviews, relevant documents of less successfully organized PD were also collected. The reason we focused on Shanghai teachers is not only that mathematics education in Shanghai has gained much international interest (e.g., Cheng, 2011; Tucker, 2014), but also that our research centre is based in Shanghai, and the academic network we have in Shanghai guarantees the feasibility of obtaining such a relatively large-scale random sample.

Research instruments

Questionnaire. The questionnaire was designed following the conceptual framework as described above (also see Appendix 1). It consists of five parts. The first part is about the participants’ background information including district where they work, gender, age, length of teaching experience, professional title, education, the frequency of PDs they attended in the past three years. The second part is about teachers’ satisfaction of overall and four different levels of PDFootnote 2 (school-level, district-level, city-level and country-level). The third part is about teachers’ perceptions of key aspects in less successful PD. The fourth part is about the underlying reasons behind less successful PD. The last part is an open-ended question to elicit teachers’ suggestions on how to improve the quality of PD.

Interview. The semi-structured follow-up interview was designed to gain in-depth information about teachers’ less successful PD practices and their perceptions of underlying reasons for those less successful PD practices (also see Appendix 2).

Relevant documents including teachers’ field notes, outlines, syllabus, lecture notes of less successful PD programs they attended, if available, were also collected after the interview to supplement and triangulate the data obtained from the questionnaire and interviews.

To ensure reasonable validity and reliability, a pilot study was conducted with five teachers from two non-sample schools in Shanghai. The results from the pilot study were used to refine the instruments. Based on the feedback, we adjusted some items, for example, deleting the item “insufficient funding for PD” as participants (teachers) did not know whether PD were adequately financed. For the final questionnaire, the Cronbach’s α of each part using the Likert scale is between 0.83 and 0.96 (see Table 2), indicating a high internal consistency.

Table 2 Cronbach’s α of each part of the questionnaire
Full size table

Data collection

A highly strict multi-stage stratified random sampling method was used for selecting the participants. First, we randomly selected 8 of the 16 districts in Shanghai. Second, we categorized all 299 secondary schools in the 8 districts into high-performing and ordinary schools, based on the school ranking reported in a local educational website.Footnote 3 Third, we randomly selected a high-performing and an ordinary school from each sample district. Then, we sent an invitation letter to invite these 16 schools, and finally 9 of them, 6 high-performing and 3 ordinary schools, replied to us and agreed to participate in the study.

Due to the COVID-19 pandemic, we sent out our electronic anonymous questionnaire to all 138 mathematics teachers from the nine schools, and collected back 132 valid ones, with a response rate of 95.7%. Table 3 shows the profile of the participating teachers. The follow-up interviews were conducted with 18 of the participants, with 2 from each school.Footnote 4 For anonymity, 18 interviewees were renamed T1 to T18. With the teachers’ agreement, all interviews were recorded, and the average duration was about 20 minutes. After the interview, we collected the available relevant documents of the less successful PD that the teachers mentioned during the interview.

Table 3 Profile of participants (N = 132)
Full size table

Data analysis

Descriptive statistics such as means, standard deviations and percentages were used to analyse the relevant data collected from the questionnaire. Significance tests including chi-square tests and independent t-tests were applied, when appropriate, to detect if there were statistically significant associations or differences between different groups of participants concerning their perceptions of different aspects of less successful PD practices. Suggestions in the open-ended question in the questionnaire were gathered in a table and classified by two researchers.

The data collected from the interview were first transcribed verbatim and then coded using the conceptual framework described above. Each less successful PD could be coded with more than one inadequacy. For instance, T3 described a less successful PD with two inadequacies: “lack of practical contents” and “too few sessions”. To ensure reliability, two researchers coded these transcripts independently. Inconsistencies were discussed until agreement was reached. Relevant information in the supplementary documents collected after the interview was identified and then, when applicable, used to triangulate the interview data.

Results and discussions

An overall picture

The results show that, overall, mathematics teachers in Shanghai participated rather frequently in school-level and district-level PD while they rarely attended city- and country-level PD in the past three years. As can be seen in Fig. 1, 69.7% of the teachers attended school-level PD at least twice or three times a month and the figure for the district-level PD was 42.4%. It was found that the teachers participated in school-level PD more frequently than district-level PD, which is understandable as school-based PD has become a national policy about teachers’ PD in China since 2003 (Liu & Xie, 2021). In comparison, 62.9% of the teachers had no opportunity or only once to participate in city- or country-level PD. This result was consistent with the national training plan (Ministry of Education (China) [MOE], 2021), in which many national training programs targeted leading teachers, rather than all teachers.

Fig. 1
figure 1

Distribution of frequency of teachers’ participation at different levels of PD in the past three years (\(N = 132\))

Full size image

Table 4 shows teachers’ satisfaction with different levels of PD, in which teachers were most satisfied with school-level PD (\(M = {1}.{86},SD = 1.13\)). This result was not surprising, since the contents of school-level PD for teachers were generally more relevant to what they taught or directly address their professional needs, and what teachers learned from their school-level PD had direct practicality in their daily instruction. As T17 noted that: “[In comparison with school-level PD, the higher level] district-, city- and country-level PD may not be so practical for us, but more of theoretical guidance.” Teachers were least satisfied with country-level PD (\(M = {2}.{17},SD = 1.37\)). It seems related to the result that teachers had much fewer opportunities to attend country-level PD practices, as aforementioned, and moreover, they were often less relevant to classroom instruction.

Table 4 Teachers’ satisfaction of PD practices at different levels
Full size table

Overall, 13 (9.8%) of the 132 participants were unsatisfied with the PD they attended (\(M = {2}.{35},SD = 1.04\)), which is largely consistent with the findings from TALIS 2018, in which 87.5% of the Shanghai teachers thought that the PD they attended had a positive impact (OECD, 2019b). It should be noted that even if a teacher had an overall positive evaluation on the PD, he/she may have experienced less successful PD due to various problems or inadequacies of the PD.

In the follow-up interviews, all the 18 interviewees described their experiences in attending less successful PD that did not meet their expectation one way or another. In total, they revealed 65 problems/inadequacies that they perceived unsatisfied about school-, district- and city-level PD. 81.5% of them were for the district- and city-level PD, such as (1) “lack of practical contents” (11 times), (2) “too few sessions” (5 times), (3) “objectives not specific for the mathematics subject” (4 times), “lack of focus and coherence” (4 times) and “lack of substance” (4 times). Below we shall report the results regarding teachers’ perceptions of key aspects in less successful PD practices, which are the focus of this study.

Key aspects of less successful PD practices

By “key aspects” in the study, we refer to the aspects that teachers perceived as most unsatisfactory in less successful PD practices, which has fundamental importance to examine those key aspects for improving the quality of PD for mathematics teachers.

In the questionnaire, all the participating teachers were first asked to identify two to three aspects of less successfully organized PD practices that they perceived as most unsatisfactory, and then, for each identified aspect, indicate to what extent they agree or disagree with a set of statements about possible problems/inadequacies mentioned in the conceptual framework as discussed earlier. Table 5 presents a detailed picture of the results.

Table 5 Teachers’ perceptions of problems/inadequacies in the unsatisfactory PD aspects identified
Full size table

As can be seen from Table 5, in terms of the percentages of selection, the three most unsatisfactory aspects to the teachers were found in “time, duration and frequency” of the PD activities they attended (68.9%), followed by “assessment and management” (62.1%) and “objective” (43.2%).

A further look at Table 5 reveals that, regarding the problems about “time, duration and frequency” of the PD activities, most respondents (62.4%) who were unsatisfied with this aspect indicated that the durations of the PD activities they attended were too long (\(M = 3.88,SD = 1.35\)). This is likely related to the fact that the teachers usually have heavy workloads. During the interview, T11 recalled her unpleasant experience of taking a district-level PD program, “[For the required district-level PD] there were generally a set of online courses, in which I had to watch dozens of course videos online with certain time requirements. But we were not free to watch those videos every day, and then the courses tended to lose their substance. [… As you know] I had lots of work to do. As a teacher [having at least two lessons each day], each lesson takes time (40 min), and we don’t have a complete block of half day time or such a timeslot to watch course videos. […] Thus, I have to watch these videos when grading students’ assignments.” T14 also pointed out that “Once there was a city-level PD, four full days in a row, I felt that time arrangement was too tight, so I hesitated to sign up. Due to the time for commuting, I had to stop my own instruction because of the long-time span. Yes, the most painful thing for in-service teachers here, is that we found it hard to stop everyday teaching.”

About one third (34.1%) of the respondents reported that PD having too short duration was a problem for them. During the interview, T16 explained that she participated in a school-level training program on educational research and academic writing, “The trainer spent only one afternoon teaching us a [step-by-step] general guide to writing a research paper. As a result, I felt that this training was not so helpful as we could not absorb the content due to the short duration.”

It should be noted the fact that teachers had different views about the problems concerning the “time, duration and frequency” of the PD activities they attended is understandable and, in a sense, expected. This is because not only there were different PD practices with regard to this aspect, and in fact, to all the other five aspects as well (see more below), but also the teachers who attended the PD activities had different needs and backgrounds. It reminds us that the issues about PD are rather complex.

About the aspect concerning “assessment and management”, as shown in Table 5, the largest percentage of respondents (64.0%) felt that “inappropriate assessment for participants” was a problem (\(M = {3}.{81},SD = 1.26\)). In the interview, many teachers pointed out some inappropriate assessments, such as a final paper irrelevant to the PD content (T12), a summary paper of about 1,000 words assigned on the last day of the PD program without any formative assessment (T11). In addition, more than half of the respondents considered that “lack of necessary management”, “lack of necessary assessment for participants” and “inappropriate management” were problems in the PD activities they attended.

About the problems concerning “objective”, two thirds (66.7%) of the respondents reported that the objectives were unclear (\(M = {3}.{92},SD = 1.48\)). As setting clear objectives is the first step for an effective PD and they also influence the assessment of PD, this result suggests that the organizers of PD need to specify clear objectives and define these objectives in observable terms at the initial phase of PD design, and then conduct assessments drawn on these objectives (Guskey, 2000). In addition, there were more than half of the respondents who perceived that the objectives of PD activities were set too high or not specific for the mathematics subject. For example, in the same PD seminar on educational research and academic writing (with participants from different school subjects as mentioned by T16), T15 indicated that “the trainer provided rough steps of how to conduct a research project. […] As interdisciplinary research was popular, at the beginning, we [mathematics teaching team] tried to conduct an interdisciplinary study with the teaching team of physical education. Then we found it too difficult to conduct such [an ambitious] study [without sufficient research experiences, etc]. [...] Finally, we did not continue our project.”

There were only around 10% of the teachers unsatisfied with the other three aspects: “way to participate” (11.4%), “content” (10.6%) and “organization” (9.8%). Among them, most (71.4%) of the respondents considered that having “limited”, “passive” and “inconvenient” ways to participate in the PD activities was a problem for them (for each item: \(M = {4}.00,SD = 1.36\)). During the interview, T17 mentioned that the COVID-19 pandemic limited the ways to participate in the district-level PD activities. For example, before the COVID-19 pandemic, district-level PD practices were usually held in different schools, while during the pandemic, district-level PD practices were generally delivered online. Moreover, online PD may lead to teachers’ passive participation. For example, T2 mentioned her passive participation experience in a city-level online PD, saying: “We watched the video of teaching contests, but it is quite different from when we observe the classroom onsite, [as] I could not observe students’ reaction, teachers’ teaching style and had no opportunity to know critiques and discussions between teachers and teaching research fellows [in great detail].” In addition, she also pointed out that it was very inconvenient for her to participate in a city-level PD activity that required physical (instead of online) attendance, since the duration of PD was only one hour while the commuting time was two to three hours.

As to the “content” aspect, the main problems were found in “lacking of focus and coherence (\(M = {4}.0{8},SD = 1.71\)), of practicality (\(M = {4}.00,SD = 1.73\)) and of new contents (\(M = 3.92,SD = 1.66\))”. In the interview, T17 summarized her experience of attending an intensive district-level PD with lack of focus and coherences: “[In a PD program with several sessions] we could only attend one session in a given time period. None of those sessions has a follow-up/related session either on how to write a paper or a follow-up research study on the same topic. For us, the content is not very systematic, which failed to provide us with continuous support [regarding the same topic]”. T3 shared her experience in a district-level PD about task design (see Fig. 2): “In fact, this online PD is mainly the presentation of several mathematical tasks that teachers designed. However, the PD had no information about the design process and the guidelines for implementing tasks with participants. I still remain confused about how to design tasks by myself. T11 reported that “Some PD activities’ contents were out of date. For one PD course, we are quite familiar with the trainer. When we first took that course, the trainer at that time was quite young… [even though many years have passed] many instructional videos used in the PD course remain the same, without changes or updates.”

Fig. 2
figure 2

An example of PD on task design (excerpt from supplementary materials collected in the study)

Full size image

Finally, about the “organization” aspect, the main problems were found in the “lack of substance” (\(M = {3}.{92},SD = 1.{04}\)), followed by “out-of-date forms” (\(M = 3.75,SD = 3.97\)) and “lack of diverse forms” (\(M = 3.69,SD = .95\)). This is consistent with what has been reported (e.g., Gao & Liu, 2014) that some PD activities, e.g., school-based curriculum development and online training, in China were merely for formality without adequate substance. In the interview, T16 mentioned that some PD she attended were merely for formality: “We should not have participated in the PD just to fulfill the official requirement. […] We should not have gathered all different groups of teachers and asked every participant to do the same thing, as teachers’ needs are quite different. Yes, some city-level PD offered us some options to choose from, but those options are still relatively rough and not so detailed [catering for our own needs].”

Perceptions of less successful PD between teachers with different demographic features

To further detect if there were differences in teachers’ perceptions about less successful PD practices between different groups of teachers in terms of demographic features, statistical significance tests were applied to analyse the relevant data. The results of chi-square tests revealed that there were no statistically significant differences between the distributions of different teachers’ dissatisfaction with all the six aspects except the “time, duration and frequency” aspect, which was identified as unsatisfactory by a significantly higher proportion of teachers from private schools (81.6%) than from public schools (63.8%, \(\chi^{2} = 3.891,p < .05\)). In addition, some interviewees from private schools indicated that many less successful district-level PD failed to address the content used in private schools or bilingual schools as those PD activities mainly targeted content taught at public schools. For example, T12 said, “I hope [district-level] TRO can pay attention to the needs of [our] private schools or bilingual schools, and provide us with more guidance.”

Independent t-tests were applied to the data about the teachers’ perceptions of specific problematic areas in the unsatisfactory aspects they identified for less successful PD practices to detect if there are statistically significant differences between different groups of teachers. Overall, there were statistically significant differences between different groups of teachers in terms of demographic features, but not in terms of school characteristics. More detailed results are shown in Table 6 (for brevity, items without statistically significant differences were excluded in the table).

Table 6 Statistical results of different teachers’ perceptions of problems/inadequacy about less successful PD
Full size table

As shown in Table 6, compared with novice teachers, experienced teachers perceived significantly more having “aims too low” (\(t({47}) = {2}{\text{.225}},p < .05\)), “too short duration” (\(t({80}) = {2}{\text{.071}},p < .05\)), “too many sessions” (\(t({83}) = {2}{\text{.377}},p < .05\)), “inappropriate assessment for participants” (\(t(7{3}) = {2}{\text{.253}},p < .05\)), “inappropriate management” (\(t(7{3}) = {2}{\text{.012}},p < .05\)) and “lack of necessary management” (\(t(7{3}) = {2}{\text{.333}},p < .05\)) as problems in the less successful PD activities they attended.

The above results between novice and experienced teachers are understandable. Due to different professional backgrounds and needs, it was more likely that experienced teachers had higher expectations of “objective”, “assessment and management” for their PDs. The results were also confirmed during the interviews. For example, concerning the number of sessions, T4, who had more than 20 years of teaching experience, said: “In fact, I think nowadays novice teachers are more diligent and have a strong desire to learn more [meaningful or new content] from PD.” A novice teacher, T3, pointed out: “I have gained a lot from every district-level PD, twice a month, and I hope there will be more district-level PD”.

Significant differences were also found between male and female teachers in terms of (1) “way to participate” and (2) “time, duration and frequency” of the PD activities. Female teachers perceived significantly more that having “limited” (\(t(12) = 3.{240},p < .01\)) and “inconvenient” (\(t(12) = 3.{240},p < .01\)) ways to attend PD as a problem, while male teachers agreed significantly more that the duration of the PD they attended were too short (\(t(33.174) = {2}{\text{.243}},p < .05\)). It appears that the results are largely related to the differences in the family responsibility, in other words, female teachers may spend more time than male teachers in their families, which is particularly evident in Chinese culture (e.g., Zhang & Ryden, 2002), and thus female teachers may have inadequate time for long-duration PD and would like to take part in PD in more flexible and convenient ways.

Regarding professional titles, compared with those with a lower-level title, teachers with a senior and above title perceived significantly more having “too short duration” (\(t(80) = {2}{\text{.441}},p < .05\)), “lack of necessary assessment for participants” (\(t(74) = {2}{\text{.256}},p < .05\)) and “lack of necessary management” (\(t(28.022) = {2}{\text{.816}},p < .01\)) as problems in the PD activities they attended. Again, the results are understandable as those with senior level titles were likely more experienced teachers and had higher expectations of PD activities. In short, we think it is reasonable to argue that the differences found in teachers’ perceptions of problems in PD activities between teachers with different teaching experiences, educational backgrounds and professional titles are largely related to the differences in their professional backgrounds and needs for PD.

To sum up, in answer to research question 1, the results show that the teachers perceived “time, duration and frequency”, “assessment and management” and “objective” as the three most unsatisfactory aspects for less successfully organized PD they attended, while a small percent of the teachers were unsatisfied with the other three aspects: “way to participate”, “content” and “organization”. The specific problems in different aspects reported by the teachers included that, to various degrees, the contents were lack of practicality, novelty, focus and coherence, the ways of participation were limited, passive and inconvenient, the objectives were unclear or inappropriate, the organization was merely for formality, the duration was unsuitable and the management and assessment for participants were lacking or inappropriate. The statistical significance tests show that teachers’ demographic features, especially their teaching experience, play an important role in their perceptions of key aspects and problems in less successful PD.

Underlying reasons for less successful PD practices

The questionnaire data revealed that teachers perceived that PD practices were less successful mainly due to “lack of appropriate evaluation for organizers” (\(M = 3.54,SD = 1.4{1}\)) and “lack of necessary coordination between organizers at different levels” (\(M = 3.52,SD = 1.36\)). Concerning this, we noticed that researchers have reported that the lack of rigorous evaluation (e.g., Pedder et al., 2008), as well as external supervision and evaluation (e.g., Xue & Chen, 2012) was manifested in less effective PD. Also, it was consistent with Wang and Hu’s (2020) report on the lack of proper coordination and smooth communication among TROs at different levels (PD organizers) in China. Similar situations also existed in Shanghai teachers’ PD practices, and to a large extent, the organizers were not held accountable for the less successful PD programs.

About the other possible reasons like trainers’ lack of certain competencies or organizers’ lack of a clear plan, as mentioned in the conceptualization of the study, the teachers generally did not perceive that such reasons led to less successful PD (\(M < 3.50\)), with the lowest rating on the trainers’ “lack of theoretical knowledge” (\(M = 2.82,SD = 1.3{5}\)). In the interview, teachers also pointed out some other reasons for less successful PD, which included deviation from pre-defined objectives, lack of continuous support or access to PD materials and the organizer/trainer’s failure to accommodate teachers’ specific needs.

Among the above reasons, it is particularly noteworthy that a tension or inconsistency was revealed between teachers’ needs for customized PD and the organizers/trainers’ unawareness or failure to cater for those needs, even though they may be aware of the existence of such needs. In the interviews, six teachers pointed out that the organizers only focused on teachers’ general needs rather than their individual needs. For example, T7 described his experience in a PD program on classroom discourse: he and his fellow trainees first watched selected videotaped lessons from one or two of the trainee teachers, and then attended a presentation on the results of video analysis based on selected videos by a team of researchers from one university, and finally listened to lesson critiques by teaching research fellows from the TRO of the education bureau. T7 reported that this practice was not very helpful “especially for those teachers whose lessons were not selected”, as “it was not as effective as focusing specifically on their own lessons”. He suggested the organizers reduce the size of the PD, having a smaller number of teachers in such PD activities to allow every participant to have the opportunity to get individualized advice.

Finally, in responding to the open-ended question in the questionnaire and the corresponding question in the interview which asked their opinions on how to improve the effectiveness of PD, teachers called for more practical (nine teachers) and enriched (eight teachers) PD contents, a more variety of forms of PD activities (six teachers), and more sessions to choose (six teachers). Other suggestions included setting the objectives more mathematics-specific, offering more customized and systematic content, organizing PD activities with more substance and providing more resources. These results are consistent with those discussed above.

Discussion and conclusions

This study examined key aspects and underlying reasons for less successfully organized PD practices in Shanghai from teachers’ perspectives. The data were collected from 132 mathematics teachers in nine randomly selected secondary schools through a questionnaire survey and follow-up interviews with 18 of them. This study is the first to document Shanghai mathematics teachers’ perspectives of less successful PD and provides a comprehensive view of their perspectives with empirical evidence in a systematic way. The results show that although Shanghai mathematics teachers were largely satisfied with different levels of organized PD, there were indeed less successfully organized PD practices with various problems and inadequacies that call for attention and improvement.

First, Shanghai mathematics teachers perceived “time, duration and frequency”, “assessment and management” and “objective” as the three most unsatisfactory aspects in less successfully organized PD practices, while a small percent of the teachers were unsatisfied with other three aspects, namely, “way to participate”, “content” and “organization”. This result highlights the importance of addressing the issue concerning the “time, duration and frequency” of PD for participating teachers, which was also supported by the data of TALIS 2018 that more than half of Shanghai teachers of all school subjects (54.9%) indicate continuing PD conflicts with the teacher’s work schedule (OECD, 2019b). By contrast, the percentages of teachers who were dissatisfied with this aspect in other East Asian countries (e.g., Japan: 87.0%; Korea: 88.1%) are far above that of Shanghai, as well as TALIS average (OECD, 2019b). It appears clear that the issue calls for particular attention from PD providers and organizers of all school subjects, and further cross-cultural comparisons of the PD schedule and mathematics teachers’ work schedule may produce valuable insights to designing successful PD globally.

Furthermore, it should be noted that the six unsatisfactory aspects are inter-related, to a varying degree. For example, making PD objectives explicit could assist in designing proper assessment, which in turn could be used to set clear objectives of the next PD, supporting an evidence- or data-based continuous improvement process (e.g., Desimone et al., 2002). It implies that a holistic view of those issues is needed when designing and implementing PD.

Second, in the different aspects, the specific problems perceived by the teachers include that, to a more or less degree, the contents were lack of practicality, novelty and coherence, the ways of participation were limited, passive and inconvenient, the objectives were unclear or inappropriate, the organization was merely for formality without adequate substance (content), the duration was unsuitable, and the management and assessment for participants were lacking or inappropriate. All those problems merit careful attention in designing and delivering PD for specific groups of teachers. In particular, as revealed in the study (e.g., T11 and T12’s interview), prior assessments for PD participants were generally summative assessments (which was consistent with the findings from Cai and Zhang, 2012), and there was a lack of formative assessments during the PD and a disconnection between the assessment and teachers’ lifelong professional development. Consequently, some teachers either found the assessment to be a burden or felt empty-handed on what they have obtained at the end of the PD. Thus, a more balanced form of assessment for PD participants is needed in the design and delivery of PD. Concerning this issue, several researchers have also provided related suggestions, including raising organizers’ awareness of evaluation, involving school leaders in the strategic planning of PD as well as follow-up actions and supports (Pedder et al., 2008), and integrating professional learning into initiatives or reforms concerning school and students (Darling-Hammond et al., 2017). Future study in this direction is needed in order to obtain research-based evidence to address the issue.

Third, there were statistically significant differences in teachers’ perceptions of a variety of issues concerning less successful PD between different groups of teachers in terms of demographic features (e.g., length of teaching experience and gender). Compared with teachers’ demographic features, the school characteristic (e.g., school type, location) played a less important role in their perceiving of less successful PD.

In particular, experienced teachers perceived significantly more having “too low objectives”, “too short duration”, “inappropriate assessment for participants” and “inappropriate management” as problems in the less successful PD activities, and female teachers perceived significantly more having limited and inconvenient ways to attend PD as a problem, while male teachers agreed significantly more that the durations of the PD they attended were too short. It appears clear that those results are related to the fact that not only were there different PD practices but also school teachers had different professional and personal backgrounds and needs.

This result also reminds us of the crucial distinction between pre-service teacher education and in-service PD programs. In China, most of the pre-service teachers are trained in normal universities, where the trainees are homogenous. On the contrary, for in-service PD activities, especially those organized ones that cover a large group of teachers, the trainees are often heterogeneous, and hence they have various backgrounds and needs for PD. This heterogeneity can affect the effectiveness of PD and bring different backgrounds and challenges, implying that, different from pre-service teacher training, the issues about in-service teacher PD are more complex and there is no single solution in pursuing the improvement of PD for in-service teachers. As revealed in this study, many PD practices were not well-designed and not specifically tailored to the needs of teachers with different backgrounds and needs, and hence they were less successful than expected. Therefore, instead of using a uniform PD model, there is a need for the organizers to adopt a differentiated approach to the design and delivery of PD for in-service teachers, tailored to different groups of teachers with various needs. This also accords with the findings by Goldsmith et al. (2014), who identified an alternative by analysing what learning pathways look like for teachers with different belief and knowledge systems, and for various pedagogical practices. This implication can also be generalized beyond the Chinese context, as the heterogeneity of in-service teachers for PD exists globally.

Fourth, the study revealed that Shanghai teachers perceived that PD practices were less successful mainly due to a lack of appropriate evaluation for the PD organizers and necessary coordination between the PD organizers at different levels. It should be noted that this issue has also been identified in other educational settings, for example, Ingvarson et al. (2013) reported that most of the 30 countries they studied lacked specific evaluation systems geared to teacher education institutions or programs. Moreover, in China, appropriate collaboration between various organizers (including TRO, TRG, or university research teams) was found critical to the teachers’ sustained participation and growth, in which the TRO officers (as providers of professional incentives and reassurance of policy-related matters) and school leaders in TRG (as tacit supporters and anticipated organizers for future PD) played different roles in supporting teachers (Taplin et al., 2007). In this regard, more attention should be paid to the coordination in ensuring the coherence of the contents and avoiding unnecessary repetitions or overlaps. Furthermore, our study also revealed that PD offered at a higher level (city/country-level) placed more emphasis on theoretical knowledge without adequate connections to practical teaching that were emphasized in school-level PD. Further research is needed to investigate the internal mechanisms in the teacher professional development system and possible ways to improve such coordination. Based on the result, we would also argue that an evaluation system should be set up for the organizers of PD, and ideally the evaluation be conducted by a third-party agency or the participating teachers.

Finally, we should point out that, given this study was conducted in the Shanghai educational settings, the results obtained about the mathematics teachers’ perceptions of less successfully organized PD practices might not be generalizable to different parts of China or other countries, which is a limitation of our study. In addition, the study focuses on teachers’ perspectives. While teachers’ voices about the quality of PD as they perceived has critical importance and must be heard and studied, their voices and expectations are subject to their professional experiences and needs. In the study, we intended to obtain an overall picture of Shanghai secondary mathematics teachers’ views about the problems or inadequacies and underlying reasons behind the less successful PD as they perceived, which is the first step for us to understand and address the issues about less successful PD practices. In future, more research with a larger scale with a specific focus on city/country -level, district-level, and school-level PD practices, in different social and educational contexts, and from different perspectives (e.g., the PD organizers’ perspectives), is needed to further advance the understanding of related issues and challenges, and hence improve the quality of the PD for teachers.