Introduction

In 1994, Andy Hargreaves published a landmark book, Changing Teachers, Changing Times: Teachers’ Work and Culture in the Postmodern Age. My own copy of this book is tattered, highlighted on almost every page, and dog-eared, as I have returned to the book over and over again since my first reading of it in graduate school. Clearly, Hargreaves has written much since 1994, but this work is particularly rich and insightful in its contributions to our understanding about educational change. When it was published, the volume received accolades from educational researchers, practitioners, and policymakers across the globe, and won the 1995 Outstanding Writing Award from the American Association of Colleges for Teacher Education.

In the book, Hargreaves examined how teachers and teaching had changed (in the late 1980s and early 1990s) and what changes we might expect in an increasingly post-industrial, postmodern world. Hargreaves cautioned readers that as school systems were already showing signs of resisting the postmodern condition in unproductive ways that led to increased bureaucratic control, teachers would be caught in the middle. Much as he was prescient in these statements, I do not think any of us would have predicted the degree of systematization that now characterizes teachers’ work, especially in the US. We have reached a stage of unprecedented federal involvement in K-12 public education via the No Child Left Behind Act, and more recently, the Race to the Top program. These landmark federal legislative efforts have launched us into the age of accountability, where teachers’ work is shaped by ever-increasing pressure to improve student performance on standardized tests.

The age of accountability has shaped the culture of teaching in significant ways; however, much of what Hargreaves taught us in 1994 about the importance of teacher collaboration (and its ugly twin, contrived collegiality) has remained tremendously important in understanding how teachers can work together (or not) to improve student learning. Decades of research have revealed that collaboration and collegiality are essential for school improvement. However, Hargreaves’ research taught us that despite having lofty goals around collaboration, many efforts at productive collaborative work among teachers have gone awry because they underestimate the micropolitics of schools. Hargreaves (1994) explained that, “The micropolitical perspective encourages us to discriminate between the different forms that collaboration and collegiality take” (p. 191). In collaborative cultures, working relationships between teachers tend to be spontaneous, voluntary, development-oriented, pervasive across time and space, and unpredictable. These cultures are incompatible with “school systems where decisions about curriculum and evaluation are highly centralized” (Hargreaves 1994, p. 193). Contrived collegiality is administratively regulated, compulsory, implementation-oriented, fixed in time and space, and predictable. Collaboration among teachers with these characteristics does not generally lead to meaningful or sustainable change.

In this article, I will bring Hargreaves’ ideas about teacher collaboration to bear as I investigate a current educational reform movement: data-driven decision making (DDDM). I wish to note that although I will refer to data-driven decision making, I do not presume that data drive decisions. In fact, it is quite clear that data don’t drive (Dowd 2005), as data do not imply a course of action. Rather, I use the term data-driven decision making to refer to the process by which administrators and teachers collect and analyze data to help inform educational decisions (Copland et al. 2009; Ikemoto and Marsh 2007). It is worth noting that the use of data to inform instructional decisions is not new. Teachers have always looked at student assignments and test results to make judgments about how much their students have learned. What is new, however, is that the use of data is now inextricably connected with accountability for the improvement of student achievement (Mandinach and Honey 2008).

The research questions guiding this paper are: What is the role of collaboration in schools implementing data-driven decision making? What evidence do we see of collaborative cultures versus contrived collegiality?

Methods

The qualitative data for this paper were gathered as part of a case study of urban schools across the US that exhibited positive outcomes related to their implementation of data-driven decision making (for a full description, see Datnow et al. 2007, 2008). Purposive sampling was used to identify and select the research sites (Yin 2003) since we were interested in studying how school systems implemented DDDM practices. Sites were chosen on the basis of their status as leaders in DDDM and for their record of improving student achievement over time. In order to be selected, schools also had to serve student populations that were diverse in terms of race/ethnicity and socioeconomic status. Selected sites were chosen from a list of over 75 sites that had been recommended as fitting the criteria. The research team narrowed down the list of possible sites after reviewing district and school sites, speaking with experts in the field, and conducting phone interviews with district leaders. While acknowledging the successes they had experienced in becoming more data driven, all leaders also were careful to note that their work was “in progress.”

Although there were 12 sites in the larger study, for the purposes of this paper, I have chosen to highlight 2 districts and the schools we studied within each of them. Table 1 provides an overview of the sites. One district was in California, and the other was in Texas. These two districts were purposefully chosen for this paper, as they shared a great deal of commonalities in their approaches to data use and in their general characteristics (e.g., leadership stability, history and contexts for reform, size, urbanicity, etc.). They are not contrasting cases.

Table 1 Characteristics of school sites
Full size table

The research team’s site visits to the school systems and schools took place between March 2006 and May 2008. In each site, the team interviewed two to three administrators from the central office, two to three administrators at each school site, and teachers across a mix of grade levels and academic disciplines (e.g., science, English). We asked principals to send our invitations for interviews to all teachers and noted that we wished to have a range of teachers in terms of experience level, grade, subject area (at the secondary level), and involvement with data use. Based on the data we collected, we believe we were able to gather a range of teachers with a diversity of opinions. All of the leaders we interviewed were chosen on the basis of their institutional position (e.g., principal, superintendent) and/or their involvement in data use efforts (e.g., a Title I coordinator who was engaged in helping teachers use data). It is worth nothing that there was considerable stability in the teaching corps as well as in the leadership at both the district and school levels in the sites we studied.

The team conducted interviews with approximately 50 individuals total in the two locations. In addition, the team observed in classrooms, conducted focus groups, and attended key teacher development workshops in order to collect data to triangulate findings. Finally, the team gathered a plethora of documents at the school and system levels that were pertinent to our study.

All of the interviews were recorded and transcribed verbatim. Field notes of classroom and meeting observations were also typed. Subsequently, qualitative data were analyzed using a case report outline based on a conceptual framework developed for the study. Next, case reports were produced for each school site to aid in cross-site analysis. The detailed case reports included representative quotes from the participants in our study. Many of these quotes were chosen for inclusion in this paper. In constructing the narrative of our findings, we attempted to use quotes that represented typical teacher perspectives on DDDM. However, we were careful to include dissenting voices—quotes from teachers that departed from the typical response—so that we could expose some of the tensions and inconsistencies that existed as teachers’ attempted to engage in collaborative work around data use.

Findings

The context for data use

Before delving into the role of teacher collaboration in the schools implementing DDDM, it is important to understand the broader context of the two districts. The teachers in this study were working in schools and districts where there was overwhelming consensus about the importance of using data to improve teacher performance and student achievement. School and district leaders in Texas and California helped teachers make sense of data by defining the purpose of data use, stressing the importance of using evidence, and emphasizing improvement efforts. Leaders directly tied the use of data to improving teaching and learning.

For example, the district administrators in the California site believed strongly in data-informed decision making, but made a point to say how important it was to use data properly and thoughtfully. Reflecting on their use of data to improve student placement and scheduling, the superintendent said, “I mean [in the past], we were using data and thought we were doing the right thing, but were really aligned to the wrong things.” She added, “So that’s the big message in data: It’s only good…. if you think real carefully about what you are trying to achieve.”

Paralleling the district’s message about using data thoughtfully, the principal of a high school in California worked hard to support the culture of DDDM. One teacher told us that before she was hired, the principal made it clear that DDDM was a focus at the school, explaining, “It was really something that [the principal] wanted to push. …It’s just assumed that you will use the data to help drive what you need to do and where you need to go.” The principal was credited by a teacher as “presenting [DDDM] in a positive light ….I mean she’s offered opportunities that if you struggle, here’s your safety net. If you’ve failed, try again. She’s done it in a very non-judgmental way and let people get to their levels.”

In addition to establishing a culture for data use, districts devoted attention, time, and resources to building structures of support for teachers. Common across both districts were data from systemwide interim assessments aligned to state curriculum standards. Schools were required to administer these benchmark assessments, and teachers were required to discuss the results from them. However, leaders also encouraged teachers to use a range of assessments (e.g., homework, quizzes, etc.) of student learning for formative purposes in planning their instruction. Both systems also invested in web-based information systems to store data that could be accessed by teachers. Such systems enabled users to access student achievement and demographic data in a timely manner. Both districts utilized staff at the system and school site levels to aid teachers in managing and using data. In spite of these supports, some teachers still faced barriers related to the technology of the information management systems or in making sense of the reports they generated. As one teacher said, “If you’re having a hard time just logging on, that can be overwhelming!”

The role of collaboration

Providing structured time for collaboration was one of the supports that both districts thought was essential for data-driven decision making to take place among teachers. The California district established weekly “data-centered discussions” among faculty. The administrators considered these instructional planning meetings to be “sacred.” The district provided collaboration time every 2 weeks on Wednesdays from 7:30–9:00 a.m. Students arrived at school at 9:30 a.m. allowing the teachers 30 min from the end of the meeting to the beginning of instruction. Building in this transition time was seen as important by school administrators who didn’t want teachers to feel rushed and thus tempted to use data discussion time for class preparation. Similarly, in Texas, grade-level teams gathered every 2 weeks on Wednesday afternoons to discuss data in a meeting facilitated by the grade-level lead teacher.

In both California and Texas, principals also set expectations for how meetings regarding data would be conducted. They took time to cover issues such as how to behave in meetings, what materials teachers and principals were expected to bring to meetings, what not to bring (e.g., papers to grade), and how to compile data binders. While these expectations may seem contrived or intrusive, educators indicated that these discussions helped set the tone for accountability among the staff members and ensured that meetings were purposeful. Several teachers also stressed the importance of creating norms and rules for discussions about students so that the level and type of discussion would not deteriorate into inappropriate “nit-picking or trash-talking.” When asked about how the school established such expectations for teachers, several teachers indicated that positive “peer pressure” was important and that productive facilitation of discussions to keep conversations on track was necessary. Therefore, in conjunction with gaining buy-in from staff members, many schools strategically attempted to nurture high expectations and mutual accountability among the staff.

One principal remarked that the key to making data relevant is the working relationships developed between staff because “without collaboration and the collegiality, data use is impossible.” She also acknowledged that staff members have “lively discussions.” She recalled one incident where teachers in one department argued that assessing students every week was too much. She acknowledged that “we fought, we fought, we fought” but eventually resolved the issue through multiple discussions about the relevancy of data and the need for frequent testing. Another principal described their staff as a family and explained that like all families, they should disagree and argue. She also promoted the slogan, “Whatever happens in your meeting stays in your meeting.” Her philosophy is that disagreements should occur because “if you don’t disagree, then there’s something wrong because you’ve got 10 different personalities.”

Teachers met in grade-level (elementary) or course-alike groups for these data discussions (e.g., all teachers of English 9 met together). Both districts developed data discussion protocols in order to ensure that discussions about classroom-level data occurred and that actions were taken on the basis of these conversations. Teachers were asked to prepare ahead of time by filling out data summary sheets. They were also required to bring in an assessment (e.g., pre- and post-test, benchmark, or unit test) and they typically shared what worked well, areas of struggles, and their action plans.

As one teacher explained, during their data discussions, they addressed questions including: “What did the majority of students do well on? Do you have an explanation for that? What strategies did you use? What level of Bloom’s Taxonomy [was that]?” The teachers also carefully examined the assessment itself asking questions such as, “How was this question worded? Do these kids know what the words in the question mean?” This was important, the teachers believed, to understanding the causes of students’ struggles with the assessment, such as poor reading skills or the limited English proficiency. During the team meetings, they sometimes shared class report graphs or an item analysis graph. Lastly, the team came to a consensus about actions to take or strategies for improvement.

The data discussion meetings were intended to be used for examining data from benchmark or quarterly assessments, rewriting test items, and ultimately, joint instructional planning. One teacher explicitly stated that the expected outcome of the data discussion was “modification of instruction.” “I love our collaboration time,” said one English teacher, noting that “we’ve hit upon the right amount of time” for collaboration. One teacher described an informal meeting during lunch time when she compared the progress of students with another teacher. She jokingly added, “There’s a tremendous amount of collaboration, even when we don’t feel like it.” A teacher at another school also believed that collaboration is key because “data is only good as you understand it and you analyze it, and then it’s only good as you do something about it.”

One teacher remarked that while looking at “the numbers” did not inspire her, as she already knew how her students were doing, the opportunity to compare results with her colleagues and share instructional strategies generated through the data discussions were very useful. She wished for more collaboration time so that teachers could share lesson ideas:

We want to get into sharing lessons, sharing great ideas that the kids got, what the kids understood. … [For example, a teacher might say], “Hey, I did this really cool lesson on character. Try it! [Another teacher might say], “Hey I did this really great lesson on ambiguity. Try it.” [And then someone else might say], “Oh cool, you know my kids have already struggled with that. I’d love to try it.” Who wouldn’t want new strategies, new ideas?

She noted that the English department grade group that she is a part of tended to focus less on the numbers and more on instructional strategies compared to their colleagues in other groups or departments:

We go through the data and some groups I’m sure spend the entire hour and a half on just the numbers, but we don’t want that. We want to truly know how did you teach it? What did you do differently that I didn’t do?

She added that teachers were very comfortable sharing their results and their lessons. “Seeing someone that can actually find a solution” really helps, explained one teacher. One science teacher said that she and the teacher next door collaborate continuously: “Our door is open between every period, all the time. We’re constantly going back and forth.”

Many teachers described productively working together on a frequent basis. However, in other cases, the administrative regulation of meetings led to teachers focusing on the tasks (e.g., completing a form describing the outcome of their discussions of data) rather than having more meaningful discussions around data. While the majority of the educators across the systems seemed to embrace the use of data, district leaders noted that there were pockets of resistance among some teachers. Lack of buy-in was attributed to a large wave of reforms and programs implemented all at once. One assistant principal in Texas shared that, although all the district-driven changes were “excellent, right now, what we have to do is to not have any more changes. Let us get better at what we’re doing now.” A principal in another school agreed that some teachers were still uncomfortable with the district’s requirement of data discussions and said that some “people felt it was somewhat of a mandate and a little bit paper-heavy.” District leaders in both school systems recognized this problem. As a district leader in California stated, they attempted to “build the thirst for [data]” among teachers, but this was a process which took time.

Teachers also mentioned that it was sometimes challenging to decide when to re-teach, how to re-teach, and whom to re-teach, especially given the constraints of both districts’ pacing plans. Some also admitted that they lacked the skills to create action plans based on data. One teacher stated, “Don’t just throw the data out there and expect the teachers to be able to pick it up and run with it.” Another teacher asked, “How am I supposed to teach differently?” Although the use of data could pinpoint areas for improvement and areas of strength, data alone could not help improve student learning. In the best functioning data groups, teachers could learn from their colleagues, but others hoped for more professional development at the district or school level.

One of the ways in which school leaders attempted to nurture trust was to depersonalize the process of looking at data. A principal shared that when a teacher expresses sentiments such as, “This is so depressing. I worked so hard, and these are my scores,” she responds with, “Don’t go there. Don’t look at it that way. What we need to do then is to say, okay, what can we do differently next time?” Instead of viewing data as highlighting what “I can’t do,” they now begin to see it as “what I need help on.” One teacher compared their situation to the dentist. She stated, “[They] do their results, and you know, you don’t hear them say, oh you know, [a patient] has a cavity, and I filled three last year…oh, I’m not a good dentist.” In one school, a teacher noted, “The teachers don’t take it personally; I failed. I didn’t do… It’s like okay, this is what the data says; how can we do something to make the kids perform better? There’s not that negative.” Nevertheless, developing trust amongst some groups of teachers was still a struggle. A principal acknowledged, “You have to take it step-by-step because if you don’t, you can send people over the edge, in terms of their motivation, of their self-confidence and burn them out.”

As much as teachers and principals emphasized viewing data objectively, several teachers admitted that it was difficult not to take data personally because they were very invested in their students. A teacher pointed out that an additional problem was that “[data] don’t always tell the full truth of what’s going on because data can be misinterpreted.” She was skeptical about some of the data that teachers presented during meetings. For example, one teacher removed some test questions from her analysis because the material was not covered in her class, whereas others included those same questions. She shared that then, “We get disappointed because you know, I got 40% and she got 80%.” She stressed that honesty was important if data use was going to be an effective tool to improve teaching and learning.

Collaborative cultures or contrived collegiality?

Earlier in this paper, I presented Hargreaves’ (1994) definitions of collaborative cultures and contrived collegiality. He elaborated on his definition of a collaborative culture when he outlined the following characteristics:

Collaborative cultures

  • May be administratively supported and facilitated by helpful scheduling arrangements, but ultimately must be sustained by the teaching community.

  • Do not arise from compulsion but from their perceived value among teachers and a belief that working together is productive and enjoyable.

  • Teachers establish the tasks and the purposes for working together, rather than implementing the purposes of others.

  • May be characterized by scheduled meetings, but such sessions do not dominate the arrangements for working together.

  • Outcomes of collaboration are uncertain and unpredictable. (Hargreaves 1994, pp. 192–193)

Hargreaves explained that in their most robust form, collaborative cultures extend to joint work, mutual observation, and focused, reflective inquiry. In these cases, teachers interact knowledgeably and assertively with each other, rather than simply being congenial and complacent.

By contrast, contrived collegiality has the following features:

Contrived collegiality

  • Working together does not evolve spontaneously but results from administrative regulation.

  • Teachers are required to work together to meet the mandates of others.

  • Takes place at particular times in particular places.

  • Control over purposes and regulation of time are designed to produce highly predictable outcomes. (Hargreaves 1994, pp. 195–196)

In other words, contrived collegiality “replaces spontaneous, unpredictable, and difficult-to-control forms of teacher-generated collaboration with forms of collaboration that are captured, contained, and contrived by administrators instead” (Hargreaves 1994, p. 196).

So what might Hargreaves say about the qualitative data presented earlier? In examining these definitions alongside the data presented in this paper, a complex picture emerges. On the one hand, the form of collaboration promoted by both districts was administratively regulated and designed to meet the purposes set forth by school and district leaders. Not only was time set aside for collaboration, but the collaborative activities were scaffolded by meeting protocols that prompted teachers to address a set of prescribed questions as they engaged in the analysis of student achievement data. In these respects, one might say that the examples we observed were perfect examples of contrived collegiality. However, in these sites, we didn’t see many of the negative effects of contrived collegiality taking hold. Rather, for many teachers, what began as contrived meetings to discuss data evolved into spaces for more genuine collaborative activity wherein teachers challenged each other, raised questions, and shared ideas for teaching.

How did this occur? Let us return to the context in which data-driven decision making was implemented. The cultures and structures of these schools and districts supported this reform in almost every way, shape, and form. Leaders took great care to develop trust among teachers, to assuage their concerns about how the data reflected upon them as individual teachers, and to promote a positive orientation towards data use. Most importantly, these schools and districts had high levels of capacity for change. They were chosen for the study on the basis of being leaders in data use and in improving student achievement over time. They benefitted from strong and stable leadership at all levels. They were places where, by and large, positive relationships and practices were already in place before data use was introduced. They were already focused on continuous improvement, even before data use became a priority. All of these factors made these sites unique.

Thus, these findings suggest that good things might happen when teachers collaborate around data use in schools and districts with high capacity. They do not inform us about what might happen in schools and districts with lower capacity. Hargreaves (1994) may predict that contrived collegiality would be more likely to result in schools where there is distrust among teachers and between administrators and the teaching staff. Such schools often have tremendous need for building instructional and organizational capacity, a problem which would not be solved simply by introducing data use (Hatch 2011).

The past decade of research on school change has made it clear that capacity-building approaches need to be differentiated, and a one-size-fits-all approach certainly does not work (Stoll 2009). Across the globe, we have seen various combinations of capacity-building activities, some which involve a great degree of administrative regulation and control, and others which have much less and/or combine that with professional involvement on the part of teachers (Fullan 2009). When it comes to data use, it is clear that teachers do need supportive structures and cultures in which to collaborate, but also enough autonomy to be able to make decisions and take local action on the basis of their analyses of information about student learning. Little and Curry’s (2009) recent study of conversation among secondary teachers in a critical friends group exemplified one aspect of this delicate balance. They found that having discussions guided by specifically designed protocols both promote—and limit—teachers’ opportunities to learn. If they are formulaic, they limit the spontaneous learning and unexpected positive outcomes that Hargreaves observed in his early work on collaborative cultures. After all, collaborative cultures need to allow teachers and their peers the space to develop new capacities and competencies (Lieberman and Miller 2008). Discussions about data may help form the basis for teachers to share their practices with colleagues, but this is not likely to occur if such conversations are overly prescriptive and constrained.

Conclusion

The findings in this paper raise questions for the field as we consider the impact of recent educational reform movements on the professional lives of teachers. It is clear that sharing and analyzing more and more information on students takes additional teacher time, even if it is likely to inform their practice. In her study of a district that is working on increasing information sharing about youth among various stakeholders, Pollack (2011) explains that the density of information flow can become overwhelming. We know that increased, regular communication among teachers and between teachers and students is critical to improving students’ achievement, but it does add to teachers’ work load. Teaching is much likely to be better informed by students’ learning needs, but using data to guide instruction is time intensive work.

When we ask teachers to collaborate around data use, we also need to consider several other implications of this study’s findings in connection to Hargreaves’ work. First, leaders need to ensure that teachers have the flexibility to actually make changes in classroom instruction based on data. As the teachers in this study noted, it is all well and good to get teachers together to create action plans, but their efforts will be thwarted if the district pacing plan does not allow them the space or time to teach in new ways. Teaching in new ways may very well require curricular changes, the need for new materials, and changes in student grouping plans. Moreover, leaders should encourage teacher feedback on the usefulness of the assessments that are being administered. Most importantly, leaders should allow teachers to educate them on what information about student learning appears most useful in guiding improvements in teaching and learning. As Hatch (2011) aptly notes, data don’t make decisions, people do, and teachers’ own knowledge, experience, and intuition are important ingredients of data discussions, as well as more seemingly “objective” forms of information.

Finally, as we consider the future of teacher collaboration as part of data-informed school improvement, we need to continually evaluate collaborative efforts lest they move to the dark side of contrived collegiality. Hargreaves (1994) stated, “Collaboration and restructuring can be helpful or harmful, and their meanings and realizations therefore need to be inspected repeatedly to ensure that their educational and social benefits are positive” (p. 248). Thank you, my good friend Andy, for this critically important reminder.