1 Introduction

Assessment and the use of data are recognized as critical capabilities for teachers as highlighted in the Teacher Education Ministerial Advisory Group (2014), and other international reviews in Scotland and the United Kingdom (Scottish Parliamentary Corporate Body, 2019; The Department for Education, 2019). The Australian Professional Standards for Teachers have specified the need for teachers at graduate level to demonstrate their capacity to interpret student and assessment data and utilize this information to “evaluate student learning and modify their teaching practice” (Australian Institute for Teaching and School Leadership, 2020, unpaginated). To date, there is limited research beyond small scale studies regarding how teachers use large-scale assessment data, both alone and in conjunction with classroom assessment evidence that they collect, to improve student learning and inform teaching practices (Comber, 2012; Author, 2019). The need for greater expertise in analysing data to support student improvement has fast become a necessary condition, presenting a critical need for building this capability in the teaching profession (Gonski et al., 2018; Goss et al., 2015).

The research question, what is the utility of NAPLAN data for teachers and members of the school leadership team in informing teaching and improving learning? is ultimately concerned with exploring whether, and how, teachers integrate national test data with their ongoing classroom teaching and assessment to inform instruction for all students. The questions bring to the surface situated perspectives evident in school leaderFootnote 1 and teacher accounts to examine whether there are consistencies or differences within and across the communities of practice, defined as “an activity system about which participants share understandings concerning what they are doing and what that means in their lives and for their communities” (Lave & Wenger, 1991, p. 98) and how their views and experiences are placed as part of their wider lived policy experience of NAPLAN. The question will also explore how school leaders and teachers access NAPLAN data and the value that is attached to data for informing ‘next-step teaching’. Next-step teaching is defined as practitioners having clarity around the direction in which learning is headed, based on the engagement with data for individual students or whole class learning.

The issue of who uses data and how it was used is discussed in Selwyn, Henderson, and Chao’s research (2015). They highlighted the need to “develop better understandings of how digital data is implicated in the shaping of what people can and cannot do, in the shaping of social opportunities and social constraints—in short, in the operation of power” (p. 768) and questioned whether data use altered the social relations within schools in terms of control or conditions of performativity. This paper takes on board these concerns focusing on exploring what are the opportunities for and barriers to NAPLAN data use, presenting for the first time both school leader and teacher accounts of the utility of NAPLAN data. The paper examines both similarities and differences of school leader and teacher experiences and considers pathways for greater utilization of performance data.

This study adopts a sociocultural view of learning exploring how teachers’ historical, social, and cultural contexts influence their values, beliefs, understanding and sense-making of the world and their work. This will be explored as it relates to teachers and school leaders experience of engaging with NAPLAN data and their identity as part of their lived experiences of NAPLAN (Murphy & Hall, 2008). It draws on a conceptualisation of assessment as a social practice (Broadfoot & Black, 2004) which “mediates human relations with the world and with others” and understands that assessment addresses “a societal need … in broader systems of relations and social structures in which they have meaning” (Elwood & Murphy, 2015, p. 183). This conceptualisation of assessment is complementary with Wenger’s (1998) notion of the social theory of learning and nature of knowledge as a shared enterprise in a community of practice.

The construction of this theoretical framework serves the study’s focus for exploring school leaders’ and teachers’ legitimate peripheral participation defined as a multidimensional but interconnected system that looks at how learning occurs as one engages in the social practices of a community (Lave & Wenger, 1991). Through this socio-cultural lens, examination of leaders and teachers reported accounts of their experiences of using NAPLAN data is vital to understanding the barriers and enablers that inform teaching and student learning.

2 Background

2.1 The utility of NAPLAN data

On 8 December 2008, the Australian Curriculum, Assessment and Reporting Authority (ACARA) was established under the Australian Curriculum, Assessment and Reporting Authority Act (Cth) (ACARA, 2018). ACARA’s functions included the “development of national curriculum, administration of national assessments and associated reporting on schooling in Australia” (ACARA, 2018), inclusive of the management and reporting of NAPLAN. The introduction of national testing in Australia was set against a larger international backdrop of the ‘Global Education Reform Movement’ which Sahlberg (2011) discussed as “the standardization of education”. Currently, standardized testing and reporting of data have become critical for informing governments, education policy and decision making.

The reporting of NAPLAN data as intended from the Melbourne Declaration (MCEETYA, 2008), was to provide schools with “rich data on the performance of their students because they have the primary accountability for improving student outcomes” (MCEETYA, 2008, p. 16). This commitment to data use continues to be referenced in the most recent Alice Springs (Mparntwe) Declaration (Education Council, 2019) with the assurance of continued provision of “good data” to inform “educational experience and outcomes” (p.11). This perspective is reiterated in the 2008 NAPLAN National Report (ACARA, 2008) which states that NAPLAN is a diagnostic tool to be used for diagnostic purposes and teachers can use the data “to gauge the achievement of the most able students, as well as focus on students who have yet to reach the national minimum standard and who may need further support” (ACARA, 2008, p. 2). More recent infographics have also highlighted NAPLAN data is to be used by teachers and schools to “identify any areas for improvement and track progress from previous years” and to “use data to better understand knowledge and skills, abilities and achievements of students in a school” (ACARA, 2022, unpaginated).

If teachers and students are to benefit from NAPLAN data, some researchers assert they need to have access and ownership of test data to understand how to use it. A substantial literature review by Lobascher (2011) drew on research primarily from the US and the UK and concluded, if literacy education in schooling is to benefit from NAPLAN, “teachers must be involved with the process of designing, implementing and evaluating the tests, as well as supporting the distribution and application of test data” (Lobascher, 2011, p. 18). He asserted, for teachers to be productive in this process, it is critical they engage in “professional development in designing and evaluating assessment” (p. 18).

One of the necessary conditions for teachers and members of the school leadership team to utilize NAPLAN data in order to inform teaching and improve learning would be the opportunity for professional development in data use as part of teaching practice. Attention needs to be directed to improving teachers’ data literacy, defined as the use of “data with standards, disciplinary knowledge and practice, curricular knowledge, pedagogical content knowledge, and an understanding of how children learn” (Mandinach et al., 2015, p. 3).

An examination into the diagnostic potential of NAPLAN for teachers was initiated by the Senate Education, Employment and Workplace Relations References Committees Inquiry into the Effectiveness of the National Assessment Program – Literacy and Numeracy in 2013. One of the submissions from the Whitlam Institute (Dulfer et al., 2012) discussed a survey which included questions asking whether NAPLAN was a diagnostic tool for teachers. Fifty-eight percent of teachers believed “NAPLAN was not a diagnostic tool, while two thirds of principals believed it was” (Senate Committee, 2013, Submission 2.24, 2.25, unpaginated). The report indicated that one of the reasons for the differing perceptions of the use of data could be the differing view of how it is used. Teachers tended to focus on individual students whereas principals were looking at the overall performance of the school.

The Committee noted, “Teachers and student teachers do not receive sufficient training or support to enable them to properly use or analyse data obtained by NAPLAN testing” (Senate Committee 2013: Submission 2.28, unpaginated). It was also recommended in their 2013 report, Teaching and Learning – maximizing our investment in Australian schools, the need for greater support for teachers in the use of evaluative data. All Australian states use differing software analytical platforms for disseminating NAPLAN results, and from the report it was “clear that more work could be done to support teachers in becoming skilled at interpreting and using NAPLAN data” (Senate Committee 2013: Submission 2.28, unpaginated). ACARA has also reported the need for professional development opportunities in interpreting and using data for pedagogical improvement. Professional development is critical if teachers are to feel confident within themselves and less reliant on others to inform their understanding of data, notwithstanding the fact that interpreting student data are a key professional capability outlined in Standard 5.4 of the Professional Standards for Teachers (AITSL, 2020).

A review commissioned by the Education Council into NAPLAN and My School, found that all contributing sample schools “demonstrated a deep understanding of their students’ achievement data and actively used it for school improvement purposes” however it was noted, “NAPLAN was rarely used to inform day-to-day teaching practice” (Louden, 2019, p. 89). More recently a NAPLAN review has been commissioned by some state and territory governments (Australian Capital Territory (ACT), Queensland, New South Wales (NSW), and Victoria) to consider the extent to which NAPLAN has met its original objectives, with one of the terms of reference to consider “the effectiveness in tracking student and system progress over time” (McGaw et al., 2020, p.1).

This NAPLAN review found little consensus among stakeholders about the appropriateness of school systems’ and sectors’ use of NAPLAN data. School system and sector representatives couched the value of NAPLAN in the context of managing a large-scale system and providing the conduit for initiating conversations with schools about performance (McGaw et al., 2020). Teachers on the other hand indicated a need for greater access “to detailed individual reports on students’ achievement” (p.109). The survey conducted as part of the review also indicated a range of experiences with the data, some stakeholders reporting high levels of teachers’ use of this information while other indicated “teachers do not engage with or talk about NAPLAN data” (p. 109). Despite teachers’ and principals’ reservations about NAPLAN, the review found NAPLAN data was “routinely used by schools in local planning and monitoring of school-level progress” (p.108).

Professional identity and data use.

How teachers see themselves as participants in a community of practice is connected to their identity as a practitioner. Wenger (1998) suggested that ignoring issues associated with identity in a community serves only those whose identity is already established. A community of practice needs those who have a “claim to ownership of meaning” (p. 269–270) to abandon this claim to knowledge to include those who do not have this knowledge. Inclusion as part of analysing and sharing data for all members of a school community is a critical learning experience, imperative for the development of all participants to ensure all members have opportunities for meaningful membership and open acquisition of knowledge.

For NAPLAN data to be utilized by teachers and school leaders it firstly needs to be valued by the community. Enabling discussions of liberating NAPLAN data from the judgement of teacher performance and instead acknowledging the data as an opportunity for targeting improvement of student skills in school communities is critical for increasing utilization. Hardy and Lewis (2016) found that data use, when viewed as a reflective sharing and knowledge building exercise, has the potential to be a positive experience. However, if positioned in the context of accountability, narrowly understood as measurement for comparative purposes, values can shift to data for validating personal worth or as a measure of success or failure as a teacher. How schools facilitate data dissemination and analysis from tests such as NAPLAN as a community can be critical in shaping teachers’ identity as participants in a community or, alternatively, feeling the need to “validate one’s worth as a teacher” (p. 6). A professional community needs to establish a “climate of trust” (Datnow & Hubbard, 2016, p. 23) in order to create a cycle of inquiry, rather than “simply being about accountability” (p. 23).

Pertinent, professional development designed to improve data literacy is critical to engaging teachers’ use of NAPLAN data. Datnow and Hubbard (2016) found that the relevancy of professional development in building teacher capability in data literacy was often “limited to information on how to access a data management system” (p. 23) rather than training that could offer teachers “more fine-grained information about student achievement that will allow teachers to address students’ individual needs” (p. 23).

Data literacy is a skill that is essential for both school leaders and teachers. How data can be analysed and used as part of a cycle of inquiry is an essential skill for all members of a school community to ensure transparency and open, knowledgeable discussions. A burgeoning issue reported by Selwyn (2015) is the challenge of “intensifications of inequalities of power and control arising from the generation, processing and circulation of digital data” (p. 71). Selwyn suggests digital data is creating sociological differences evidenced through unequal agency. He states there is a growing divide between social groups dependent on data literacy capabilities, or more simply termed, the difference between those who have “data ‘done to them’, as opposed to those who have the ability to ‘do data’” (p. 71). This suggests a need to take account of the changing nature of professional identities connected to data literacy capabilities and highlights the importance of creating opportunities to become data literate to ensure the open acquisition of knowledge for all members of a community of practice.

This paper will examine the utility of NAPLAN data and reported barriers to data use, specifically looking at what is currently known about how school leaders and teachers use NAPLAN data in their planning, teaching and opportunities for intervention in the following section.

3 Methodology

The data for this study are drawn from an Australian Research Council (ARC) Discovery Project that sought to investigate school and teacher use of NAPLAN data for student learning improvement and, under what conditions this occurred. This paper pays close attention to the accounts of school leaders and classroom teachers in NSW and Queensland schools as participants in a community of practice, drawing specifically on membership group views.

3.1 Participant selection

Letters of invitation were sent to education sector authorities, including state, catholic and independent schooling to advise of the study’s aims and participant involvement. Interviews were undertaken in nine case study schools in Queensland (n = 7) and NSW (n = 2). These represent a range of schools (size, sector, primary/secondary, classroom organization, student characteristics), and geographic (metropolitan, rural) and socio-economic factors. Six schools were primary (PY-7), one was secondary (8–12) and two provided all years of schooling (PY-12). Three schools were state, four were independent and two were Catholic. In total the study draws on 68 interview transcripts for school leaders (n = 21) and teachers (n = 47). The gender ratio of the sample was as follows: overall (males = 14, females = 54); school leaders (males = 6, females = 15); teachers (males = 8, females = 39).

3.2 Data analysis

Interview transcripts were analysed using a process for coding qualitative information called thematic analysis. The purpose of thematic analysis is to find “broad units of information that consist of several codes aggregated to form a common idea” (Creswell, 2013, p. 186) of significance across a suite of data (in this case, school leader and teacher accounts) that may connect to or support clarification for the research question being addressed. This inductive approach to analysing data allows the jigsaw pieces to be identified through a rigorous process of preparing and organizing, coding and condensing the codes, developing the themes that come from the suite of data followed by revision, evaluation and representation of data in figures, tables or discussions (Creswell, 2013).

To ensure reliability in the identification of the codes and themes, data were re-explored and organized, coded and analysed using NVivo software. The software supports the management of complex collections of data and the development of ideas and theories through annotations and memos. Coding in NVivo can occur at multiple levels from the whole document to a single word with different layers of coding applied to the same text. This software enables data to be organized in different groupings while maintaining control of the analytic process.

3.3 Findings

The talk from both school leaders and teachers provided an opportunity to look at the socio-cultural environment of the school and provided insights into individuals constructed social worlds in patterns of social organization (Silverman, 1993). The accounts evoked an exploration into how school leaders and teachers looked at the significance and nature of educational practices in the context of access to NAPLAN data and found commonalities within the two groups regarding some sub-themes. However, there was often disagreement across the two groups in relation to opportunities to view data. The collation of sources, percentages and frequency of school leader and teacher accounts can be seen in Table 1 below.

Table 1 Frequencies and Percentages of School Leaders’ and Teachers’ Accounts
Full size table

4 Access to data: A comparison of school leaders’ and teachers’ accounts

4.1 Open access to data

Discussions around the first sub-theme of “Open access to data” had collective agreement from both teachers and school leaders around examination of data at a faculty or individual level; that is the pathway of school leaders analysing NAPLAN data before presenting or passing it to teaching staff. However, discussions relating to all teachers (whole school) having access to data presented differing accounts, with school leaders suggesting there was greater access to NAPLAN data compared with the comments from teachers.

When reflecting on whole school access to NAPLAN data, 48% of school leaders indicated all teaching staff could access NAPLAN data, while only 9 % of teachers indicated the whole school could access NAPLAN data. While nearly half of the school leaders reported data were accessible to all teachers, there were numerous comments relating to whether teachers needed NAPLAN data or indeed whether it was accessed.

I think that the ownership of data is, is everyone’s. It’s not just that year level so if there’s a trend over time you need to look at that and say well okay, why and question the data.

Anyway, but that was originally how we used it, but now it’s much more concentrated, so every teacher has a password to access the data. Now how much they actually do and do not access it. I cannot actually tell you, although I know that we have had professional learning meetings where they are actually shown how to use it, given their passwords, and then actually encouraged to look at it in that particular medium with their grade partners.

Deputy Principal, Catholic Primary P-6.

Those school leaders who reported whole school access to data saw teacher access as a normal operational process for their schools. The extent of engagement from those teachers with NAPLAN data for their own purposes or strategic goal setting for next-step teaching was not readily quantifiable. While some schools offered access and professional development to teachers, this did not necessarily assure the analysis of their students’ data, with minimal references to how access and professional development played out as part of next-step teaching and student improvement.

Looking to the teachers’ accounts, the comment below demonstrates NAPLAN data are accessible, and the teacher has the data literacy capability to analyse the information for next-step teaching.

Yeah, we all get, well it gets sent, a copy gets sent to the parents, and a copy gets sent to the school. So, what I do is I get the NAPLAN results back, and I analyse them and then I put them into a graph. And I work out exactly where my kids are and who really, it is useful. It’s confirmation and I look at it as a whole. As I said, I get all the NAPLAN tests out and we have to put it together and do an analysis and hand it in to the principal.

Year 3/4 Classroom Teacher, Independent Primary School P-6.

The comment highlights that the principal values the staff and their ability to use data as part of their professional practice and the emphasis the teacher places on analysing NAPLAN data as part of a cycle on inquiry. Of interest is the operational process within this community of practice. The teacher reports “we have to put it together and do an analysis” before they have to “hand it in” to the principal, as a student would, to have their work checked.

Those teachers who reported whole school access revealed a positive engagement with data reporting opportunities for engagement with data beyond staff meetings in a reflective, diagnostic context. A whole-school approach is reiterated in the teacher’s comment below, reporting that school leaders demonstrate how to analyse the NAPLAN data and how to “read the progression between Years 3 to 5”.

We had a staff meeting on it last year, after the results came back where the Leader of Pedagogy and the principal presented the staff meeting on the data on how to read it, on how to read the progression between Year 3 to 5, et cetera, et cetera. And of course, the request is that we would use the data individually if you were in especially a Year 5–6 class.

Year 3 Classroom Teacher, Catholic Primary School P-6.

The teachers’ involvement with data and their engagement with its potential in terms of analysis speak to their active participation, demonstrating their contribution to meetings with their new knowledge as participants of the community (Wenger, 1998).

Restricted access to data.

Thirty-eight percent of school leaders reported they were the only ones who routinely used NAPLAN data, with some stating it was not their expectation that teachers use it, suggesting it was not “the be all or end all of their worlds”. This statement was reinforced by 30% of the teachers who indicated they did not in fact use the NAPLAN data as part of their teaching and learning. Not all comments from school leaders in this sub-theme indicated teachers were deliberately denied access to the NAPLAN data however, a dominant viewpoint was the lack of expectation for teachers to use data beyond the school leaders’ initial analysis.

A pathway for NAPLAN data analysis and dissemination that resides with school leaders was explained from the head of middle school at an independent girls’ school. The comment reiterates that teachers “individually don’t work with the data” and while there is a suggestion that data are not given to the teachers, there is indication that teachers are welcome to it, though the accompanying suggestion is there is no reason for them to engage with NAPLAN data after they have been “provided with an overview”.

The teachers individually do not work with the data. Again, they are provided with an overview and with, say for example, the situation with numeracy, areas that NAPLAN have highlighted as a school that we need to probably spend more time and relook at our programs to make them produce better outcomes. But we do not give the data, I mean the teachers are welcome to it, but there’s no, occasionally you get the English teachers particularly, even though it’s not just English of course, asking for a copy of their NAPLAN writing task.

Head of Middle School Principal, P-12 Independent Girls’ School.

This comment suggests that the analysis conducted by the school leaders provides sufficient information for the teachers. There is no joint experience of learning for the teachers with the leaders; the suggestion is that there is no real reason for the teachers to have access to data as the information is analysed and provided for them. Their role is clear: to look at their programs to “produce better outcomes”.

School leaders only use the data.

The third data dissemination pathway identified from the interviews were school leaders as primary users of the NAPLAN data. Thirty-eight percent (n = 8) of those interviewed indicated that while the NAPLAN data were not kept from teachers, in the main, school leaders indicated that they mostly utilized data, or that it was not necessary for the teachers to use data.

The following comment from a school leader in an independent primary school indicates that there was no expectation for teachers to integrate NAPLAN data as part of their “classroom outcome data” or as part of their practice. In this instance, the deputy principal indicates there are clear roles for participants in this school as they relate to NAPLAN data, and in this community of practice, teachers are not expected to engage with NAPLAN data in the context of their classrooms or for next-step teaching.

I1: So, are teachers expected to integrate NAPLAN data with their classroom outcome data?

P: No.

I1: No?

P: No.

I2: They’re not, basically the focus is on the classroom data, not the NAPLAN data?

P: Yes, yeah. Well, if they, like we do look at the NAPLAN data as it comes in and we look at each student. If there was to be a major discrepancy between the NAPLAN data and what we have noticed here at school, then obviously some questions would be asked as to why the student either performed so poorly or so well on NAPLAN and yet our data is not showing that….

Deputy Principal, Independent Primary School P-7.

The focus on classroom data in this instance, is clearly where the school leadership places value. The analysis of NAPLAN data serves the function of ensuring there are no ‘major discrepanc[ies]’ between data sets, rather than approaching the data as an opportunity to look for gaps in skills that could provide a platform for targeted teaching.

Six percent of teachers cited difficulties accessing data and 13% were denied or were unable to access data at all. This created frustration for some of these teachers who genuinely wished to engage with data for planning purposes. The classroom teacher below expressed the challenge of time in relation to utilizing data in the context of the day to day responsibilities of being a teacher.

When you are in Year 5 generally we find ourselves in modern school time poor. Information rich and time poor. You are battling, let alone the data, so no, you do not sit down, some people may, you do not sit down and play with data, you are flat out preparing lessons, assessing, running your behavioural problems, organizing modern teaching, being asked to account for, accountability. Outcomes, accountability, differentiation, everyone’s knocking on your door, you do not have spare time to go and look up the data.

Year 3 Classroom Teacher, Catholic Primary School P-6.

The difficulties around downloading the information were also cited as a barrier to engagement. If NAPLAN data were too challenging to access, teachers reported a reluctance to pursue its retrieval, ultimately leading to disengagement and a devaluing of the utility of the information, or a missed opportunity to use data to inform next-step teaching.

These pathways of data sharing fell into three distinct operational processes for dissemination of the NAPLAN data as highlighted in Fig. 1 below. The common starting premise as evident in the talk, is that NAPLAN data start with the school principal (still included by definition as school leaders). Then, three operational processes are:

  1. 1.

    Open: The reflective stance of the principal is that analysis of the data is a whole school process that has collective engagement by staff in whole staff meetings or directly with staff groups and individuals.

  2. 2.

    Restricted: The principal and school leaders analyse the NAPLAN data and then take several courses of action: they present their analysis of data at staff meetings, pass their analysis of data to the faculty where it is examined from a faculty perspective down to individual class analysis or do a combination of both.

  3. 3.

    Closed: The principal and school leaders analyse the data and do not pass the data on to teachers. There is minimal expectation for teachers to utilize the data as part of teaching and learning.

Fig. 1
figure 1

Pathways of NAPLAN data dissemination

Full size image

4.2 Expertise in data literacy

The discussion around expertise in analysing NAPLAN had the greatest thematic alignment between school leaders and teacher accounts. However, while this demonstrated the greatest alignment, it should be recognized that the number of school leaders and teachers discussing this sub-theme was smaller by comparison with the other sub-themes.

Of the school leaders that discussed expertise, a cautionary note of ignoring data “at our own peril” was raised. A comment from the acting head summarizes some of the barriers to engaging with NAPLAN data, highlighting that teachers are “emotionally-based creatures” with strong connections to students not data.

I think, with the information we are able to gather about learning and how kids learn and what to do with that information, we ignore data gathering at our peril. I think teachers have taken a bit of time to get on board with that because we tend to be emotionally based creatures where you just want to be with kids and teach kids. But I think we have moved into a different kind of teaching era, and we need to start really accessing the data that we can and using it wisely.

Acting Head of Primary, Independent P-12.

However, the acting head also acknowledges that the evolution of the education system means the use of data as part of teaching is a critical part of the landscape and consideration of how data can be utilized is important. Expertise in data literacy will become critical so teachers understand “what to do with that information” in order to use data “wisely”.

The mathematics teacher below also identified the potential of data is not being realized in practice, acknowledging the place of data as part of the teaching and learning cycle. This teacher highlighted the issue of holding all the information with school leaders and not engaging the wider teacher population. Professional development would go some way to support teachers’ understanding of data and what this means “in the whole scheme of things”.

So, oh yeah, they’re at some schools, some private schools, I even worked in a private school where they had a Dean or a Director of reporting and statistics, basically. And his sole job was to try and interpret this data and let teachers know. I think it was pretty ineffectual, honestly speaking. But really, we need, I think we need some professional development on what these mean, these numbers and graphs, what do they all mean in the whole scheme of things? Because I think there’d be little regard for those, all these, all this data.

I don’t think it’s a lack of time. I think it’s just a lack of knowledge.

Year 8–10 Math Teacher, State High School.

Moving towards a more sophisticated culture of data use in schools requires attention to professional development in data literacy and data use, by not only teachers but also school leaders. Empowering teachers and school leaders (as reported by some school leaders who do not have expertise) with new knowledge will allow both groups to become active participants with expertise in data analysis rather than recipients of other people’s analysis. This may go some way to enable teachers to have ownership of whole school and student data, rather than the imposition of school leaders directing the teachers where they should target teaching and learning goals.

This is not to suggest that whole-school goals are not relevant, or school leaders’ firsthand analysis of data is not an important school operation process. Rather, the goals to target and shift across bands, as directed by school leaders, may not be directly relevant to the learning needs of students in the respective teachers’ class. Making visible the value of data and the teacher’s role in the utilization of NAPLAN data in teaching and learning are important steps for teachers to be active, informed and fully participatory community members in their school environments.

5 Discussion

5.1 Barriers and enablers of NAPLAN data use

The findings point to several barriers to NAPLAN data use but also the potential to enable opportunities for greater use. The study found that access to data resided predominantly with school leaders, with most teachers not given direct access to NAPLAN data. If teachers were given access, then it was after it had been analysed and interpreted by the school leaders in the first instance. The school leaders saw their role as experts and guides of where the data should be used; and the interpretation of NAPLAN data from school leaders for the most part was accepted as truth by the teachers. The study found there were three distinct operational processes for dissemination of the NAPLAN data as highlighted in Fig. 1, that is, open, restricted, and closed access.

This observation of differing types of access to data is not uncommon. Renshaw et al. (2013) found a similar definitive line between principals’ and teachers’ use of data. In some schools they found access to and use of data in some cases were related to authority structures within schools, with classroom teachers being denied the same access and option to engage with and understand data compared to key leadership roles in the school. When teachers were not directly involved in the analysis of data, the study found a tendency for them to become reliant on the expertise of others rather than trusting their own understanding. This finding is echoed by Hardy (2014) who reported that “some teachers expressed reliance upon the principal and other members of the leadership team to ‘translate’ NAPLAN data, and the confidence placed in the principals to assist with this work” (p. 15).

This study’s findings would suggest that socio-cultural practices of shared learning relating to NAPLAN data are not evidenced in the respective communities of practice in the schools from this study. Primarily this observation relates to sharing knowledge between school leaders and teachers, but there is also evidence that the restricted access to data at times happened within the school leadership team. In most cases, those in authority decided who needs to use the NAPLAN data and for what purpose. These school leaders also made related decisions regarding who has the expertise to disseminate and educate colleagues on data use, which in most cases is school leaders.

Some school sites appear to have engendered teacher passivity in analysing and using the data for next-step teaching. These teachers are not engaged in the data analysis process as they often are required to sit through someone else’s analysis of school data where they cannot see the relevancy of the NAPLAN data in the context of students in their own classrooms. This is a missed opportunity. Matters (2009) stated, there are prospects to use “assessment information to improve student achievement” and further occasions to use feedback from data analysis to support student learning along with the “enhancement of teachers’ pedagogical repertoires” (p. 209). Personal engagement for teachers in the process of dissemination is critical to the attribution of value in using NAPLAN data as part of next-step teaching and improving student learning outcomes. Pierce and Chick (2011) found similar barriers for teachers’ engagement with data in their research, identifying that most teachers did not have access to NAPLAN, and often it was in a form that did not allow teachers to do the analysis they required or indeed have guidance on how to interpret the data.

For these barriers to be addressed, a new set of conditions needs to be enacted to engage teachers in the possibilities of using NAPLAN data to create a community of practice around such data. Timperley (2009) outlined the need for professional development as “many teachers’ previous training and approaches to teaching practice did not require them to interpret and use these kinds of data, because assessment information was about labelling and categorising students, and not for guiding and directing teaching practice” (p. 22). Where teachers have had an opportunity to engage in professional learning relating to how to interpret data in a community of trust and relevancy, there has been evidence of changed perceptions of the value of using data to contribute to varying pedagogical practices to improve student learning, as demonstrated in Cook’s (2005) Data Club. The Data Club provided opportunities for teachers to come together as legitimate participants to help demystify data use and discuss opportunities for next-step teaching.

A challenge for the provision of professional development is the need for data to ‘speak’ directly to classroom practitioners. While the NAPLAN National Report (ACARA, 2008) and more recent infographics (ACARA, 2022) state that data have the diagnostic capability to inform next-step teaching, this message is not necessarily translated into schools. Clearer policy messaging of how NAPLAN data can be used for teaching and learning coupled with a greater emphasis on data literacy in professional development is needed (Datnow & Hubbard, 2016).

In order to enable data use, there needs to be a change in teacher’s assessment identities (Looney et al., 2017). School leaders play a critical role in shaping how teachers engage with NAPLAN data and consequently how it is used in a community of practice. In this study, some school leaders saw the utility of NAPLAN data as a strategic approach to school improvement, inclusive at times of targets and looking to gaps and patterns in data. However, school leaders did not always value data as a diagnostic tool for teachers to connect results to strategic goal setting or next-step teaching. While some school leaders and teachers recognized the benefits of data literacy and the value of professional development to enact this pathway, some teachers found barriers of access and time impeded the utilization of the data.

For school leaders to enable the use of NAPLAN data for informing teaching and improving student learning, greater stewardship is needed from school leaders to build a school culture of data literacy and professional capability. Cumming et al. (2016) assert that learning communities need to be built in schools, with school principals playing a critical role in “establishing a quality assessment for learning culture” (p. 234) that utilizes data to inform learning, acknowledging “assessment is for learning; that is, all assessments contribute to understanding student progress in learning and assisting further learning” (p. 232).

To enable this change at a policy level, there are two critical issues that need to be addressed. First, NAPLAN data needs to be valued as a source of information to improve learning. Second, there needs to be recognition of the need for professional development in data literacy. If teachers and schools have “primary accountability for improving student outcomes” (MCEETYA, 2008, p. 16), then teachers need greater access to NAPLAN data and professional development to improve their expertise in analysing NAPLAN, and other data. This was once a policy priority of the National Literacy and Numeracy Plan which emphasized the importance of “professional development for teachers to support the key elements of the plan” (DEETYA, 1998, p. 10). However, public acknowledgement of the priority of professional development has been omitted from more recent national policy documents (MCEETYA, 2008; Education Council, 2019). If data use and accountability are stated professional expectations, then valuing data literacy for next-step teaching needs to be acknowledged as an essential professional capability. This needs to be enabled at a policy level, but also prioritized by school leaders and teachers.

6 Conclusion

This paper has sought to explore how both school leaders and teachers engage with NAPLAN data once results are available to them. It has highlighted how preventing access to data or the necessary skills to interpret data may undermine improvement efforts. As Hargreaves and Shirley (2009) observed, data that we have available to us deserves “intelligent interpretation” (p. 39). Highlighted in this paper are opportunities that will enable a greater engagement of all staff to be involved in professional development to improve their data literacy and for schools to operate as communities of practice when engaging with NAPLAN data.

School leaders are in a position to “offer new forms of identification and negotiability” to create “meaningful forms of membership and empowering forms of ownership of meaning” (Wenger, 1998, p.269). Sharing ownership and supporting professional development of meaningful use of NAPLAN data will go some way to reassert the value of NAPLAN data as a tool for next-step teaching at schools. The risk of not establishing these new communities is the potential for data classes (Manovich, 2012) to develop and the evolution of social divisions between those that do data and those who had data done to them within practices and processes (Selwyn, 2015).

This assertion is not to contest the need for school leadership to analyse NAPLAN data as part of a whole school strategy, instead the assertion highlights the need to prioritize the relevancy of NAPLAN data for teachers’ work as part of a cycle of inquiry. Selwyn (2015) argues for greater emphasis in “empirical work that strives to understand and account for the manner in which data are accumulated; to make visible the flow and circulation of data and begin to understand the ways in which data are then integrated back into everyday education practices” (p. 75).

There is a need to create opportunities to develop school leaders’ and teachers’ data literacy to inform goal setting and decision making with innovation for next-step teaching and learning. Through greater stewardship, building sustainable cultures of inclusion that improve professional capability in data literacy have the potential of creating a value and utility of NAPLAN data to inform teaching and impact through student improvement.