Teacher perceptions of impact upon pedagogy and practice
The first three questions presented in Table 1 asked about the teacher’s perceptions of the impact NAPLAN was having upon their pedagogy (e.g. does NAPLAN narrow the focus of the curriculum?). It was found that 40–50% answered “not at all” or “slightly” to these questions which suggests that little impact of NAPLAN was perceived by a substantial proportion of the teachers. However, there was also a reasonable proportion (albeit much less) reporting a significant impact, with 15–30% stating “very much” or “extremely”. While our sample is limited to teachers from Independent schools, the results are consistent with prior studies suggesting that teachers from Independent schools do not tend to experience such a large impact on their curriculum as what may be the experience of teachers in government and/or lower SES contexts (Thompson and Harbaugh 2013; APPA 2013).
Table 1 Teacher (n = 40) perceptions regarding the impact and use of NAPLAN
In previous work we presented evidence to suggest that the parents and teachers surveyed in the present study reported “a little bit” of emotional distress during NAPLAN testing (Rogers et al. 2016). Consistent with those previously reported results, as shown in Table 1, around 50% of the teachers responded “not at all” or “slightly” to a question asking how stressed NAPLAN makes them feel. Additionally, around 70% reported “not at all” or “slightly” to a question asking if NAPLAN makes them want to teach in another year level that does not have NAPLAN. Therefore, most teachers did not report much impact to their general well-being due to NAPLAN testing. However, it must also be noted that around 10% answered “very much” or “extremely” to these two questions.
The final few questions reported in Table 1 asked about some potential uses of NAPLAN. Results show that most teachers surveyed are sceptical that NAPLAN testing can ensure a consistent experience for all students, that NAPLAN can be used to identify strengths and weaknesses for individual students, and that NAPLAN provides useful feedback regarding their own teaching performance. Therefore, while the results from the present study suggest most teachers did not report any large impact on their curriculum or any great deal of stress associated with the testing, neither did they appear to perceive the testing as particularly useful.
When examining Spearman correlations between the question items presented in Table 1, it was revealed that teachers reporting more stress associated with the testing also tended to report a stronger desire to teach in a different year without NAPLAN (Spearman r = .62, p < .01). Both questions were also positively associated with the items that asked if NAPLAN narrows the focus of the curriculum (Spearman rs = .45 and .61, ps < .01, respectively), and the extent that NAPLAN is perceived to place too much emphasis on literacy and numeracy to the detriment of other areas (Spearman rs = .61 and .45, ps < .01, respectively). Therefore, a teacher reporting more impact upon their curriculum also tended to report more stress associated with NAPLAN. The relationship between stress and perceived curriculum impact suggests there may be greater stress associated with NAPLAN in other contexts where the impact of NAPLAN is felt more intensely. It has been suggested that curriculum impact is greater in non-Independent schools (APPA 2013; Thompson and Harbaugh 2013). No other correlations between items listed in Table 1 reached statistical significance.
Parent and teacher perceptions of transparency and accountability
From the beginning of implementation to the present day, NAPLAN testing has been claimed to serve the Australian public by providing increased transparency and accountability across primary and secondary schooling (Australian Senate Enquiry 2010, 2014). These kind of perceptions have been investigated in the context of other testing regimes in other parts of the world (Brockmeier et al. 2014). However, a nuanced understanding of teacher and parent beliefs about how well NAPLAN achieves these over-arching transparency and accountability goals is lacking in the Australian context. In the present study, we directly asked teachers and parents if NAPLAN results are an indicator of how well schools are doing, and how well teachers can teach (i.e. transparency). We also asked about their perceptions regarding the perceived extent that NAPLAN increases accountability of schools, teachers, and parents. Responses to these questions are presented in Tables 2 (parents) and 3 (teachers).
Table 2 Parent (n = 345) responses to survey items asking about the perceived transparency and accountability of NAPLAN
For parents, responses were evenly spread across the response scale. While about 1/5 reported a belief that NAPLAN fosters transparency and accountability “not at all”, another 1/5 reported “very much” or “extremely” (see Table 2). A strong positive association was found between all items. Compared with the parents, overall the teachers tended to report less endorsement of NAPLAN as fostering transparency and accountability (see Table 3). The teachers largely did not endorse NAPLAN as an indicator of teacher performance, as around 60% reported “not at all”. Both parents and teachers perceived NAPLAN as fostering accountability of teachers and schools more than fostering any accountability of parents. For teachers, correlations between all items were moderate-strong, except for the ‘accountability of parents’ item. Therefore, an overall transparency and accountability measure was created by averaging across the first four items (this excludes the ‘accountability of parents’ item). This composite measure is further discussed later in the results section (Section “Summary of composite variables”).
Table 3 Teacher (n = 40) responses to survey items asking about the perceived transparency and accountability of NAPLAN Parent and teacher perceptions of usefulness, validity, and fairness of the testing
The Australian Senate NAPLAN enquiries reported that stakeholders generally perceive NAPLAN as a useful initiative for providing broad-scale comparisons across schools at the state and national levels (Australian Senate Enquiry 2010, 2014). However, the senate enquiries also reported that stakeholder perceptions regarding the usefulness of NAPLAN for serving the needs of individual teachers and students were mixed. Mixed positive and negative parent and teacher perceptions are also evident in the research literature (Polesel et al. 2012). As shown in Tables 4 and 5 (see the first three items in each table) it is evident that mixed positive/negative perceptions also existed within our sample of parents and teachers regarding the usefulness of NAPLAN to help individual students. The inter-correlations between items reveal strong positive associations between the ‘usefulness’ items for both parents and teachers. We therefore created a composite ‘usefulness’ score from these items that is discussed later in the results section (Section “Summary of composite variables”).
Table 4 Parent (n = 345) responses to survey items asking about the perceived usefulness of NAPLAN
Table 5 Teacher (n = 40) responses to survey items asking about the perceived usefulness of NAPLAN
Concerns have also previously been raised about the validity and fairness of NAPLAN testing (Australian Senate Enquiry 2010, 2014; Polesel et al. 2012). We asked respondents their perception of the extent that NAPLAN measures all aspects of maths and reading (validity type question), and the extent that it is a fair form of testing for children from all cultural backgrounds (fairness type question). Again, there are mixed responses to these items; however, participants expressed more doubt about these validity and fairness questions compared to the questions targeting usefulness. For the validity question, approximately 35% of both parent and teacher groups responded “not at all”. A similar proportion of parents responded “not at all” for the fairness question, and approximately 60% of the teachers responded “not at all” to this question. This suggests that more work needs to be done by the test administrators to convince stakeholders of the validity and appropriateness of widespread usage of the testing. Our data are however limited to only single questions regarding validity and fairness, with a specific population of respondents (i.e. parents and teachers from Independent schools). Future research with a wider set of items and a broader sample would be useful. The validity and fairness questions were found to be strongly (for parents) and moderately (for teachers) positively associated with one another. This suggests that a respondent perceiving the test to be more appropriate for widespread use also tended to perceive the test as encompassing most aspects of maths and reading. For parents, items were also positively associated with the usefulness items. This suggests that parents perceiving the test as more valid and fair also tended to perceive the testing as more useful for comparing across/within students and helping individual students learn. Low-moderate positive associations among these variables were observed for teachers.
Parent and teacher perceptions of the clarity of communication of NAPLAN results
In this section the results from parents are from Year 5 parents (n = 198) only, since they had prior experience regarding the communication of NAPLAN results when their child was in Year 3. In contrast, the surveyed Year 3 parents would not have had sufficient prior experience to answer the items with any confidence. Previous discussion about the communication of NAPLAN results has largely focused upon the presentation of results on the My School website (Australian Senate Enquiry 2010, 2014; Ragusa and Bousfield 2015). Pierce and Chick (2011) have reported findings that teachers may experience difficulties interpreting results of their students as provided to them by the governing body of NAPLAN, the Australian Curriculum and Assessment Reporting Authority (ACARA). In the present study, we focus on perceptions regarding how well the results of students are communicated to the students and parents by the teacher and the school report. Parent and teacher responses were similar with around 50% responding “not at all” or “slightly” and around 15% responding “very much” or “extremely” to these items, from both groups (see Tables 6, 7). Therefore findings reveal some mixed perceptions situated within a prevailing negativity regarding the communication of results to students and parents (by the teacher and school report). Strong associations were found between the first four items for both parents and teachers. An overall communication appraisal score was created by averaging across these items that will be discussed further in Section “Summary of composite variables”.
Table 6 Year 5 Parent (n = 198) responses to survey items asking about how well NAPLAN results are communicated
Table 7 Teacher (n = 40) responses to survey items asking about how well NAPLAN results are communicated
A single question was also asked to gauge parent and teacher perception regarding how well results are communicated via the My School website. As can be seen in Tables 6 and 7, perceptions were very mixed with relatively even responses across the response scale. Another question asked if respondents believed NAPLAN results were communicated in a timely manner, and not surprisingly the most frequent response by both parents and teachers was “not at all” (around 35% parents, and around 60% teachers). One reason behind the upcoming switch to online NAPLAN assessment is to improve the turnaround time for the reporting of results to address criticisms regarding the long wait for results that has plagued the testing initiative since inception (Australian Senate Enquiry 2014).
Summary of composite variables
While prior sections have provided the frequency response data for individual survey items, in this section we examine composite variables. The composite variables represent an overall measure of the perception of the ‘transparency and accountability’ afforded by NAPLAN (see Section “Parent and teacher perceptions of transparency and accountability”, questions A1–A4), the ‘usefulness of NAPLAN for helping individual students’ (see Section “Parent and teacher perceptions of usefulness, validity, and fairness of the testing”, questions U1–U3), and the ‘clarity of the communication of NAPLAN results to parents and students by the teacher and school report’ (see Section “Parent and teacher perceptions of the clarity of communication of NAPLAN results”, questions C1–C4). To statistically justify the appropriateness of our composite variables we conducted confirmatory factor analysis (CFA) using a structural equation modelling (SEM) approach (Acock 2013; Rubio and Gillespie 1995), see Fig. 1. The parent models acquired adequate goodness of fit indices, but the teacher model did not as this group has a relatively low sample size. Across all groups the reliability of all factors was high (i.e. equal to or greater than .79). A strong positive association was found between perceptions of accountability/transparency and usefulness for individual students for parents (Year 5 parents r = .81, Year 3 parents r = .77), and a moderate association for teachers (r = .46). These data suggest that continued work to foster engagement with NAPLAN results will likely help to improve NAPLAN’s image as a testing initiative that promotes a more transparent and accountable primary and secondary Australian education system. For Year 5 parents, a positive association was found between perceptions surrounding the clarity of communication of results with perceptions of transparency/accountability (r = .38), and usefulness of the testing (r = .44). These results suggest that increasing parental sense of the clarity of communication may promote their general attitudes towards the testing. These same associations were not observed for teachers.
While previous sections highlight the existence of mixed negative and positive perceptions, the average perception as obtained by the composite scores obtained by averaging across items for each factor for both parents and teachers, is around “somewhat”, see Fig. 2. The only exception is the average teacher perception of transparency and accountability that is closer to “slightly”. The transparency and accountability measure was the only measure of the three that yielded a statistically significant difference between parent and teacher perceptions, t(384) = 2.65, p < .01, d = .48. As can be seen in Fig. 2 results suggest that parents typically report a stronger perception of transparency and accountability than teachers, although the mean level for parents is still low on the scale (i.e. “somewhat”).
Open-ended responses from parents
As part of the survey an optional open-ended response box was available for parents. One hundred and eighty-five (54%) of the parents provided a written response. The frequency of occurring themes is summarised in Table 8. This qualitative data complements the variation that was noted in the quantitative data. While a substantial proportion provided comments that suggested they can see the potential of NAPLAN as a tool to help individual students (19%), a similar proportion also adamantly stated a belief that NAPLAN was not a good measure of a child’s ability (16%). It was also expressed that too much emphasis is placed upon NAPLAN by the general community (10%), and that the additional pressure placed upon stakeholders was not needed or helpful (17%). Therefore, similar to the quantitative findings, a mix of generally ‘positive’ and ‘negative’ attitudes are evident.
Table 8 Frequencies of different themes present within the open-ended comments provided by parents