Abstract
This paper will highlight the successful evaluation approaches deployed for long-term exploration of the impact of a national policy initiative as well as some of the results and outcomes. An interactive feedback process informed decision-making at the national and local level thereby enhancing both the initiative and its implementation in schools. The integration of information and communication technologies (ICT) into schools has been a focus of a plethora of policy initiatives by governments worldwide for over 20 years. The provision of laptops to teachers is one component of the New Zealand compulsory school sector ICT strategy. The evaluation research design was to use a mixed-methods approach incorporating three yearly cycles of annual nationwide surveys, regional focus groups, and school-based longitudinal case studies. Randomized sampling was used to identify 20% of the available laptop schools as potential participants in the survey. The same schools participated in each of the 3 years. A distinctive feature of this evaluation has been the regular feedback loops between the evaluators and the stakeholders. The stakeholders included the Ministry of Education and laptop suppliers as well as schools, school leaders, and teacher unions. In the current accountability, environment schools are increasingly being asked to participate in evaluation studies. In long-term studies, it is important to establish effective relationships with the schools and the policy-makers and to deploy robust evaluation methodologies. This is essential for both validity and reliability of the findings and the useability and utilization of the findings.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Coburn C. (2005) The role of nonsystem actors in the relationship between policy and practice: The case of reading instruction in California. Educational Evaluation and Policy Analysis 27(1): 23–52
Cowie B., Jones A. (2009) IT and the teaching environment. In: Saha L., Dworkin A. (eds) International handbook of research on teachers and teaching. Springer, London, pp 791–802
Cowie B., Jones A., Harlow A. (2005) The digital horizons: Laptops for teachers policy initiative: Impacts and consequences. New Zealand Annual Review of Education 15: 111–132
Cowie, B., Jones, A., & Harlow, A. (2006). Laptops as fulcrum or lever: Changes in secondary teacher practices over the three years subsequent to their access to TELA laptops. Paper presented at NZARE Conference, Rotorua, 5–9 December 2006.
Cowie B., Jones A., Harlow A., McGee C., Cooper B., Forret M., Miller T., Gardiner B. (2008) Laptops for teachers evaluation: Years 9–13. Ministry of Education, Wellington
Cowie B., Jones A., Harlow A., with Forret M. (2010) Laptops for teachers evaluation: Years 1–3. Ministry of Education, Wellington
Cowie B., Jones A., Harlow A., with Forret M., McGee C., Miller T. (2010) Laptops for teachers evaluation: Years 4–6. Ministry of Education, Wellington
Edwards, A. (2007). Conceptualising user engagement in educational research: Working on the boundaries. Paper presented in the AERA Symposium: User Engagement in Enhancing the Quality of Research on Policy Implementation: Lessons from the UK, Chicago, April, pp. 9–13.
Fishman B., Marx R., Blumenfeld P., Krajcik J. S., Soloway E. (2004) Creating a framework for research on systemic technology innovations. Journal of the Learning Sciences 13(1): 43–76
Fullan M. (2005) Leadership and sustainability: System thinkers in action. Corwin Press, Thousand Oaks
Hannaway J., Woodroffe N. (2003) Policy instruments in education. Review of Research in Education 27: 1–24
Honig M. (2006) Complexity and policy implementation: Challengers and opportunities for the field. In: Honing M. New directions in education policy implementation: Confronting complexity. State University of New York Press, Albany, pp 1–23
Knapp M. (2002) Understanding how policy meets practice: Two takes on local response to a state reform initiative. Centre for the Study of Teaching and Policy, University of Washington, Washington
Levin, B. (2003). Improving research–policy relationships: Lessons from the case of literacy. Paper prepared for the OISE/UT International Literacy Conference: Literacy Policies for the Schools We Need, Toronto, November 8.
Levin, B. (2004). Making research matter more. Education Policy Analysis Archives, 12(56). Retrieved March 12, 2010 from http://www.epaa.asu.edu/epaa/v12n56/.
Levin B. (2009) Does politics help or hinder education change? Journal of Educational Change 10: 69–72
Ministry of Education: (2002) Digital horizons: Learning through ICT. Learning Media, Wellington
Ministry of Education: (2003) The digital horizons: Laptops for teachers scheme. Learning Media, Wellington
Ministry of Education: (2006) Enabling the 21st century learner: An e-learning action plan for schools 2006–2010. Ministry of Education, Wellington
Nutley, S. (2003). Bridging the policy/research divide: Reflections and lessons from the UK. Paper presented at National Institute of Governance Conference, Canberra. Retrieved from http://www.st-andrews.ac.uk/-ruru.
Olson J. (2000) Trojan horse or teacher’s pet? Computers and the culture of the school. Journal of Curriculum Studies 32(1): 1–8
Patton M. (2002) Qualitative research and evaluation methods, 3rd edn. Sage, Thousand Oaks
Patton M. (2008) Utilization-focused evaluation, 4th edn. Sage, Thousand Oaks
Putman R., Borko H. (2000) What do new views of knowledge and thinking have to say about research on teacher learning? Educational Researcher 29(1): 4–15
Selwyn N., Facer K. (2007) Beyond the digital divide: Rethinking digital inclusion for the twenty-first century. Futurelab, Bristol
Senge P. (1990) The fifth disciple: The art and practice of learning organization. Currency Doubleday, New York
Spillane J. (2004) Distributed leadership. Jossey-Bass, San Francisco
Spillane J., Reisner B., Gomez L. (2006) Policy implementation and cognition: The role of human, social, and distributed cognition in framing policy implementation. In: Honing M. (ed.) New directions in education policy implementation: Confronting complexity. State University of New York Press, Albany, pp 47–64
Webb R., Vulliamy G. (2007) Changing classroom practice at key stage 2: The impact of new labour’s national strategies’. Oxford Review of Education 33(5): 561–580
Weiss C. (1998) Have we learned anything new about the use of evaluation? American Journal of Education 19: 21–34
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Jones, A., Cowie, B. Evaluation approaches for a national ICT initiative: the example of laptops for New Zealand teachers. Educ Res Policy Prac 10, 3–15 (2011). https://doi.org/10.1007/s10671-010-9089-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10671-010-9089-y