Skip to main content

Adopting and adapting: school leaders in the age of data-informed decision making

Educational Assessment, Evaluation and Accountability volume 30, pages 133–158 (2018)Cite this article

Abstract

The concept of data-informed decision making (DIDM), a term used interchangeably with data-driven decision making (DDDM) and data-based decision making (DBDM), is relatively new to Irish education and the school planning process. This research sought to clarify what data principals use and how they use that information for school improvement considering new school self-evaluation requirements. The paper begins by charting the rise internationally of data use in school planning, decision making and accountability. It proceeds to describe the policy context in this area in Ireland and then reports recent research with school leaders around how data is collected and used in their work. Although the paper focusses on Ireland, it is tentatively suggested that school leaders, teachers and policymakers in other countries, and there are many, which have come late to the expectation that school improvement and accountability should be heavily data-informed may find the efforts of Irish principals in this regard of interest.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

References

  • Anderson, S., Leithwood, K., & Strauss, T. (2010). Leading data use in schools: organizational conditions and practices at the school and district levels. Leadership and Policy in Schools, 9, 292–327.

    Article  Google Scholar 

  • Bernhardt, V. (2013). Data analysis for continuous school improvement. New York: Routledge.

    Google Scholar 

  • Brown, M., McNamara, G., & O’Hara, J. (2016). Quality and the rise of value-added in education: the case of Ireland. Policy Futures in Education, 14, 810–829.

    Article  Google Scholar 

  • Bush, G. W. (2001). No child left behind act. Washington, DC: United States Department of Education Available from: http://www2.ed.gov/admins/lead/account/nclbreference/index.html. Accessed 02 Aug 2016.

    Google Scholar 

  • Celio, M. B., & Harvey, J. (2005). Buried treasure: developing a management guide from mountains of school data. Bothell: Center on Reinventing Public Education.

    Google Scholar 

  • Coburn, C. E., & Talbert, J. E. (2006). Conceptions of evidence use in school districts: mapping the terrain. American Journal of Education, 112, 469–495.

    Article  Google Scholar 

  • Coburn, C. E., & Turner, E. O. (2011). Research on data use: a framework and analysis. Measurement: Interdisciplinary Research & Perspective, 9, 173–206.

    Google Scholar 

  • Cohen, L., Manion, L., & Morrison, K. (2013). Research methods in education. New York: Routledge.

    Google Scholar 

  • Copland, M. A. (2003). Leadership of inquiry: building and sustaining capacity for school improvement. Educational Evaluation and Policy Analysis, 25, 375–395.

    Article  Google Scholar 

  • Creswell, J. W. (2012). Educational research: planning, conducting, and evaluating quantitative. Harlow: Pearson Education.

    Google Scholar 

  • Darling-Hammond, L. (2007). Race, inequality and educational accountability: the irony of ‘No Child Left Behind’. Race Ethnicity and Education, 10, 245–260.

    Article  Google Scholar 

  • Datnow, A., & Hubbard, L. (2016). Teacher capacity for and beliefs about data-driven decision making: a literature review of international research. Journal of Educational Change, 17, 7–28.

    Article  Google Scholar 

  • Datnow, A., & Park, V. (2009). School system strategies for supporting data use. In T. Kowalski & T. J. Lasley (Eds.), Handbook of data-based decision making in education. New York: Routledge.

    Google Scholar 

  • Datnow, A., Park, V., & Wohlstetter, P. (2007). Achieving with data: how high-performing school systems use data to improve instruction for elementary students. Los Angeles: University of Southern California.

    Google Scholar 

  • Datnow, A., Park, V., & Kennedy-Lewis, B. (2013). Affordances and constraints in the context of teacher collaboration for the purpose of data use. Journal of Educational Administration, 51, 341–362.

    Article  Google Scholar 

  • DES. (2005). DEIS (delivering equality of opportunity in schools): an action plan for educational inclusion. Dublin: DES.

    Google Scholar 

  • DES. (2011). School Self-Evaluation, Guidelines for Post-Primary Schools. Dublin: DES.

    Google Scholar 

  • DES. (2012). Circular 0040/2012: implementation of school self-evaluation. Dublin: DES.

    Google Scholar 

  • DES. (2014). Circular 70/2014: guidance for post-primary schools on the provision of resource teaching and learning support. Dublin: Government Publications.

    Google Scholar 

  • DES. (2016a). Looking at our School 2016. A quality framework for post-primary schools. Dublin: Government Publications.

    Google Scholar 

  • DES. (2016b). Circular 34/2016: information in relation to standardised testing and other matters. Dublin: Government Publications.

    Google Scholar 

  • DES. (2016c). Circular 40/2016: continuing implementation of school self-evaluation 2016–2020. Dublin: Government Publications.

    Google Scholar 

  • Earl, L., & Katz, S. (2006). Leading in a data rich world: harnessing data for school improvement. Thousand Oaks: Corwin.

    Google Scholar 

  • Fabry, D. L., & Higgs, J. R. (1997). Barriers to the effective use of technology in education: current status. Journal of Educational Computing Research, 17, 385–395.

    Article  Google Scholar 

  • Gelderblom, G., Schildkamp, K., Pieters, J., & Ehren, M. (2016). Data-based decision making for instructional improvement in primary education. International Journal of Educational Research, 80, 1–14.

    Article  Google Scholar 

  • Gill, B., Borden, B. C. & Hallgren, K. (2014). A conceptual framework for data-driven decision making. Final report of research conducted by Mathematica Policy Research, Princeton, submitted to Bill & Melinda Gates Foundation, Seattle, WA.

  • Gilleece, L. (2014). Understanding achievement differences between schools in Ireland—can existing data-sets help? Irish Educational Studies, 33, 75–98.

    Article  Google Scholar 

  • Greller, W., & Drachsler, H. (2012). Translating learning into numbers: a generic framework for learning analytics. Journal of Educational Technology & Society, 15, 42–57.

    Google Scholar 

  • Hallinger, P. (2010). A review of three decades of doctoral studies using the principal instructional management rating scale: a lens on methodological progress in educational leadership. Educational Administration Quarterly, 47, 271–306.

    Article  Google Scholar 

  • Hislop, H. (2013). Applying an evaluation and assessment framework: an Irish perspective. In Irish presidency of the Council of the European Union, presidency conference: Better assessment and evaluation to improve teaching and learning. Dublin: DES.

    Google Scholar 

  • Hoogland, I., Schildkamp, K., Van Der Kleij, F., Heitink, M., Kippers, W., Veldkamp, B. & Dijkstra, A. M. (2016). Prerequisites for data-based decision making in the classroom: Research evidence and practical illustrations. Teaching and Teacher Education. 60, 3770386

  • Honig, M. I., & Coburn, C. (2007). Evidence-based decision making in school district central offices: toward a policy and research agenda. Educational Policy, 22, 578–608.

    Article  Google Scholar 

  • Hout, M., & Elliott, S. W. (2011). Incentives and test-based accountability in education. Washington, DC: National Academies Press.

    Google Scholar 

  • Ikemoto, G. S., & Marsh, J. A. (2007). Cutting through the “data-driven” mantra: different conceptions of data-driven decision making. Reprints. Santa Monica: RAND Corporation.

    Google Scholar 

  • Ingram, D., Seashore Louis, K., & Schroeder, R. (2004). Accountability policies and teacher decision making: barriers to the use of data to improve practice. The Teachers College Record, 106, 1258–1287.

    Article  Google Scholar 

  • Johnson Jr., B. L., & Kruse, S. D. (2012). Decision making for educational leaders: underexamined dimensions and issues. Albany: SUNY Press.

    Google Scholar 

  • Kerr, K. A., Marsh, J. A., Ikemoto, G. S., Darilek, H., & Barney, H. (2006). Strategies to promote data use for instructional improvement: actions, outcomes, and lessons from three urban districts. American Journal of Education, 112, 496–520.

    Article  Google Scholar 

  • Killion, J., & Bellamy, G. T. (2000). On the job. Journal of Staff Development, 21, 27–31.

    Google Scholar 

  • Knapp, M. S., Swinnerton, J. A., Copland, M. A., & Monpas-Huber, J. (2006). Data-informed leadership in education. Seattle: Center for the Study of Teaching and Policy.

    Google Scholar 

  • Koretz, D. (2003). Using multiple measures to address perverse incentives and score inflation. Educational Measurement: Issues and Practice, 22, 18–26.

    Article  Google Scholar 

  • Kowalski, T., & Lasley, T. J. (2009). Handbook of data-based decision making in education. New York: Routledge.

    Google Scholar 

  • Kowalski, T. J., Lasley, T. J. & Mahoney, J. W. (2008). Data-driven decisions and school leadership: Best practices for school improvement, Pearson education. Boston: MA.

  • Lachat, M. A., & Smith, S. (2005). Practices that support data use in urban high schools. Journal of Education for Students Placed at Risk, 10, 333–349.

    Article  Google Scholar 

  • Lai, M. K., & Schildkamp, K. (2013). Data-based decision making: an overview. Data-based decision making in education. Dordtrecht: Springer.

    Google Scholar 

  • Leithwood, K., Seashore Louis, K., Anderson, S., & Wahlstrom, K. (2004). Review of research: how leadership influences student learning. Minneapolis: University of Minnesota.

    Google Scholar 

  • Louis, K. S., Leithwood, K., Wahlstrom, K. L., Anderson, S. E., Michlin, M., Mascall, B., Gordon, M., Thomas, E., Tiiu, S., & Moore, S. (2010). Learning from leadership: investigating the links to improved student learning. Toronto: Center for Applied Research and Educational Improvement/University of Minnesota and Ontario Institute for Studies in Education/University of Toronto.

    Google Scholar 

  • Love, N. (2009). Using data to improve learning for all: a collaborative inquiry approach. Thousand Oaks: Corwin Press.

    Google Scholar 

  • Love, N., Stiles, K. E., Mundry, S., & Diranna, K. (2008). The data coach’s guide to improving learning for all students: unleashing the power of collaborative inquiry. Thousand Oaks: Corwin Press.

    Google Scholar 

  • MacBeath, J. (1999). Schools must speak for themselves: the case for school self-evaluation. London: Routledge.

    Google Scholar 

  • MacBeath, J. (2009). Self-evaluation for school improvement. Second international handbook of educational change. Dordrecht: Springer.

    Google Scholar 

  • MacBeath, J. (2013). Learning in and out of school: the selected works of John MacBeath. London: Routledge.

    Google Scholar 

  • Mandinach, E. B., & Gummer, E. S. (2016). Data literacy for educators: making it count in teacher preparation and practice. New York: Teachers College Press.

    Google Scholar 

  • Mandinach, E. B., & Jackson, S. S. (2012). Transforming teaching and learning through data-driven decision making. Thousand Oaks: Corwin Press.

    Book  Google Scholar 

  • Mandinach, E. B., Honey, M., & Light, D. (2006). A theoretical framework for data-driven decision making. In Annual meeting of the American Educational Research Association. San Francisco: AERA.

    Google Scholar 

  • Marsh, J. A. (2012). Interventions to promote data use: an introduction. Teachers College Record. New York: Columbia University.

    Google Scholar 

  • Marsh, J. A., & Farrell, C. C. (2014). How leaders can support teachers with data-driven decision making: a framework for understanding capacity building. Educational Management Administration & Leadership, 43, 269–289.

    Article  Google Scholar 

  • Marsh, J. A., Pane, J. F. & Hamilton, L. S. (2006). Making sense of data-driven decision making in education: Evidence from Recent RAND Research. Santa Monica, CA: RAND Corporation.

  • Matthews, P., & Lewis, P. (2009). How do school leaders successfully lead learning? Nottingham: National College for School Leadership.

    Google Scholar 

  • McNamara, G., & O’Hara, J. (2008). Trusting schools and teachers: developing educational professionalism through self-evaluation. New York: Peter Lang.

    Google Scholar 

  • Morris, A. (2011). Student standardised testing: current practices in OECD countries and a literature review. OECD Education Working Papers No. 65. Paris: OECD Publishing.

  • Nayir, K. F., & McNamara, G. (2014). The increasingly central role of school self-evaluation in inspection systems across Europe: the case of Ireland. Turkish Journal of Education, 3, 48–59.

    Article  Google Scholar 

  • Nelson, R., Ehren, M., & Godfrey, D. (2015). Literature review on internal evaluation. London: UCL Institute of Education. Available.

    Google Scholar 

  • O’Brien, S., McNamara, G., & O’Hara, J. (2015). Supporting the consistent implementation of self-evaluation in Irish post-primary schools. Educational Assessment, Evaluation and Accountability, 27, 377–393.

    Article  Google Scholar 

  • O'Day, J. (2002). Complexity, accountability, and school improvement. Harvard Educational Review, 72, 293–329.

    Article  Google Scholar 

  • Park, V., & Datnow, A. (2008). Collaborative assistance in a highly prescribed school reform model: the case of success for all. Peabody Journal of Education, 83, 400–422.

    Article  Google Scholar 

  • Park, V., & Datnow, A. (2009). Co-constructing distributed leadership: district and school connections in data-driven decision-making. School leadership and Management, 29, 477–494.

    Article  Google Scholar 

  • Parylo, O., & Zepeda, S. J. (2014). Describing an ‘effective’ principal: perceptions of the central office leaders. School Leadership & Management, 34, 518–537.

    Article  Google Scholar 

  • Patton, M. Q. (2015). Qualitative research & evaluation methods: integrating theory and practice. Thousand Oaks: SAGE.

    Google Scholar 

  • PDST. (2016). Senior Cycle Subject Analysis Spreadsheets [Online]. Dublin: PDST Available: http://PDST.ie/postprimary. Accessed Web Page 2016.

    Google Scholar 

  • Picciano, A. G. (2006). Data-driven decision making for effective school leadership. Englewood Cliffs: Prentice-Hall.

    Google Scholar 

  • Prøitz, T. S., Mausethagen, S., & Skedsmo, G. (2017). Investigative modes in research on data use in education. Nordic Journal of Studies in Educational Policy, 3(1), 42–55.

    Article  Google Scholar 

  • Romero, C., & Ventura, S. (2007). Educational data mining: a survey from 1995 to 2005. Expert Systems with Applications, 33, 135–146.

    Article  Google Scholar 

  • Rosenkvist, M. A. (2010). Using student test results for accountability and improvement. Paris: OECD.

    Book  Google Scholar 

  • Schildkamp, K. (2007). The utilisation of a self-evaluation instrument for primary education Accessed on-line http://doc.utwente.nl/57803/1/thesis_Schildkamp.pdf on 24/07/2015. Enshede: University of Twente.

    Google Scholar 

  • Schildkamp, K., & Kuiper, W. (2010). Data-informed curriculum reform: which data, what purposes, and promoting and hindering factors. Teaching and Teacher Education, 26, 482–496.

    Article  Google Scholar 

  • Schildkamp, K., & Tiddly, C. (2008). School performance feedback systems in the USA and in The Netherlands: a comparison. Educational Research and Evaluation, 14, 255–282.

    Article  Google Scholar 

  • Schildkamp, K., & Visscher, A. (2014). Data-centered school self-evaluation in the Netherlands: Characteristics and prerequisites. In M. Lai & S. Kushner (Eds.), A Developmental and Negotiated Approach to School Self-Evaluation (Advances in Program Evaluation) (Vol. 14, pp. 233–252). Bingley: Emerald Group Publishing, Limited.

    Chapter  Google Scholar 

  • Schildkamp, K., Rekers-Mombarg, L. T. M., & Harms, T. J. (2012). Student group differences in examination results and utilization for policy and school development. School Effectiveness and School Improvement, 23, 229–255.

    Article  Google Scholar 

  • Schildkamp, K., Karbautzki, L., & Vanhoof, J. (2013a). Exploring data use practices around Europe: identifying enablers and barriers. Studies in Educational Evaluation, 42, 15–24.

    Article  Google Scholar 

  • Schildkamp, K., Lai, M. K., & Earl, L. (2013b). Data-based decision making in education: challenges and opportunities. Dordrecht: Springer.

    Book  Google Scholar 

  • Schildkamp, K., Poortman, C. L., & Handelzalts, A. (2015). Data teams for school improvement. School Effectiveness and School Improvement, 27, 228–254.

    Article  Google Scholar 

  • Senge, P. M., Cambron-Mccabe, N., Lucas, T., Smith, B., & Dutton, J. (2012). Schools that learn: a fifth discipline fieldbook for educators, parents, and everyone who cares about education. Boston: Nicholas Brealey Publishing.

    Google Scholar 

  • Shah, M. (2014). Impact of management information systems (MIS) on school administration: what the literature says. Procedia-Social and Behavioral Sciences, 116, 2799–2804.

    Article  Google Scholar 

  • Shen, J., & Cooley, V. E. (2008). Critical issues in using data for decision-making. International Journal of Leadership in Education, 11, 319–329.

    Article  Google Scholar 

  • Smith, M. (2005). Data for schools in NSW: what is provided and can it help? Using data to support learning. Melbourne: Council for Educational Research.

    Google Scholar 

  • Smyth, E. (1999). Do schools differ? Dublin: Economic and Social Research Institute.

    Google Scholar 

  • Smyth, E., McCoy, S., & Kingston, G. (2015). Learning from the evaluation of DEIS. Dublin: ESRI.

    Google Scholar 

  • Spillane, J. P. (2012). Data in practice: conceptualizing the data-based decision-making phenomena. American Journal of Education, 118, 113–141.

    Article  Google Scholar 

  • Spillane, J. P., Halverson, R., & Diamond, J. B. (2004). Towards a theory of leadership practice: a distributed perspective. Journal of Curriculum Studies, 36, 3–34.

    Article  Google Scholar 

  • Sun, J., Przybylski, R., & Johnson, B. J. (2016). A review of research on teachers’ use of student data: from the perspective of school leadership. Educational Assessment, Evaluation and Accountability, 28:5.

    Article  Google Scholar 

  • Tan, H. C., Anumba, C. J., Carrillo, P. M., Bouchlaghem, D., Kamara, J., & Udeaja, C. (2009). Capture and reuse of project knowledge in construction. Chichester: Wiley.

    Google Scholar 

  • The Inspectorate (2013). Promoting the quality of learning, Chief Inspector’s Report 2010–2011. Dublin: DES.

  • The Inspectorate. (2016). Inspectorate publications [Online]. Dublin: DES Available: http://www.education.ie/en/Publications/Inspection-Reports-Publications/Evaluation-Reports-Guidelines/. Accessed 01/10/2016.

    Google Scholar 

  • Tusla. (2016). Research and Statistics [Online]. Dublin: TUSLA Available: http://www.tusla.ie/services/educational-welfare-services/publications/research-and-statistics. Accessed 01/10/2016.

    Google Scholar 

  • Valli, L., Croninger, R. G., & Walters, K. (2007). Who (else) is the teacher? Cautionary notes on teacher accountability systems. American Journal of Education, 113, 635–662.

    Article  Google Scholar 

  • Visscher, A. J., & Coe, R. (2013). School improvement through performance feedback. London: Routledge.

    Google Scholar 

  • Wayman, J. C., & Stringfield, S. (2006). Data use for school improvement: school practices and research perspectives. American Journal of Education, 112, 463–468.

    Article  Google Scholar 

  • Wayman, J. C., Midgley, S., & Stringfield, S. (2006). Leadership for data-based decision-making: Collaborative educator teams. In A. B. Danzig (Ed.), Learner Centered leadership: Research, policy, and practice (pp. 189–206). Mahwah: Lawrence Erlbaum Associates.

    Google Scholar 

  • Wayman, J. C., Jimerson, J. B., & Cho, V. (2012b). Organizational considerations in establishing the data-informed district. School Effectiveness and School Improvement, 23, 159–178.

    Article  Google Scholar 

  • Wayman, J. C., Spring, S. D., Lemke, M. A., & Lehr, M. D. (2012c). Using data to inform practice: effective principal leadership strategies. Annual Meeting of the American Educational Research Association. Vancouver: AERA.

    Google Scholar 

  • Wohlstetter, P., Datnow, A., & Park, V. (2008). Creating a system for data-driven decision-making: applying the principal-agent framework. School Effectiveness and School Improvement, 19, 239–259.

    Article  Google Scholar 

  • Wrigley, T. (2013). Rethinking school effectiveness and improvement: a question of paradigms. Discourse: Studies in the Cultural Politics of Education, 34, 31–47.

    Google Scholar 

  • Yoon, S. Y. (2016). Principals’ data-driven practice and its influences on teacher buy-in and student achievement in comprehensive school reform models. Leadership and Policy in Schools, 15, 500–523.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. EQI – The centre for Evaluation, Quality and Inspection and DCU Institute of Education, Room F217, DCU St. Patrick’s Campus, Drumcondra, Dublin 9, Ireland

    Cornelius Young, Gerry McNamara, Martin Brown & Joe O’Hara

Authors
  1. Cornelius Young
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gerry McNamara
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Martin Brown
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Joe O’Hara
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cornelius Young.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Young, C., McNamara, G., Brown, M. et al. Adopting and adapting: school leaders in the age of data-informed decision making. Educ Asse Eval Acc 30, 133–158 (2018). https://doi.org/10.1007/s11092-018-9278-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11092-018-9278-4

Keywords

  • Assessment
  • Evaluation
  • Data-informed decision making
  • Leadership
  • School inspection
  • School self-evaluation