A known group analysis validity study of the Vanderbilt Assessment of Leadership in Education in US elementary and secondary schools

  • Elizabeth Covay Minor
  • Andrew C. Porter
  • Joseph Murphy
  • Ellen B. Goldring
  • Xiu Cravens
  • Stephen N. Elloitt
Article

Abstract

The Vanderbilt Assessment of Leadership in Education (VAL-ED) provides educators with a tool for principal evaluation based on principal, teacher, and supervisor reports of principals’ learning-centered leadership. In this study, we conduct a known group analysis as part of a larger argument for the validity of the VAL-ED in US elementary and secondary schools. We asked superintendents to select the principals in their district who they believe in performance of their duties are in the top 20 % and the bottom 20 %. We ask how accurately VAL-ED scores can identify membership of the two known groups. Using a discriminant analysis, the VAL-ED places principals in the superintendent groups, on average, 70 % of the time for both elementary and secondary schools. Placement accuracy is greater for the top group than the bottom group.

Keywords

Known group study Learning-centered leadership 

References

  1. American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.Google Scholar
  2. Atwater, L. E., & Yammarino, F. J. (1992). Does self-other agreement on leadership perceptions moderate the validity of leadership and performance predictions? Personnel Psychology, 45, 141–164.CrossRefGoogle Scholar
  3. Bossert, S., Dwyer, D., Rowan, B., & Lee, G. (1982). The instructional management role of the principal. Educational Administration Quarterly, 18(3), 34–64.CrossRefGoogle Scholar
  4. Brookover, W. B., & Lezotte, L. W. (1977). Changes in school characteristics coincident with changes in student achievement. East Lansing: College of Urban Development, Michigan State University.Google Scholar
  5. Burns, R., & Burns, R. (2008). Business research methods and statistics using SPSS. London: Sage Publications Ltd.Google Scholar
  6. Bryk, A. S., Sebring, P. B., Allensworth, E., Luppescu, S., & Easton, J. Q. (2010). Organizing schools for improvement: Lessons from Chicago. Chicago: University of Chicago Press.Google Scholar
  7. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.Google Scholar
  8. Cravens, X., Goldring, E., Porter, A., Polikoff, M., Murphy, J., & Elliott, S. (2013). Standard setting for principal leadership assessment: a deliberative process. Educational Administration Quarterly, 49(1), 124–160.Google Scholar
  9. Edmonds, R., & Frederiksen, J. R. (1978). Search for effective schools: The identification and analysis of city schools that are instructionally effective for poor children. Cambridge, MA: Center for Urban Studies, Harvard University.Google Scholar
  10. Elliott, S. N., Murphy, J., Goldring, E., & Porter, A. (2008). VAL-ED users’ guide. Nashville: Discovery Education Assessments.Google Scholar
  11. Goldring, E., Cravens, X., Murphy, J., Elliott, S., Porter, A., & Carson, B. (2009b). The evaluation of principals: what and how do states and urban districts assess? Elementary School Journal, 110(1), 19–32.Google Scholar
  12. Goldring, E., Cravens, X., Porter, A. C., Murphy, J., & Elliott, S. N. (2013). Instructional leadership and emotional intelligence: the convergent and divergent validity of the Vanderbilt Assessment of Leadership in Education Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA.Google Scholar
  13. Goldring, E., Porter, A. C., Murphy, J., & Elliott, S. (2009a). Assessing learning-centered leadership: connections to research, professional standards, and current practice. Leadership and Policy in Schools, 8(1), 1–36.Google Scholar
  14. Hallinger, P., & Heck, R. (1996). Reassessing the principal’s role in school effectiveness: A review of empirical research, 1980–1995. Educational Administration Quarterly, 32(1), 5–44 (JM).CrossRefGoogle Scholar
  15. Hallinger, P., & Heck, R. (1998). Exploring the principal’s contribution to school effectiveness: 1980–1995. School Effectiveness and School Improvement, 9(2), 157–191.CrossRefGoogle Scholar
  16. Hallinger, P., & Murphy, J. (1985, November). Assessing the instructional management behavior of principals. Elementary School Journal, 86(2), 217–247.Google Scholar
  17. IBM Corporation. (2011). Standardized canonical discriminant function coefficients. In IBM SPSS Statistics (Version 20) [Computer Software].Google Scholar
  18. Kane, M. T. (1992). An argument-based approach to validity. Psychological Bulletin, 112(3), 527–535.CrossRefGoogle Scholar
  19. Kane, M. T. (2001). Current concerns in validity theory. Journal of Educational Measurement, 38(4), 319–342.CrossRefGoogle Scholar
  20. Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), 1121–1134.CrossRefGoogle Scholar
  21. Leithwood, K., & Montgomery, D. J. (1982). The role of the elementary school principal in program improvement. Review of Educational Research., 52(3), 309–339.CrossRefGoogle Scholar
  22. Leithwood, K., Jantzi, D., & McElheron-Hopkins, C. (2006). The development and testing of a school improvement model. School Effectiveness and School Improvement, 17(4), 441–464.CrossRefGoogle Scholar
  23. Leithwood, K., Louis, K. S., Anderson, S., & Wahlstrom, K. (2004). Review of research: How leadership influences student learning. Minneapolis, MN: Center for Applied Research and Educational Improvement, University of Minnesota.Google Scholar
  24. Leithwood, K., Patten, S., & Jantzi, D. (2010). Testing a conception of how school leadership influences student learning. Educational Administration Quarterly, 46(5), 671–706.CrossRefGoogle Scholar
  25. Marks, H. M., & Printy, S. M. (2003). Principal leadership and school performance: An integration of transformational and instructional leadership. Educational Administration Quarterly, 39(3), 370–397.CrossRefGoogle Scholar
  26. May, H., & Supovitz, J. A. (2011). The scope of principal efforts to improve instruction. Educational Administration Quarterly., 47(2), 332–352.CrossRefGoogle Scholar
  27. Murphy, J. (1989, Fall). Educational Reform in the 1980s: explaining some surprising success. Educational Evaluation and Policy Analysis, 11(3), 209–223.Google Scholar
  28. Murphy, J. (1992). School effectiveness and school restructuring: contributions to educational improvement. School Effectiveness and School Improvement, 3(2), 90–109.Google Scholar
  29. Murphy, J. (2013). The architecture of school improvement. Journal of Educational Administration, 51(3), 252–263. [Invited Article].Google Scholar
  30. Murphy, J., & Shipman, N. J. (1999, September). The Interstate School Leaders Licensure Consortium: a standards-based approach to strengthening educational leadership. Journal of Personnel Evaluation in Education, 13(3), 205–224.Google Scholar
  31. Murphy, J., Elliott, S. N., Goldring, E., & Porter, A. (2007, April). Leadership for learning: a research-based model and taxonomy of behaviors. School Leadership & Management, 27(2), 179–201.Google Scholar
  32. Murphy, J., Goldring, E., & Porter A. C. (2012). Building productive principal evaluation systems. Nashville: Vanderbilt University, VALED.Google Scholar
  33. Murphy, J., Hallinger, P., & Mesa, R. P. (1985, Summer). School effectiveness: checking progress and assumptions and developing a role for state and federal government. Teachers College Record, 86(4), 615–641.Google Scholar
  34. Murphy, J., Hallinger, P., Weil, M., & Mitman, A. (1983a, Fall). Instructional leadership. A conceptual framework. Planning and Changing, 14(3), 137–149.Google Scholar
  35. Murphy, J., Hallinger, P., Weil, M., & Mitman, A. (1983b, Fall). Problems with research on educational leadership: issues to be addressed. Educational Evaluation and Policy Analysis, 5(3), 297–305.Google Scholar
  36. Murphy, J., Yff, J., & Shipman, N. J. (2000, January-March). Implementation of the interstate school leaders licensure consortium standards. The International Journal of Leadership in Education, 3(1), 17–39.Google Scholar
  37. Norusis, M. (2003). SPSS 12.0 Statistical Procedures Companion. Upper Saddle River, NJ: Prentice Hall.Google Scholar
  38. Polikoff, M. S., May, H., Porter, A. C., Elliott, S. N., Goldring, E., & Murphy, J. (2009, November). An examination of differential item functioning on the Vanderbilt Assessment of Leadership in Education. Journal of School Leadership, 19(6), 661–679.Google Scholar
  39. Porter, A. C., Murphy, J., Goldring, E., Elliott, S. N., Polikoff, M. S., & May, H. (2008). Vanderbilt assessment of leadership in education: technical manual. Retrieved from http://www.valed.com/documents/VAL-ED_TechManual_030509.pdf.
  40. Porter, A. C., Polikoff, M. S., Goldring, E., Murphy, J., Elliott, S. N., & May, H. (2010a, April). Developing a psychometrically sound assessment of school leadership: the VAL-ED as a case study. Educational Administration Quarterly, 46(2), 135–173.Google Scholar
  41. Porter, A. C., Polikoff, M. S., Goldring, E. B., Murphy, J., Elloitt, S. N. & May, H. (2010b). Investigating the validity and reliability of the Vanderbilt Assessment of Leadership in Education. The Elementary School Journal, 111(2), 282–313.Google Scholar
  42. Preston, C., Goldring, E., Guthrie, J., & Ramsey, R. (2012). Conceptualizing essential components of effective high schools. Paper presented at the National Conference on Achieving Success at Scale: research in scaling up effective schools. Nashville, TN.Google Scholar
  43. Robinson, V. M. J., Lloyd, C. A., & Rowe, K. J. (2008). The impact of leadership on student outcomes: An analysis of the differential effects of leadership types. Educational Administration Quarterly, 44(5), 635–674.CrossRefGoogle Scholar
  44. Supovitz, J., Sirinides, P., & May, H. (2009). How principals and peers influence teaching and learning. Educational Administration Quarterly, 46(1), 31–56.CrossRefGoogle Scholar
  45. Witziers, B., Bosker, R. J., & Krüger, M. L. (2003). Educational leadership and student achievement: The elusive search for an association. Educational Administration Quarterly, 39(3), 398–425 (JM).CrossRefGoogle Scholar
  46. Young, M. D., Crow, G., Murphy J., Ogawa, R. (Eds.). (2009). Handbook of research on the education of school leaders. New York: Routledge.Google Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Elizabeth Covay Minor
    • 1
  • Andrew C. Porter
    • 2
  • Joseph Murphy
    • 3
  • Ellen B. Goldring
    • 3
  • Xiu Cravens
    • 3
  • Stephen N. Elloitt
    • 4
  1. 1.College of EducationMichigan State UniversityEast LansingUSA
  2. 2.University of PennsylvaniaPhiladelphiaUSA
  3. 3.Vanderbilt UniversityNashvilleUSA
  4. 4.Arizona State UniversityPhoenixUSA

Personalised recommendations