Abstract
The Vanderbilt Assessment of Leadership in Education (VAL-ED) provides educators with a tool for principal evaluation based on principal, teacher, and supervisor reports of principals’ learning-centered leadership. In this study, we conduct a known group analysis as part of a larger argument for the validity of the VAL-ED in US elementary and secondary schools. We asked superintendents to select the principals in their district who they believe in performance of their duties are in the top 20 % and the bottom 20 %. We ask how accurately VAL-ED scores can identify membership of the two known groups. Using a discriminant analysis, the VAL-ED places principals in the superintendent groups, on average, 70 % of the time for both elementary and secondary schools. Placement accuracy is greater for the top group than the bottom group.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
We define principal learning-centered leadership as intentional mediated actions focused on student learning.
We wish to acknowledge the generous support of the Wallace Foundation whose grant to Vanderbilt University made this research-anchored development work possible.
We recognize that it is possible that the superintendent was aware of the VAL-ED scores prior to categorizations, especially for the three districts with VAL-ED scores from 2009 to 2010. These three districts do have higher accuracy rates than the other three districts. Since the districts with VAL-ED scores from 2009 to 2010 are the three smallest districts, it is possible that the superintendent has a better sense of the principals’ effectiveness. However, to control for the possibility that the superintendent used the VAL-ED scores to place principals in the two groups and as a robustness test, we run the analyses without those districts. We reach the same conclusions from our results.
References
American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
Atwater, L. E., & Yammarino, F. J. (1992). Does self-other agreement on leadership perceptions moderate the validity of leadership and performance predictions? Personnel Psychology, 45, 141–164.
Bossert, S., Dwyer, D., Rowan, B., & Lee, G. (1982). The instructional management role of the principal. Educational Administration Quarterly, 18(3), 34–64.
Brookover, W. B., & Lezotte, L. W. (1977). Changes in school characteristics coincident with changes in student achievement. East Lansing: College of Urban Development, Michigan State University.
Burns, R., & Burns, R. (2008). Business research methods and statistics using SPSS. London: Sage Publications Ltd.
Bryk, A. S., Sebring, P. B., Allensworth, E., Luppescu, S., & Easton, J. Q. (2010). Organizing schools for improvement: Lessons from Chicago. Chicago: University of Chicago Press.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
Cravens, X., Goldring, E., Porter, A., Polikoff, M., Murphy, J., & Elliott, S. (2013). Standard setting for principal leadership assessment: a deliberative process. Educational Administration Quarterly, 49(1), 124–160.
Edmonds, R., & Frederiksen, J. R. (1978). Search for effective schools: The identification and analysis of city schools that are instructionally effective for poor children. Cambridge, MA: Center for Urban Studies, Harvard University.
Elliott, S. N., Murphy, J., Goldring, E., & Porter, A. (2008). VAL-ED users’ guide. Nashville: Discovery Education Assessments.
Goldring, E., Cravens, X., Murphy, J., Elliott, S., Porter, A., & Carson, B. (2009b). The evaluation of principals: what and how do states and urban districts assess? Elementary School Journal, 110(1), 19–32.
Goldring, E., Cravens, X., Porter, A. C., Murphy, J., & Elliott, S. N. (2013). Instructional leadership and emotional intelligence: the convergent and divergent validity of the Vanderbilt Assessment of Leadership in Education Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA.
Goldring, E., Porter, A. C., Murphy, J., & Elliott, S. (2009a). Assessing learning-centered leadership: connections to research, professional standards, and current practice. Leadership and Policy in Schools, 8(1), 1–36.
Hallinger, P., & Heck, R. (1996). Reassessing the principal’s role in school effectiveness: A review of empirical research, 1980–1995. Educational Administration Quarterly, 32(1), 5–44 (JM).
Hallinger, P., & Heck, R. (1998). Exploring the principal’s contribution to school effectiveness: 1980–1995. School Effectiveness and School Improvement, 9(2), 157–191.
Hallinger, P., & Murphy, J. (1985, November). Assessing the instructional management behavior of principals. Elementary School Journal, 86(2), 217–247.
IBM Corporation. (2011). Standardized canonical discriminant function coefficients. In IBM SPSS Statistics (Version 20) [Computer Software].
Kane, M. T. (1992). An argument-based approach to validity. Psychological Bulletin, 112(3), 527–535.
Kane, M. T. (2001). Current concerns in validity theory. Journal of Educational Measurement, 38(4), 319–342.
Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), 1121–1134.
Leithwood, K., & Montgomery, D. J. (1982). The role of the elementary school principal in program improvement. Review of Educational Research., 52(3), 309–339.
Leithwood, K., Jantzi, D., & McElheron-Hopkins, C. (2006). The development and testing of a school improvement model. School Effectiveness and School Improvement, 17(4), 441–464.
Leithwood, K., Louis, K. S., Anderson, S., & Wahlstrom, K. (2004). Review of research: How leadership influences student learning. Minneapolis, MN: Center for Applied Research and Educational Improvement, University of Minnesota.
Leithwood, K., Patten, S., & Jantzi, D. (2010). Testing a conception of how school leadership influences student learning. Educational Administration Quarterly, 46(5), 671–706.
Marks, H. M., & Printy, S. M. (2003). Principal leadership and school performance: An integration of transformational and instructional leadership. Educational Administration Quarterly, 39(3), 370–397.
May, H., & Supovitz, J. A. (2011). The scope of principal efforts to improve instruction. Educational Administration Quarterly., 47(2), 332–352.
Murphy, J. (1989, Fall). Educational Reform in the 1980s: explaining some surprising success. Educational Evaluation and Policy Analysis, 11(3), 209–223.
Murphy, J. (1992). School effectiveness and school restructuring: contributions to educational improvement. School Effectiveness and School Improvement, 3(2), 90–109.
Murphy, J. (2013). The architecture of school improvement. Journal of Educational Administration, 51(3), 252–263. [Invited Article].
Murphy, J., & Shipman, N. J. (1999, September). The Interstate School Leaders Licensure Consortium: a standards-based approach to strengthening educational leadership. Journal of Personnel Evaluation in Education, 13(3), 205–224.
Murphy, J., Elliott, S. N., Goldring, E., & Porter, A. (2007, April). Leadership for learning: a research-based model and taxonomy of behaviors. School Leadership & Management, 27(2), 179–201.
Murphy, J., Goldring, E., & Porter A. C. (2012). Building productive principal evaluation systems. Nashville: Vanderbilt University, VALED.
Murphy, J., Hallinger, P., & Mesa, R. P. (1985, Summer). School effectiveness: checking progress and assumptions and developing a role for state and federal government. Teachers College Record, 86(4), 615–641.
Murphy, J., Hallinger, P., Weil, M., & Mitman, A. (1983a, Fall). Instructional leadership. A conceptual framework. Planning and Changing, 14(3), 137–149.
Murphy, J., Hallinger, P., Weil, M., & Mitman, A. (1983b, Fall). Problems with research on educational leadership: issues to be addressed. Educational Evaluation and Policy Analysis, 5(3), 297–305.
Murphy, J., Yff, J., & Shipman, N. J. (2000, January-March). Implementation of the interstate school leaders licensure consortium standards. The International Journal of Leadership in Education, 3(1), 17–39.
Norusis, M. (2003). SPSS 12.0 Statistical Procedures Companion. Upper Saddle River, NJ: Prentice Hall.
Polikoff, M. S., May, H., Porter, A. C., Elliott, S. N., Goldring, E., & Murphy, J. (2009, November). An examination of differential item functioning on the Vanderbilt Assessment of Leadership in Education. Journal of School Leadership, 19(6), 661–679.
Porter, A. C., Murphy, J., Goldring, E., Elliott, S. N., Polikoff, M. S., & May, H. (2008). Vanderbilt assessment of leadership in education: technical manual. Retrieved from http://www.valed.com/documents/VAL-ED_TechManual_030509.pdf.
Porter, A. C., Polikoff, M. S., Goldring, E., Murphy, J., Elliott, S. N., & May, H. (2010a, April). Developing a psychometrically sound assessment of school leadership: the VAL-ED as a case study. Educational Administration Quarterly, 46(2), 135–173.
Porter, A. C., Polikoff, M. S., Goldring, E. B., Murphy, J., Elloitt, S. N. & May, H. (2010b). Investigating the validity and reliability of the Vanderbilt Assessment of Leadership in Education. The Elementary School Journal, 111(2), 282–313.
Preston, C., Goldring, E., Guthrie, J., & Ramsey, R. (2012). Conceptualizing essential components of effective high schools. Paper presented at the National Conference on Achieving Success at Scale: research in scaling up effective schools. Nashville, TN.
Robinson, V. M. J., Lloyd, C. A., & Rowe, K. J. (2008). The impact of leadership on student outcomes: An analysis of the differential effects of leadership types. Educational Administration Quarterly, 44(5), 635–674.
Supovitz, J., Sirinides, P., & May, H. (2009). How principals and peers influence teaching and learning. Educational Administration Quarterly, 46(1), 31–56.
Witziers, B., Bosker, R. J., & Krüger, M. L. (2003). Educational leadership and student achievement: The elusive search for an association. Educational Administration Quarterly, 39(3), 398–425 (JM).
Young, M. D., Crow, G., Murphy J., Ogawa, R. (Eds.). (2009). Handbook of research on the education of school leaders. New York: Routledge.
Acknowledgments
The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through grants # R305A0803070, R305B100013-01, and R305E100008 of the U.S. Department of Education. The authors declare a potential conflict of interest (e.g., a financial relationship with the commercial organizations or products discussed in this article) as follows: The Vanderbilt Assessment of Leadership in Education (VAL-ED) instrument is authored by Drs. Porter, Murphy, Goldring, and Elliott and copyrighted by Vanderbilt University, all of whom receive a royalty from its sales by Discovery Education Assessment. The VAL-ED authors and their research partners have made every effort to be objective and data based in statements about the instrument and value the independent peer review process of their research. With any publication, readers should judge the facts and related materials for themselves.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Covay Minor, E., Porter, A.C., Murphy, J. et al. A known group analysis validity study of the Vanderbilt Assessment of Leadership in Education in US elementary and secondary schools. Educ Asse Eval Acc 26, 29–48 (2014). https://doi.org/10.1007/s11092-013-9180-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11092-013-9180-z