Skip to main content

Advertisement

SpringerLink
Log in

A test-retest analysis of the Vanderbilt Assessment for Leadership in Education in the USA

  • Published:
Educational Assessment, Evaluation and Accountability Aims and scope Submit manuscript

Abstract

The Vanderbilt Assessment for Leadership in Education (VAL-ED) is a 360-degree learning-centered behaviors principal evaluation tool that includes ratings from the principal, supervisors, and teachers. The current study assesses the test-retest reliability of the VAL-ED for a sample of seven school districts as part of multiple validity and reliability assessments based on various samples of real users of the VAL-ED. We administered the VAL-ED twice and examined the correlations and mean differences between time 1 and time 2. We find that the principal and teacher ratings from time 1 and time 2 have large, positive, and significant correlations. Additionally, for both time points, principals are rated as being at least satisfactorily effective. Principals rate themselves slightly higher at time 2, while teachers rate principals slightly higher at time 1.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. The total principal effectiveness ratings for time 1 for those principals who completed the time 2 rating (3.66) were higher compared to those who did not complete the time 2 assessment (3.52), but the differences are not significant.

  2. In the discussion of the magnitude of correlations, we use the rules of thumb from Cohen (1988) [see Fan (2001) for an application] where a r 2 of 0.02 is a small correlation, r 2 of 0.12 is a medium correlation, and r 2 of 0.25 is a large correlation.

References

  • American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (1999). Standards for educational and psychological testing. Washington: American Educational Research Association.

    Google Scholar 

  • Bryk, A. S., Sebring, P. B., Allensworth, E., Luppescu, S., & Easton, J. Q. (2010). Organizing schools for improvement: lessons from Chicago. Chicago: University of Chicago Press.

    Google Scholar 

  • Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Chicago: Rand McNally.

    Google Scholar 

  • Carmine, E., & Zeller, R. (1979). Reliability and validity assessment. Thousand Oaks: Sage Publications, Inc..

    Book  Google Scholar 

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale: Lawrence Erlbaum Associates, Inc..

    Google Scholar 

  • Condon, C. & Clifford, M. (2012). Measuring principal performance: How rigorous are commonly used performance assessment instruments? Retrieved from http://www.air.org/sites/default/files/downloads/report/Measuring_Principal_Performance_0.pdf.

  • Covay Minor, E., Porter, A., Murphy, J., Goldring, E., Cravens, X., & Elliott, S. (2014). A known group analysis study of the vanderbilt assessment of leadership in education in US elementary and secondary schools. Educational Assessment, Evaluation and Accountability, 26, 29–48.

    Article  Google Scholar 

  • Cravens, X., Goldring, E., Porter, A., Polikoff, M., Murphy, J., & Elliott, S. (2013). Standard setting for principal leadership assessment: a deliberative process. Educational Administration Quarterly, 49(1), 124–160.

    Article  Google Scholar 

  • Elliott, S. N., Murphy, J., Goldring, E., & Porter, A. (2008). VAL -ED users’ guide. Nashville: Discovery Education Assessments.

    Google Scholar 

  • Fan, X. (2001). Statistical significance and effect size in education research: two sides of a coin. The Journal of Educational Research, 94(5), 275–282.

    Article  Google Scholar 

  • Goldring, E., Porter, A. C., Murphy, J., & Elliott, S. (2009a). Assessing learning-centered leadership: connections to research, professional standards, and current practice. Leadership and Policy in Schools, 8(1), 1–36.

    Article  Google Scholar 

  • Goldring, E., Cravens, X., Murphy, J., Elliott, S., Porter, A., & Carson, B. (2009b). The evaluation of principals: what and how do states and urban districts assess? The Elementary School Journal, 110(1), 19–32.

    Article  Google Scholar 

  • Goldring, E., Cravens, X., Murphy, J., Porter, A., & Elliott, S. (2015). The convergent and divergent validity of the Vanderbilt Assessment of Leadership in Education (VAL-ED): instructional leadership and emotional intelligence. Journal of Educational Administration, 53(2), 177–196.

    Article  Google Scholar 

  • Hallinger, P. (2011). A review of three decades of doctoral studies using the Principal Instructional Management Rating Scale: a lens on methodological progress in educational leadership. Educational Administration Quarterly, 4, 271–306.

    Article  Google Scholar 

  • Hallinger, P., & Murphy, J. (1985). Assessing the instructional management behavior of principals. Elementary School Journal, 86(2), 217–247.

    Article  Google Scholar 

  • Hallinger, P., Wang, W.-C., & Chen, C.-W. (2013). Assessing the measurement properties of the Principal Instructional Management Rating Scale: a meta-analysis of reliability studies. Educational Administration Quarterly, 49, 272–309.

    Article  Google Scholar 

  • Leithwood, K., & Montgomery, D. J. (1982). The role of the elementary school principal in program improvement. Review of Educational Research., 52(3), 309–339.

    Article  Google Scholar 

  • Leithwood, K., Jantzi, D., & McElheron-Hopkins, C. (2006, Feb). The development and testing of a school improvement model. School Effectiveness and School Improvement, 17(4), 441–464.

    Article  Google Scholar 

  • Leithwood, K., Louis, K. S., Anderson, S., & Wahlstrom, K. (2004). Review of research: how leadership influences student learning. Minneapolis: Center for Applied Research and Educational Improvement, University of Minnesota.

    Google Scholar 

  • Leithwood, K., & Sun, J. (2012). The nature and effects of transformational school leadership: a meta-analytic review of unpublished research. Educational Administration Quarterly, 48(3), 387–423.

    Article  Google Scholar 

  • Marks, H. M., & Printy, S. M. (2003). Principal leadership and school performance: an integration of transformational and instructional leadership. Educational Administration Quarterly, 39(3), 370–397.

    Article  Google Scholar 

  • May, H., & Supovitz, J. A. (2011). The scope of principal efforts to improve instruction. Educational Administration Quarterly., 47(2), 332–352.

    Article  Google Scholar 

  • Murphy, J. (1989). Educational reform in the 1980s: explaining some surprising success. Educational Evaluation and Policy Analysis, 11(3), 209.

    Article  Google Scholar 

  • Murphy, J. (1992). School effectiveness and school restructuring: contributions to educational improvement. School Effectiveness and School Improvement, 3(2), 90–109.

    Article  Google Scholar 

  • Murphy, J., Hallinger, P., Weil, M., Mitman, A., & 3. (1983). Problems with research on educational leadership: issues to be addressed. Educational Evaluation and Policy Analysis, 5, 297–305.

    Article  Google Scholar 

  • Murphy, J., Hallinger, P., & Mesa, R. P. (1985). School effectiveness: checking progress and assumptions and developing a role for state and federal government. Teachers College Record, 86(4), 615–641.

    Google Scholar 

  • Murphy, J., Elliott, S., Goldring, E., & Porter, A. C. (2007). Leadership for learning: a research-based model and taxonomy of behaviors. School Leadership & Management, 27(2), 179–201.

    Article  Google Scholar 

  • Murphy, J., Goldring, E., & Porter, A. C. (2012). Building productive principal evaluation systems. Nashville: Vanderbilt University, VALED.

    Google Scholar 

  • National Instrument. (2006). Improving accuracy through averaging. Retrieved from http://www.ni.com/white-paper/3488/en/.

  • Petrides, K. V., Pérez-González, J. C., & Furnham, A. (2007). On the criterion and incremental validity of trait emotional intelligence. Cognition and Emotion, 21, 26–55.

    Article  Google Scholar 

  • Polikoff, M. S., May, H., Porter, A. C., Elliott, S. N., Goldring, E., & Murphy, J. (2009). An examination of differential item functioning on the Vanderbilt Assessment of Leadership in Education. Journal of School Leadership, 19(6), 661–679.

    Google Scholar 

  • Porter, A. C., Murphy, J., Goldring, E., Elliott, S. N., Polikoff, M. S., & May, H. (2008). VAL - ED technical manual. Nashville: Vanderbilt University.

    Google Scholar 

  • Porter, A. C., Polikoff, M. S., Goldring, E. B., Murphy, J., Elloitt, S. N., & May, H. (2010). Investigating the validity and reliability of the Vanderbilt Assessment of Leadership in Education. The Elementary School Journal, 111(2), 282–313.

    Article  Google Scholar 

  • Preston, C., Goldring, E., Guthrie, J., & Ramsey, R. (2012). Conceptualizing essential components of effective high schools. Nashville: Paper presented at the National Conference on Achieving Success at Scale: Research in Scaling Up Effective Schools.

    Google Scholar 

  • Robinson, V. M. J., Lloyd, C. A., & Rowe, K. J. (2008). The impact of leadership on student outcomes: an analysis of the differential effects of leadership types. Educational Administration Quarterly, 44(5), 635–674.

    Article  Google Scholar 

  • Supovitz, J., Sirinides, P., & May, H. (2009). How principals and peers influence teaching and learning. Educational Administration Quarterly, 46(1), 31–56.

    Article  Google Scholar 

Download references

Acknowledgements

The research reported here was supported by the Institute of Education Sciences, US Department of Education through grant numbers R305A0803070, R305B100013-01, and R305E100008 of the US Department of Education. Please direct all correspondence to Elizabeth Covay Minor (eminor1@nl.edu).

Author information

Authors and Affiliations

  1. National Louis University, 1000 Capitol Drive, Wheeling, IL, 60090, USA

    Elizabeth Covay Minor

  2. University of Pennsylvania, 3700 Walnut Street, Philadelphia, PA, 19104, USA

    Elizabeth Covay Minor & Andrew C. Porter

  3. Michigan State University, East Lansing, MI, USA

    Elizabeth Covay Minor

  4. Vanderbilt University, Peabody College, Nashville, TN, 37203, USA

    Joseph Murphy & Ellen Goldring

  5. Sanford School of Social and Family Dynamics, Arizona State University, 308K Payne Hall East, Tempe, AZ, 85287-7805, USA

    Stephen N. Elliott

Authors
  1. Elizabeth Covay Minor
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrew C. Porter
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Joseph Murphy
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ellen Goldring
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Stephen N. Elliott
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Elizabeth Covay Minor.

Ethics declarations

Conflict of interest

The authors declare a potential conflict of interest (e.g., a financial relationship with the commercial organizations or products discussed in this article) as follows: The Vanderbilt Assessment of Leadership in Education (VAL-ED) instrument is authored by Drs. Porter, Murphy, Goldring, and Elliott and copyrighted by Vanderbilt University, all of whom receive a royalty from its sales by Discovery Education Assessment. The VAL-ED authors and their research partners have made every effort to be objective and data based in statements about the instrument and value the independent peer review process of their research. With any publication, readers should judge the facts and related materials for themselves.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Minor, E.C., Porter, A.C., Murphy, J. et al. A test-retest analysis of the Vanderbilt Assessment for Leadership in Education in the USA. Educ Asse Eval Acc 29, 211–224 (2017). https://doi.org/10.1007/s11092-016-9254-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11092-016-9254-9

Keywords

Search

Navigation