Advertisement

How effective is the principal? Discrepancy between New Zealand teachers’ and principals’ perceptions of principal effectiveness

  • Claire E. L. Sinnema
  • Viviane M. J. Robinson
  • Larry Ludlow
  • Denyse Pope
Article

Abstract

Multi-source evaluation of school principals is likely to become increasingly common in education contexts as the evidence accumulates about the relationship between principal effectiveness and student achievement. The purpose of this study was to examine (1) the magnitude and direction of discrepancy between how principals and their teachers perceive the principal’s effectiveness and (2) what predicts principals who are at risk because their self-ratings considerably exceed the ratings others give them. We also investigated the appropriateness of various probability cut levels in analyses to predict overrating principals. The data sources were ratings by New Zealand principals (n = 135) and their teachers (n = 2757) of principal effectiveness—one scale (16 items) of an educational leadership practices survey. On average, both groups rated principals highly, and teachers tended to rate their principal higher than the principals rated themselves. There was more variance in teachers’ ratings than principals’ ratings. The variables of principal age (younger), time in principal role at the school (shorter), and socio-economic status of the school (lower) were all associated with greater magnitudes of discrepancy. Such discrepancies have implications for principals’ evaluations, principal development efforts, and for school improvement.

Keywords

Leadership Principal effectiveness Educational leadership Self-other agreement Discrepancy 

Notes

Acknowledgments

We wish to thank Andrew Porter (University of Pennsylvania), Joseph Murphy (Vanderbilt University), Ellen Goldring (Vanderbilt University), and Stephen N. Elliott (Arizona State University), the authors of the Vanderbilt Assessment of Leadership in Education, for use of portions of the Vanderbilt Assessment of Leadership in Education (VAL-ED) structure and other items from the VAL-ED.

References

  1. Atwater, L. E., Ostroff, C., Yammarino, F. J., & Fleenor, J. W. (1998). Self-other agreement: Does it really matter? Personnel Psychology, 51(3), 577–598. doi: 10.1111/j.1744-6570.1998.tb00252.x.CrossRefGoogle Scholar
  2. Atwater, L. E., & Waldman, D. (1998). 360 degree feedback and leadership development. The Leadership Quarterly, 9(4), 423–426. doi: 10.1016/S1048-9843(98)90009-1.CrossRefGoogle Scholar
  3. Atwater, L. E., & Yammarino, F. J. (1992). Does self–other agreement on leadership perceptions moderate the validity of leadership and performance predictions? Personnel Psychology, 45, 141–164. doi: 10.1111/j.1744-6570.1992.tb00848.x.CrossRefGoogle Scholar
  4. Atwater, L. E., & Yammarino, F. J. (1997). Self–other rating agreement: A review and model. In G. R. Ferris (Ed.), Research in personnel and human resources management (pp. 121–174). Greenwich, CT: JAI Press.Google Scholar
  5. Avolio, B. J., Mhatre, K., Norman, S. M., & Lester, P. (2009). The moderating effect of gender on leadership intervention impact: An exploratory review. Journal of Leadership & Organizational Studies, 15(4), 325–341. doi: 10.1177/1548051809333194.CrossRefGoogle Scholar
  6. Brett, J. F., & Atwater, L. E. (2001). 360° feedback: accuracy, reactions, and perceptions of usefulness. Journal of Applied Psychology, 86(5), 930–942. doi: 10.1037/0021-9010.86.5.930.CrossRefGoogle Scholar
  7. Brutus, S., Fleenor, J. W., & McCauley, C. D. (1999). Demographic and personality predictors of congruence in multi-source ratings. Journal of Management Development, 18, 417–435. doi: 10.1108/02621719910273569#sthash.dkq9ZYtV.dpuf.CrossRefGoogle Scholar
  8. Bryk, A. S., & Schneider, B. L. (2002). Trust in schools: A core resource for improvement. New York: Russell Sage.Google Scholar
  9. Bryk, A. S., Sebring, P. B., Allensworth, E., Luppescu, S., & Easton, J. Q. (2010). Organizing schools for improvement: Lessons from Chicago. Chicago: University of Chicago Press.Google Scholar
  10. Church, A. H. (1997). Do you see what I see? An exploration of congruence in ratings from multiple perspectives. Journal of Applied Psychology, 27(11), 983–1020. doi: 10.1111/j.1559-1816.1997.tb00283.x.Google Scholar
  11. Daubman, K. A., Heatherington, L., & Ahn, A. (1992). Gender and the self-presentation of academic achievement. Sex Roles, 27, 187–204. doi: 10.1007/BF00290017.CrossRefGoogle Scholar
  12. Day, C., Sammons, P., Leithwood, K., Hopkins, D., Gu, Q., Brown, E., & Ahtaridou, E. (2011). Successful school leadership: Linking with learning and achievement. Maidenhead: McGraw Hill Open University Press.Google Scholar
  13. Edwards, J. R. (1993). Problems with the use of profile similarity indices in the study of congruence in organizational research. Personnel Psychology, 46, 641–665. doi: 10.1111/j.1744-6570.1993.tb00889.x.CrossRefGoogle Scholar
  14. Edwards, J. R. (2001). Ten difference score myths. Organizational Research Methods, 4(3), 265–287. doi: 10.1177/109442810143005.CrossRefGoogle Scholar
  15. Fleenor, J. W., Smither, J. W., Atwater, L. E., Braddy, P. W., & Sturm, R. E. (2010). Self–other rating agreement in leadership: A review. The Leadership Quarterly, 21, 1005–1034. doi: 10.1016/j.leaqua.2010.10.006.CrossRefGoogle Scholar
  16. Funder, D. C. (1995). On the accuracy of personality judgment: A realistic approach. Psychological Review, 102, 652–670.CrossRefGoogle Scholar
  17. Goddard, R. D., Tschannen-Moran, M., & Hoy, W. K. (2001). A multilevel examination of the distribution and effects of teacher trust in students and parents in urban elementary schools. The Elementary School Journal, 102(1), 3–17. doi: 10.2307/1002166.CrossRefGoogle Scholar
  18. Goldring, E., Cravens, X. C., Murphy, J., Porter, A. C., Elliott, S. N., & Carson, B. (2009). The evaluation of principals: What and how do states and urban districts assess leadership? The Elementary School Journal, 110(1). doi: 10.1086/598841.
  19. Goldring, E., & Goff, P. T. (2010). Understanding feedback from teachers on leadership effectiveness: Do teachers and principals agree? Paper presented at the American Educational Research Association (AERA), Denver.Google Scholar
  20. Grissom, J. A., & Loeb, S. (2011). Triangulating principal effectiveness: How perspectives of parents, teachers, and assistant principals identify the central importance of managerial skills. American Educational Research Journal, 48(5), 1091–1123. doi: 10.3102/0002831211402663.CrossRefGoogle Scholar
  21. Halverson, S. K., Tonidandel, S., Barlow, C. B., & Dipboye, R. L. (2005). Self–other agreement on a 360-degree leadership evaluation. In S. Reddy (Ed.), Multi-source performance assessment: Perspective and Insights. Hyderabad: ICFAI University Press.Google Scholar
  22. Heck, R. H. (1992). Principals’ instructional leadership and school performance: Implications for policy development. Educational Evaluation and Policy Analysis, 14(1), 21–34. doi: 10.3102/01623737014001021.CrossRefGoogle Scholar
  23. Heck, R. H., & Hallinger, P. (2010). Testing a longitudinal model of distributed leadership effects on school improvement. The Leadership Quarterly, 21(5), 867–885. doi: 10.1016/j.leaqua.2010.07.013.CrossRefGoogle Scholar
  24. Heck, R. H., Marcoulides, G. A., & Lang, P. (1991). Principal instructional leadership and school achievement: The application of discriminant techniques. School Effectiveness and School Improvement, 2(2), 115–135. doi: 10.1080/0924345910020204.CrossRefGoogle Scholar
  25. Hosmer, D. W., & Lemeshow, S. (1989). Applied logistic regression. New York: Wiley.Google Scholar
  26. Hu, L. T., & Bentler, P. M. (1999). Cut-off criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. doi: 10.1080/10705519909540118.CrossRefGoogle Scholar
  27. Kenny, D. A., & West, T. V. (2010). Similarity and agreement in self-and other perception: A meta-analysis. Personality and Social Psychology Review, 14(2), 196–213. doi: 10.1177/1088868309353414.CrossRefGoogle Scholar
  28. Kwan, V. S. Y., John, O. P., Robins, R. W., & Kuang, L. L. (2008). Conceptualizing and assessing self-enhancement bias: A componential approach. Journal of Personality and Social Psychology, 94, 1062–1077. doi: 10.1037/0022-3514.94.6.1062.CrossRefGoogle Scholar
  29. Latham, G. P., & Locke, E. A. (2006). New directions in goal-setting theory. Current Directions in Psychological Science, 15(5), 265–268. doi: 10.1111/j.1467-8721.2006.00449.x.CrossRefGoogle Scholar
  30. Latham, G. P., & Locke, E. A. (2007). New developments in and directions for goal-setting research. European Psychologist, 12(4), 290–300. doi: 10.1027/1016-9040.12.4.290.CrossRefGoogle Scholar
  31. Le Fevre, D., & Robinson, V. M. J. (2014). The interpersonal challenges of instructional leadership: Principals’ effectiveness in conversations about performance issues. Educational Administration Quarterly, 1–38. doi: 10.1177/0013161X13518218.
  32. Le Fevre, D., Robinson, V. M. J., & Sinnema, C. E. L. (2014). Genuine inquiry: Widely espoused yet rarely enacted. Education Management and Leadership. doi: 10.1177/1741143214543204.
  33. Locke, E. A., & Latham, G. P. (1990). A theory of goal setting & task performance. Englewood Cliffs: Prentice-Hall, Inc.Google Scholar
  34. Menard, S. (1995). Applied logistic regression analysis. Thousand Oaks: Sage.Google Scholar
  35. Minor, E. C., Porter, A. C., Murphy, J., Goldring, E., Cravens, X., & Elliott, S. N. (2014). A known group analysis validity study of the Vanderbilt Assessment of Leadership in Education in US elementary and secondary schools. Educational Assessment, Evaluation and Accountability, 26(1), 29–48. doi: 10.1007/s11092-013-9180-z.CrossRefGoogle Scholar
  36. Murphy, J. (2013). The architecture of school improvement. Journal of Educational Administration, 51(3), 252–263. doi: 10.1108/09578231311311465#sthash.BUnYQyul.dpuf.CrossRefGoogle Scholar
  37. Ostroff, C., Atwater, L. E., & Feinberg, B. J. (2004). Understanding self–other agreement: A look at rater and ratee characteristics, context, and outcomes. Personnel Psychology, 57, 333–375. doi: 10.1111/j.1744-6570.2004.tb02494.x.CrossRefGoogle Scholar
  38. Paulhus, D. L., & Bruce, M. N. (1992). The effect of acquaintanceship on the validity of personality impressions: A longitudinal study. Journal of Personality and Social Psychology, 63(5), 816–824. doi: 10.1037/0022-3514.63.5.816.CrossRefGoogle Scholar
  39. Pont, B., Figueroa, D. T., Zapata, J., & Fraccola, S. (2013). Education outlook: New Zealand (pp. 1–24).Google Scholar
  40. Reeves, D. B. (2008). Assessing educational leaders: Evaluating performance for improved individual and organizational results (2nd ed.). Thousand Oaks: Corwin Press.Google Scholar
  41. Robinson, V., Lloyd, C., & Rowe, K. J. (2008). The impact of leadership on student outcomes: An analysis of the differential effects of leadership types. Educational Administration Quarterly, 44(5), 635–674. doi: 10.1177/0013161X08321509.CrossRefGoogle Scholar
  42. Robinson, V., & Timperley, H. S. (2007). The leadership of the improvement teaching and learning: Lessons from initiatives with positive outcomes for students. Australian Journal of Education, 51(3), 247–262. doi: 10.1177/000494410705100303.CrossRefGoogle Scholar
  43. Robinson, V. M. J. (2011). Student-centered leadership. San Francisco: Jossey-Bass.Google Scholar
  44. Robinson, V. M. J., & Le Fevre, D. (2011). Principals’ capability in challenging conversations: the case of parental complaints. Journal of Educational Administration, 49(3), 227–255. doi: 10.1108/09578231111129046.
  45. Scheffer, J. (2002). Dealing with missing data. Research Letters in the Information and Mathematical Sciences, 3, 153–160.Google Scholar
  46. Sinnema, C. E. L., Le Fevre, D., Robinson, V. M. J., & Pope, D. (2013). When others’ performance just isn’t good enough: Educational leaders’ framing of concerns in private and public. Leadership and Policy in Schools, 12(4), 301–336. doi: 10.1080/15700763.2013.857419.CrossRefGoogle Scholar
  47. Sinnema, C. E. L., & Robinson, V. M. J. (2007). The leadership of teaching and learning: Implications for teacher evaluation. Leadership and Policy in Schools, 6(4), 319–343. doi: 10.1080/15700760701431603.CrossRefGoogle Scholar
  48. Sinnema, C. E. L., & Robinson, V. M. J. (2012). Goal setting in principal evaluation: Goal quality and predictors of achievement. Leadership and Policy in Schools, 11(2), 135–167. doi: 10.1080/15700763.2011.629767.CrossRefGoogle Scholar
  49. Smither, J. W., London, M., & Reilly, R. R. (2005). Does performance improve following multisource feedback? A theoretical model, meta-analysis, and review of empirical findings. Personnel Psychology, 58(1), 33–66. doi: 10.1111/j.1744-6570.2005.514_1.x.CrossRefGoogle Scholar
  50. Sun, M., Youngs, P., Yang, H., Chu, H., & Zhao, Q. (2012). Association of district principal evaluation with learning-centered leadership practice: Evidence from Michigan and Beijing. Educational Assessment, Evaluation and Accountability, 24, 189–213. doi: 10.1007/s11092-012-9145-7.CrossRefGoogle Scholar
  51. Thomas, D. R., & Zumbo, B. D. (2012). Difference scores from the point of view of reliability and repeated-measures ANOVA: In defense of difference scores for data analysis. Educational and Psychological Measurement, 72(1), 37–43. doi: 10.1177/0013164411409929.CrossRefGoogle Scholar
  52. Vecchio, R. P., & Anderson, R. J. (2009). Agreement in self–other ratings of leader effectiveness: The role of demographics and personality. International Journal of Selection and Assessment, 17, 165–179. doi: 10.1111/j.1468-2389.2009.00460.x.CrossRefGoogle Scholar
  53. Visser, B. A., Ashton, M. C., & Vernon, P. A. (2008). What makes you think you’re so smart? Measured abilities, personality, and sex differences in relation to self-estimates of multiple intelligences. Journal of Individual Differences, 29, 35–44. doi: 10.1027/1614-0001.29.1.35.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Claire E. L. Sinnema
    • 1
  • Viviane M. J. Robinson
    • 1
  • Larry Ludlow
    • 2
  • Denyse Pope
    • 1
  1. 1.Faculty of EducationThe University of AucklandAucklandNew Zealand
  2. 2.Lynch School of EducationBoston CollegeChestnut HillUSA

Personalised recommendations