Advertisement

The Australian Educational Researcher

, Volume 45, Issue 2, pp 255–276 | Cite as

The development of a situational judgement test of personal attributes for quality teaching in rural and remote Australia

  • Tracy L. Durksen
  • Robert M. Klassen
Article

Abstract

Education authorities in Australia are calling for valid tools to help assess prospective teachers’ non-academic attributes, with a particular need for identifying those attributes necessary for effective teaching in specific contexts. With the New South Wales (NSW) Department of Education, we aimed to develop a scenario-based tool to help assess the non-academic attributes necessary for beginning teachers working in rural and remote settings. Using a situational judgement test (SJT) methodology, we worked with experienced teachers (n = 19) to develop scenario-based items, which were then reviewed by school principals (n = 13). The teachers also adapted items previously developed and piloted in the UK. Next, prospective NSW teachers (n = 99) tried 32 new and adapted items targeting four clusters of attributes: empathy and communication, resilience and adaptability, organisation and planning, and culture and context. Item quality analyses revealed 22 acceptable items. We conclude by suggesting SJTs for promoting non-academic growth of prospective and beginning teachers.

Keywords

Rural teaching Personal characteristics Non-academic attributes Teacher recruitment Teacher retention Critical incidents 

Notes

Acknowledgements

We gratefully acknowledge the assistance provided by Lisa Kim (University of York, UK) and the consistent dedication and commitment shown by our team of collaborators at the New South Wales (NSW) Department of Education. In particular, we are grateful for the help provided by Mark Anderson, Ian Quintos, John Wilson, and Veronica Willmott of the Human Resources Directorate. We also express our utmost thanks to the NSW teachers and principals who shared their insights and expertise and to the pilot study participants for their time and feedback.

References

  1. Austin, J. (2010). Remote ready. Remote Health Recruitment, Northern Territory Government. Charles Darwin University, Australia. Retrieved from http://www.remoterecruitment.nt.gov.au/docs/remote_ready.pdf.
  2. Australian Bureau of Statistics. (2017). Census 2016 QuickStats: Aboriginal and/or Torres Strait Islander people. Belconnen, ACT: Commonwealth of Australia. Retrieved from http://www.abs.gov.au/.
  3. Australian Institute for Teaching and School Leadership (AITSL). (2011). Accreditation of initial teacher education programs in Australia: Standards and procedures. Melbourne, VIC: Ministerial Council for Education, Early Childhood Development and Youth Affairs.Google Scholar
  4. Australian Institute for Teaching and School Leadership (AITSL). (2015). Action now: Selection of entrants into initial teacher education—guidelines. Melbourne, VIC: Australian Institute for Teaching and School Leadership.Google Scholar
  5. Bahr, N., & Mellor, S. (2016). Australian education review: Building quality in teaching and teacher education. Camberwell, VIC: Australian Council for Educational Research.Google Scholar
  6. Baxter, J., Gray, M., & Hayes, A. (2011). Families in regional, rural and remote Australia (Fact Sheet). Australia Institute of Families Studies, Melbourne, VIC: Commonwealth of Australia. Retrieved from https://aifs.gov.au/publications/families-regional-rural-and-remote-australia.
  7. Beltman, S., Mansfield, C., & Price, A. (2011). Thriving not just surviving: A review of research on teacher resilience. Educational Research Review, 6(3), 185–207.  https://doi.org/10.1016/j.edurev.2011.09.001.CrossRefGoogle Scholar
  8. Buehl, M. M., & Fives, H. (2009). Exploring teachers’ beliefs about teaching knowledge: Where does it come from? Does it change? The Journal of Experimental Education, 77(4), 367–407.CrossRefGoogle Scholar
  9. Caldwell, B., & Sutton, D. (2010). Review of teacher education and induction. First reportfull report. Education Queensland http://education.qld.gov.au/students/higher-education/resources/review-teachereducation-school-induction-first-full-report.pdf.
  10. Cox, C. B., Barron, L. G., Davis, W., & de la Garza, B. (2017). Using situational judgment tests (SJTs) in training. Personnel Review, 46(1), 36–45.  https://doi.org/10.1108/PR-05-2015-0137.CrossRefGoogle Scholar
  11. Day, C., & Gu, Q. (2014). Resilient teachers, resilient schools. New York, NY: Routledge Taylor Francis.Google Scholar
  12. Frenzel, A. C., Becker-Kurz, B., Pekrun, R., & Goetz, T. (2015). Teaching this class drives me nuts!—Examining the person and context specificity of teacher emotions. PLoS ONE, 10(6), e0129630.  https://doi.org/10.1371/journal.pone.0129630.CrossRefGoogle Scholar
  13. Goldhaber, D., Grout, C., & Huntington-Klein, N. (2014). Screen twice, cut once: Assessing the predictive validity of teacher selection tools. (CEDR Working Paper No. 9). Seattle, WA: Center for Education Data and Research. http://www.cedr.us/papers/working/CEDR%20WP%202014-9.1.pdf.
  14. Gore, J., Barron, R. J., Holmes, K., & Smith, M. (2016). Who says we are not attracting the best and brightest? Teacher selection and the aspirations of Australian school students. The Australian Educational Researcher, 43(5), 527–549.  https://doi.org/10.1007/s13384-016-0221-8.CrossRefGoogle Scholar
  15. Gosling, S. D., Rentfrow, P. J., & Swann, W. B., Jr. (2003). A very brief measure of the big-five personality domains. Journal of Research in Personality, 37, 504–528.  https://doi.org/10.1016/S0092-6566(03)00046-1.CrossRefGoogle Scholar
  16. Hamre, B. K., Pianta, R. C., Downer, J. T., DeCoster, J., Mashburn, A. J., Jones, S. M., et al. (2013). Teaching through interactions: Testing a developmental framework of teacher effectiveness in over 4000 classrooms. The Elementary School Journal, 113(4), 461–487.CrossRefGoogle Scholar
  17. Holland, J. L. (1997). Making vocational choices: A theory of vocational personalities and work environments (3rd ed.). Odessa, FL: Psychological Assessment Resources.Google Scholar
  18. Huberman, M. (1989). The professional life cycle of teachers. Teachers College Record, 91, 31–57.Google Scholar
  19. Ingersoll, R. M., & Strong, M. (2011). The impact of induction and mentoring programs for beginning teachers: A critical review of the research. Review of Educational Research, 81, 201–233.  https://doi.org/10.3102/0034654311403323.CrossRefGoogle Scholar
  20. King, R. B., & McInerney, D. M. (2016). Culture and motivation. In K. R. Wentzel & D. B. Miele (Eds.), Handbook of motivation at school (pp. 275–310). New York: Routledge.Google Scholar
  21. Klassen, R. M., Durksen, T. L., Kim, L., Patterson, F., Rowett, E., Warwick, J., et al. (2017a). Developing a proof-of-concept selection test for entry into primary teacher education programs. International Journal of Assessment Tools in Education, 4(2), 96–114.  https://doi.org/10.21449/ijate.275772.CrossRefGoogle Scholar
  22. Klassen, R. M., Durksen, T. L., Patterson, F., & Rowett, E. (2017b). Filtering functions of assessment for selection into initial teacher education programs. In J. Clandinin & J. Husu (Eds.), The Sage handbook of research on teacher education (Vol. 1, pp. 893–909). New York: Sage Publications Ltd.CrossRefGoogle Scholar
  23. Klassen, R. M., Durksen, T. L., Rowett, E., & Patterson, F. (2014). Applicant reactions to a situational judgment test used for selection into initial teacher training. International Journal of Educational Psychology, 3, 104–125. Retrieved from http://www.hipatiapress.com/hpjournals/index.php/ijep.
  24. Klassen, R. M., & Tze, V. M. C. (2014). Teachers’ self-efficacy, personality, and teaching effectiveness: A meta-analysis. Educational Research Review, 12, 59–76.CrossRefGoogle Scholar
  25. Klassen, R. M., Tze, V. M. C., Betts, S. M., & Gordon, K. A. (2011). Teacher efficacy research 1998–2009: Signs of progress or unfulfilled promise? Educational Psychology Review, 23, 21–43.  https://doi.org/10.1007/s10648-010-9141-8.CrossRefGoogle Scholar
  26. Klassen, R. M., Yerdelen, S., & Durksen, T. L. (2013). Measuring teacher engagement: The development of the engaged teacher scale (ETS). Frontline Learning Research, 1, 33–52. Retrieved from http://journals.sfu.ca/flr/index.php/journal/article/view/44/37.
  27. Kutsyuruba, B., Godden, L., & Tregunna, L. (2014). Curbing early-career teacher attrition: A pan-Canadian document analysis of teacher induction and mentorship programs. Canadian Journal of Educational Administration and Policy, 161, 1–42.Google Scholar
  28. Lyons, T. (2006). Science, ICT and mathematics education in rural and regional Australia, SiMERR National Research Centre [Report]. http://apo.org.au/node/2732.
  29. Martin, A. J., & Dowson, M. (2009). Interpersonal relationships, motivation, engagement, and achievement: Yields for theory, current issues, and practice. Review of Educational Research, 79, 327–365.CrossRefGoogle Scholar
  30. Mayer, D., Allard, A., Bates, R., Dixon, M., Doeke, B., & Kline, J., et al. (2015). Studying the effectiveness of teacher education (Final Report). Geelong, VIC: Deakin University. http://www.setearc.com.au/publication-reports/.
  31. MCEETYA. (2001). National framework for rural and remote education in Australia. Journal of Research in Rural Education, 17(2), 112–119.Google Scholar
  32. McNeilage, A. (2014, January 21). Adrian Piccoli says teacher selection criteria should be more strict. The Sydney Morning Herald. http://www.smh.com.au/national/education/adrian-piccoli-says-teacher-selection-criteria-should-be-more-strict-20140120-314vn.html.
  33. Meng, L., & Muñoz, M. (2016). Teachers’ perceptions of effective teaching: A comparative study of elementary school teachers from China and the USA. Educational Assessment, Evaluation and Accountability.  https://doi.org/10.1007/s11092-015-9230-9.Google Scholar
  34. Motowidlo, S. J., & Beier, M. E. (2010). Differentiating specific job knowledge from implicit trait policies in procedural knowledge measured by a situational judgment test. Journal of Applied Psychology, 95(2), 321.CrossRefGoogle Scholar
  35. NSW Council of Deans of Education. (2016, July). Position paper: Selection methods into initial teacher education programs [Draft v2]. NSWCDE and BOSTES Joint Forum: Assuring Graduate Quality through the New Accreditation Standards. Received at University of Technology, Sydney.Google Scholar
  36. NSW Council of Deans of Education. (2017, January). Assessing and confirming suitability for teaching: A selection framework for NSW teacher education providers [Brief Report]. Sydney, NSW.Google Scholar
  37. NSW Department of Education and Communities (NSWDEC). (2013, November). Rural and remote education: A blueprint for action. NSW Department of Education and Communities. www.dec.nsw.gov.au/.
  38. Patterson, F., Ashworth, V., & Good, D. (2013). Situational judgement tests: A guide for applicants to the UK foundation programme. London: Medical School Council.Google Scholar
  39. Patterson, F., Zibarras, L., & Ashworth, V. (2015). Situational judgment tests in medical education and training: Research, theory and practice: AMEE Guide No.100. Medical Teacher, 00(00), 1–15.  https://doi.org/10.3109/0142159X.2015.1072619.Google Scholar
  40. Reid, J.-A., Green, B., Cooper, M., Hastings, W., Lock, G., & White, S. (2010). Regenerating rural social space? Teacher education for rural—Regional sustainability. Australian Journal of Education, 54(3), 262–276.CrossRefGoogle Scholar
  41. Rice, S., Richardson, P. W., & Watt, H. M. G. (2017). Hard-to-staff Australian schools: How can we ensure that all students have access to quality teachers? In T. Bentley and G. C. Savage (Eds.), Educating Australia: Challenges for the decade ahead (Chapter 17). Melbourne, VIC: Melbourne University Publishing (Kindle Edition).Google Scholar
  42. Ryan, A. M., & Ployhart, R. E. (2014). A century of selection. Annual Review of Psychology, 65, 20.1–20.25.CrossRefGoogle Scholar
  43. Sautelle, E., Bowles, T., Hattie, J., & Arifin, D. (2015). Personality, resilience, self-regulation and cognitive ability relevant to teacher selection. Australian Journal of Teacher Education, 40(40), 1–19. http://doi.org/10.14221/ajte.2015v40n4.4.
  44. Schultheiss, O., & Brunstein, J. (2010). Implicit motives. UK: Oxford University Press.CrossRefGoogle Scholar
  45. Sorrel, M. A., Olea, J., Abad, F. J., de la Torre, J., Aguado, D., & Lievens, F. (2016). Validity and reliability of situational judgement test scores. Organizational Research Methods, 19(3), 506–532.  https://doi.org/10.1177/1094428116630065.CrossRefGoogle Scholar
  46. Stemig, M. S., Sackett, P. R., & Lievens, F. (2015). Effects of organizationally endorsed coaching on performance and validity of situational judgment tests. International Journal of Selection and Assessment, 23(2), 174–181.CrossRefGoogle Scholar
  47. Stemler, S. E., Aggarwal, V., & Nithyanand, S. (2016). Knowing what NOT to do is a critical job skill: Evidence from 10 different scoring methods. International Journal of Selection and Assessment, 24(3), 229–245.CrossRefGoogle Scholar
  48. Stenberg, K., Karlsson, L., Pitkaniemi, H., & Maaranen, K. (2014). Beginning student teachers’ teacher identities based on their practical theories. European Journal of Teacher Education, 37(2), 204–219.CrossRefGoogle Scholar
  49. TEMAG: Teacher Education Ministerial Advisory Group. (2014). Action now: Classroom ready teachers. Canberra: Australian Government Department of Education. http://www.studentsfirst.gov.au.
  50. Tripp, D. (1994). Teachers’ lives, critical incidents, and professional practice. International Journal of Qualitative Studies in Education, 7(1), 65–76.  https://doi.org/10.1080/0951839940070105.CrossRefGoogle Scholar
  51. Tschannen-Moran, M., & Woolfolk Hoy, A. (2001). Teacher efficacy: Capturing an elusive construct. Teaching and Teacher Education, 17, 783–805.CrossRefGoogle Scholar
  52. Whelpley, C. E. (2014). How to score situational judgment tests: A theoretical approach and empirical test [Unpublished Doctoral Dissertation]. Virginia Commonwealth University, USA.Google Scholar
  53. Whetzel, D. L., & McDaniel, M. A. (2016). Are Situational Judgment Tests better assessments of personality than traditional personality tests in high-stakes testing? In U. Kumar (Ed.), The Wiley Handbook of Personality Assessment (pp. 205–214). West Sussex: Wiley.CrossRefGoogle Scholar

Copyright information

© The Australian Association for Research in Education, Inc. 2017

Authors and Affiliations

  1. 1.School of Education, Faculty of Arts and Social SciencesUNSW SydneySydneyAustralia
  2. 2.University of YorkYorkUK

Personalised recommendations