The development of a student engagement instrument for the responding strand in visual arts

  • Julia E. MorrisEmail author


The ‘responding’ strand of the Australian visual arts curriculum promotes twenty-first century learning skills through students’ analytical engagement with artworks and artists. Assessing students’ experiences and engagement is one strategy to improve teaching and learning in responding. However, there are no validated, subject-specific student engagement instruments for teachers to use. This study sought to develop a student self-report diagnostic instrument that provides information on past experiences with visual arts and factors affecting both cognitive and psychological engagement, with implications for improving teaching and learning. The instrument was piloted with 266 Year 10 to 12 students, as responding has an approximate 50% assessment weighting in the Australian visual arts course for senior school students. This paper reports on the exploratory and confirmatory factor analyses conducted in the development of the instrument. It adds to the body of knowledge on developing engagement instruments, recognising that student engagement in secondary education is context-dependent.


Student engagement Diagnostic assessment Instrument development Responding Visual arts 


  1. ACARA. (2012). A guide to understanding ICSEA. Retrieved from Sydney, Australia:
  2. Ainley, M. (2012). Students’ interest and engagement in classroom activities. In S. L. Christenson, A. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement. New York: Springer.Google Scholar
  3. AITSL. (2011). The National Professional Standards for Teachers. Melbourne, Australia: Ministerial Council for Education, Early Childhood Development and Youth Affairs (MCEECDYA).Google Scholar
  4. Appleton, J. J., Christenson, S. L., Kim, D., & Reschly, A. (2006). Measuring cognitive and psychological engagement: Validation of the student engagement instrument. Journal of School Psychology, 44(5), 427–445.Google Scholar
  5. Appleton, J. J., Christenson, S. L., & Furlong, M. J. (2008). Student engagement with school: Critical conceptual and methodological issues of the construct. Psychology in the schools, 45(5), 369–386.Google Scholar
  6. Bandura, A. (2012). On the functional properties of perceived self-efficacy revisited. Journal of Management, 38(9), 9–44.Google Scholar
  7. Bandura, A., & Locke, E. (2003). Negative self-efficacy and goal effects revisited. Journal of Applied Psychology, 88(1), 87–99.Google Scholar
  8. Betts, J. E., Appleton, J. J., Reschly, A. L., Christenson, S. L., & Huebner, E. S. (2010). A study of the factorial invariance of the student engagement instrument (SEI): Results from middle and high school students. School Psychology Quarterly, 25(2), 84–93.Google Scholar
  9. Boughton, D. (1989). The changing face of Australian art education: New horizons or sub-colonial politics? Studies in Art Education, 30(4), 197–211.Google Scholar
  10. Caldwell, B., & Vaughan, T. (2011). Transforming education through the arts. Oxford: Routledge.Google Scholar
  11. Carbonneau, N., Vallerand, R. J., & Lafrenière, M. K. (2012). Toward a tripartite model of intrinsic motivation. Journal of Personality, 80(5), 1147–1178.Google Scholar
  12. Carmines, E. G., & McIver, J. P. (1981). Analyzing models with unobserved variables. In G. W. Bohrnstedt & E. F. Brogatta (Eds.), Social measurement: Current issues. Beverly Hills: Sage.Google Scholar
  13. Cherry, D. (2004). Art history visual culture. Art History, 27(4), 479–493.Google Scholar
  14. Christenson, S. L., Reschly, A., & Wylie, C. (Eds.). (2012). Handbook of research on student engagment. New York: Springer.Google Scholar
  15. Cohen, L., Manion, L., & Morrison, K. (2011). Research methods in education (7th ed.). New York: Routledge.Google Scholar
  16. Commonwealth of Australia. (2017). National Innovation and Science Agenda. Retrieved from
  17. Creswell, J. (2014). Educational research: Planning, conducting and evaluating quantitative and qualitative research (4th ed.). Essex: Pearson.Google Scholar
  18. Cumming, J., & Mawdesley, R. (2013). Australia, quality education and the ‘best interests of the child’. Australian Journal of Education, 57(3), 292–309.Google Scholar
  19. Davis, M. H., & McPartland, J. M. (2012). High school reform and student engagement. In S. L. Christenson, A. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement. New York: Springer.Google Scholar
  20. Dell-Kuster, S., Sanjuan, E., Todorov, A., Weber, H., Heberer, M., & Rosenthal, R. (2014). Designing questionnaires: Healthcare survey to compare two different response scales. BMC Medical Research Methodology, 14, 96–109.Google Scholar
  21. Department of Education. (2013). Statutory guidance: National curriculum in England: Art and design programmes of study. Retrieved from
  22. Dodge, T., & Kaufman, A. (2009). Student perceptions and motivation in the classroom: Exploring relatedness and value. Social Psychology of Education, 12(1), 101–112.Google Scholar
  23. Dulfer, N., Rice, S., & Clarke, K. (2017). Student engagement, non-completion and pedagogy: Development of a measurement tool. Australian Journal of Education, 61(1), 40–53. Scholar
  24. Efland, A. (1990). A history of art education: Intellectual and social currents in teaching the visual arts. New York: Teachers College Press.Google Scholar
  25. Eisner, E. (1987). The role of disciplined-based art education in America’s schools. Art Education, 40(5), 6–26 and 43–45.Google Scholar
  26. Eisner, E. (1990). Discipline-based art education: Conceptions and misconceptions. Educational Theory, 40(4), 423–430.Google Scholar
  27. Field, A. (2013). Discovering statistics using IBM SPSS statistics (4th ed.). London: Sage.Google Scholar
  28. Fredricks, J., Blumenfeld, P., & Paris, A. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59–109.Google Scholar
  29. Gilbert, I. (2013). Essential motivation in the classroom. New York: Routledge.Google Scholar
  30. Gilbert, A. D. (2016). The framework for 21st century learning: A first-rate foundation for music education assessment and teacher evaluation. Arts Education Policy Review, 117(1), 13–18.Google Scholar
  31. Goldstein, H. (2015). Validity, science and educational measurement. Assessment in Education: Principles, Policy & Practice, 22(2), 193–201. Scholar
  32. Gray, J., & Hackling, M. (2009). Wellbeing and retention: A senior secondary student perspective. The Australian Educational Researcher, 36(2), 119–145.Google Scholar
  33. Greer, D. W. (1987). A structure of discipline concepts for DBAE. Studies in Art Education, 28(4), 227–233.Google Scholar
  34. Greiff, S., Wüstenberg, S., Csapó, B., Demetriou, A., Hautamäki, J., Graesser, A. C., et al. (2014). Domain-general problem solving skills and education in the 21st century. Educational Research Review, 13, 74–83.Google Scholar
  35. Hardy, I. (2015). Data, numbers and accountability: The complexity, nature and effects of data use in schools. British Journal of Educational Studies, 63(4), 467–486. Scholar
  36. Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Oxon: Routledge.Google Scholar
  37. Hattie, J. A., & Brown, G. T. L. (2011). Assessment and evaluation. In C. M. Rubie-Davies (Ed.), Educational psychology: Concepts, research and challenges. New York: Routledge.Google Scholar
  38. Hu, L., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to under parameterized model specification. Psychological Methods, 3(4), 424–453.Google Scholar
  39. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. Scholar
  40. Kemp, S., & Scaife, J. (2012). Misunderstood and neglected? Diagnostic and formative assessment practices of lecturers. Journal of Education for Teaching: International Research and Pedagogy, 38(2), 181–192.Google Scholar
  41. Kim, C., & Geahigan, G. (2004). South Korean art education during the third and fourth republics, 1960–1979: Economic development and nationalistic aspirations. Journal of Cultural Research in Art Education, 22, 69–85.Google Scholar
  42. Lovelace, M. D., Reschly, A. L., Appleton, J. J., & Lutz, M. E. (2014). Concurrent and predictive validity of the student engagement instrument. Journal of Psychoeducational Assessment, 32(6), 509–520. Scholar
  43. Macdonald, S. (2005). A century of art and design education: From arts and crafts to conceptual art. Cambridge: Lutterworth Press.Google Scholar
  44. Mansour, M., Martin, A. J., Anderson, M., Gibson, R., Liem, G. A. D., & Sudmalis, D. (2016). Student, home, and school socio-demographic factors: Links to school, home, and community arts participation. The Australian Educational Researcher, 43, 221–244. Scholar
  45. Marsh, H. W., Hau, K., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling: A Multidisciplinary Journal, 11(3), 320–341. Scholar
  46. Martin, A. (2007). The relationship between teachers’ perceptions of student motivation and engagement and teachers’ enjoyment of and confidence in teaching. Asia-Pacific Journal of Teacher Education, 34(1), 73–93.Google Scholar
  47. Mazer, J. P. (2012). Development and validation of the student interest and engagement scales. Communication Methods and Measures, 6(2), 99–125.Google Scholar
  48. McDonnell, L. M. (2012). Educational accountability and policy feedback. Educational Policy, 27(2), 170–189.Google Scholar
  49. McKeon, P. (2002). The sense of art history in art education. Journal of Aesthetic Education, 36(2), 98–109. Scholar
  50. Moller, A. C., Deci, E. L., & Ryan, R. M. (2006). Choice and ego-depletion: The moderating role of autonomy. Personality and Social Psychology Bulletin, 32(8), 1024–1036.Google Scholar
  51. Moreira, P. A. S., & Dias, M. A. (2018). Tests of factorial structure and measurement invariance for the Student Engagement Instrument: Evidence from middle and high school students. International Journal of School & Educational Psychology. Scholar
  52. Morris, J. E. (2015). Assessing Western Australian year 11 students' engagement with responding in visual arts. Doctoral thesis, Edith Cowan University, Perth, Australia.Google Scholar
  53. Morris, J. E. (2018). Arts engagement outside of school: Links with Year 10 to 12 students’ intrinsic motivation and self-efficacy in responding to art. The Australian Educational Researcher, 45(4), 455–472. Scholar
  54. Morris, J. E., & Lummis, G. W. (2014). Investigating the personal experiences and self-efficacy of Western Australian primary pre-service teachers in the visual arts. Australian Art Education, 36(1), 26–47.Google Scholar
  55. Morris, J. E., Lummis, G. W., & Lock, G. (2017). Questioning art: Factors affecting students’ cognitive engagement in responding. Issues in Educational Research, 27(3), 493–511.Google Scholar
  56. Muijs, D. (2011). Doing quantitative research in education with SPSS (2nd ed.). London: Sage.Google Scholar
  57. Pekrun, R., & Linnenbrink-Garcia, L. (2012). Academic emotions and student engagement. In S. L. Christenson, A. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement. New York: Springer.Google Scholar
  58. Proust, J. (2010). Metacognition. Philosophy Compass, 5(11), 989–998. Scholar
  59. Punch, K. (2009). Introduction to research methods in education. London: Sage.Google Scholar
  60. Ryan, R. M., & Deci, E. L. (2000). Intrinsic and extrinsic motivations: classic definitions and new directions. Contemporary Educational Psychology, 25(1), 54–67. Scholar
  61. Ryan, R. M., & Deci, E. L. (2006). Self-regulation and the problem of human autonomy: Does psychology need choice, self-determination, and will? Journal of Personality, 74(6), 1557–1586. Scholar
  62. Saavedra, A. R., & Opfer, V. D. (2012). Learning 21st-century skills requires 21st-century teaching. Phi Delta Kappan, 94(2), 8.Google Scholar
  63. Scaife, J., & Wellington, J. (2010). Varying perspectives and practices in formative and diagnostic assessment: A case study. Journal of Education for Teaching: International Research and Pedagogy, 36(2), 137–151.Google Scholar
  64. School Curriculum and Standards Authority. (2014). Visual arts ATAR course: Year 12 syllabus. Perth: School Curriculum and Standards Authority.Google Scholar
  65. School Curriculum and Standards Authority. (2015). Visual arts ATAR course: Year 12 syllabus. Perth: School Curriculum and Standards Authority.Google Scholar
  66. Sinatra, G. M., Heddy, B. C., & Lombardi, D. (2015). The challenges of defining and measuring student engagement in science. Educational Psychologist, 50(1), 1–13.Google Scholar
  67. Smith-Shank, D. (2008). Visual culture and issues-based curricula. Australian Art Education, 31(1), 6–15.Google Scholar
  68. Stobart, G. (2004). Developing and improving assessment instruments. Assessment in Education: Principles, Policy & Practice, 11(3), 243–245. Scholar
  69. Sun, J. (2005). Assessing goodness of fit in confirmatory factor analysis. Measurement and Evaluation in Counseling and Development, 37(4), 240–256.Google Scholar
  70. Tarricone, P. (2011). The taxonomy of metacognition. New York: Psychology.Google Scholar
  71. Van der Kleij, F. M., Vermeulen, J. A., Schildkamp, K., & Eggen, T. J. H. M. (2015). Integrating data-based decision making, assessment for learning and diagnostic testing in formative assessment. Assessment in Education: Principles, Policy & Practice, 22(3), 324–343. Scholar
  72. van Prooijen, J. W., & van der Kloot, W. A. (2001). Confirmatory analysis of exploratively obtained factor structures. Educational and Psychological Measurement, 61(5), 777–792. Scholar
  73. Vansteenkiste, M., Simons, J., Lens, W., Soenens, B., & Matos, L. (2005). Examining the motivational impact of intrinsic versus extrinsic goal framing and autonomy-supportive versus internally controlling communication style on early adolescents’ academic achievement. Child Development, 76(2), 493–501.Google Scholar
  74. Wiley, J., & Jee, B. D. (2011). Cognition: Overview and recent trends. In V. G. Aukrust (Ed.), Learning and cognition in education (pp. 3–8). Oxford: Academic Press.Google Scholar
  75. Winchmann, S. S. (2011). Self-determination theory: The importance of autonomy to well-being across cultures. Journal of Humanistic Counseling, 50(1), 16–26.Google Scholar
  76. Yonezawa, S., Jones, M., & Joselowsky, F. (2009). Youth engagement in high schools: Developing a mulitdimensional, critical apporach to improving engagement for all students. Journal of Educational Change, 10(2), 191–209.Google Scholar

Copyright information

© The Australian Association for Research in Education, Inc. 2018

Authors and Affiliations

  1. 1.School of EducationEdith Cowan UniversityJoondalupAustralia

Personalised recommendations