Advertisement

The effect of online summative and formative teacher assessment on teacher competences

  • Zohre Mohamadi ZenouzaghEmail author
Article

Abstract

This study investigated the effect of online summative and formative assessments on 30 Iranian English as foreign language teachers’ teaching competences. Everything being equal in terms of participant homogeneity and classroom video-based teacher induction for 21 sessions, significant differences in teaching competence improvements on three assessment interventions were sought using pretest/posttest time series design. The assessment interventions included online summative assessment (teachers receiving online feedback after their classroom observation), teachers online portfolio writing formative assessment (teachers receiving feedback after content analysis of their portfolios) and online collaborative discussion formative assessment (ECDF) (teachers’ receiving feedback in the discussion forum). Classroom observation, content analysis of e-portfolios, and log analysis of ECDF were used to code indicators of teacher competence improvements according to a scoring schema to fill teacher balanced score card (TBSC). The teacher competences were measured on TBSC in pre- and post-assessment interventions. The results of Paired sample t test and ANCOVA (analysis of covariance) indicated improved teacher competences across all-time series measures. Comparatively, teaching competences were more affected in ECDF assessment intervention where collaborative reflection and feedback were exchanged. Implications for education practitioners were provided and suggestions were offered for further research in the light of the limitations of the study.

Keywords

Online summative assessment Teacher e-portfolio writing Collaborative discussion Teacher competences Online formative assessment 

Notes

Compliance with ethical standards

Conflict of interest

The authors declare that they have no competing interests.

References

  1. Abbas, A. A. (1994). Evaluating the assessment process in the EFL teaching programs and the general secondary education certificate english exams for 1989–1993.Google Scholar
  2. Admiraal, W., Louws, M., Lockhorst, D., Paas, T., Buynsters, M., Cviko, A., & Post, L. (2017). Teachers in school-based technology innovations: A typology of their beliefs on teaching and technology. Computers & Education, 114, 57–68.CrossRefGoogle Scholar
  3. Al-Thumali, A. D. A. (2011). Evaluating EFL Intermediate Teachers’ Performance in the Light of Quality Standards in Saudi Arabia. Online Submission.Google Scholar
  4. Azam, M., & Kingdon, G. G. (2015). Assessing teacher quality in India. Journal of Development Economics, 117, 74–83.CrossRefGoogle Scholar
  5. Bakker, M. E., Roelofs, E. C., Beijaard, D., Sanders, P. F., Tigelaar, D. E., & Verloop, N. (2011). Video portfolios: The development and usefulness of a teacher assessment procedure. Studies in Educational Evaluation, 37(2), 123–133.CrossRefGoogle Scholar
  6. Bala, S. S., Mansor, W. F. A. W., Stapa, M., & Zakaria, M. H. (2012). Digital portfolio and professional development of language teachers. Procedia-Social and Behavioral Sciences, 66, 176–186.CrossRefGoogle Scholar
  7. Bastian, K. C., Henry, G. T., Pan, Y., & Lys, D. (2016). Teacher candidate performance assessments: Local scoring and implications for teacher preparation program improvement. Teaching and Teacher Education, 59, 1–12.CrossRefGoogle Scholar
  8. Blašková, M., Blaško, R., & Kucharčíková, A. (2014). Competences and competence model of university teachers. Procedia-Social and Behavioral Sciences, 159, 457–467.CrossRefGoogle Scholar
  9. Burhan-Horasanlı, E., & Ortaçtepe, D. (2016). Reflective practice-oriented online discussions: A study on EFL teachers’ reflection-on, in and for-action. Teaching and Teacher education, 59, 372–382.CrossRefGoogle Scholar
  10. Çevik, Y. D., Haşlaman, T., & Çelik, S. (2015). The effect of peer assessment on problem solving skills of prospective teachers supported by online learning activities. Studies in Educational Evaluation, 44, 23–35.CrossRefGoogle Scholar
  11. Chang, C.-C., Tseng, K.-H., Liang, C., & Liao, Y.-M. (2013). Constructing and evaluating online goal-setting mechanisms in web-based portfolio assessment system for facilitating self-regulated learning. Computers & Education, 69, 237–249.CrossRefGoogle Scholar
  12. Cho, H. (2016). Under co-construction: An online community of practice for bilingual pre-service teachers. Computers & Education, 92, 76–89.CrossRefGoogle Scholar
  13. Chua, Y. P., & Chua, Y. P. (2017). How are e-leadership practices in implementing a school virtual learning environment enhanced? A grounded model study. Computers & Education, 109, 109–121.CrossRefGoogle Scholar
  14. Danielson, C. (2001). New trends in teacher evaluation. Educational leadership, 58(5), 12–15.Google Scholar
  15. Danielson, C., & McGreal, T. L. (2000). Teacher evaluation to enhance professional practice, Charlotte: ASCD.Google Scholar
  16. Darling-Hammond, L., Newton, S. P., & Wei, R. C. (2013). Developing and assessing beginning teacher effectiveness: The potential of performance assessments. Educational Assessment, Evaluation and Accountability, 25(3), 179–204.CrossRefGoogle Scholar
  17. Dede, C. (2006). Online professional development for teachers: Emerging models and methods. Boston: Harvard Education Press Cambridge.Google Scholar
  18. DeLuca, C., LaPointe-McEwan, D., & Luhanga, U. (2016). Teacher assessment literacy: A review of international standards and measures. Educational Assessment, Evaluation and Accountability, 28(3), 251–272.CrossRefGoogle Scholar
  19. Duckor, B., Castellano, K. E., Téllez, K., Wihardini, D., & Wilson, M. (2014). Examining the internal structure evidence for the performance assessment for california teachers a validation study of the elementary literacy teaching event for tier i teacher licensure. Journal of Teacher Education, 65(5), 402–420.CrossRefGoogle Scholar
  20. Duţă, N., Pânişoară, G., & Pânişoară, I. O. (2014). The profile of the teaching profession–empirical reflections on the development of the competences of university teachers. Procedia-Social and Behavioral Sciences, 140, 390–395.CrossRefGoogle Scholar
  21. Feistauer, D., & Richter, T. (2016). How reliable are students’ evaluations of teaching quality? A variance components approach. Assessment & Evaluation in Higher Education, 1–17.Google Scholar
  22. Field, A. (2013). Discovering statistics using IBM SPSS statistics. (4th edn.). Sage: London.Google Scholar
  23. Gan, Z., & Lee, F. K. J. (2016). Understanding ESL Student teachers’ learning of classroom practices in the practicum: A case study in Hong Kong. The Asia-Pacific Education Researcher, 25(2), 251–266.CrossRefGoogle Scholar
  24. García-Martín, J., & García-Sánchez, J.-N. (2016). Pre-service teachers’ perceptions of the competence dimensions of digital literacy and of psychological and educational measures. Computers & Education, 107, 54–67Google Scholar
  25. Gikandi, J. W., Morrow, D., & Davis, N. E. (2011). Online formative assessment in higher education: A review of the literature. Computers & Education, 57(4), 2333–2351.CrossRefGoogle Scholar
  26. Gil-Flores, J., Rodríguez-Santero, J., & Torres-Gordillo, J.-J. (2017). Factors that explain the use of ICT in secondary-education classrooms: The role of teacher characteristics and school infrastructure. Computers in Human Behavior, 68, 441–449.CrossRefGoogle Scholar
  27. Gillmore, G. M. (1977). How large is the course effect? A note on Romney’s course effect vs. teacher effect on students’ ratings of teacher competence. Research in Higher Education, 7(2), 187–189.CrossRefGoogle Scholar
  28. Gold, B., & Holodynski, M. (2017). Using digital video to measure the professional vision of elementary classroom management: Test validation and methodological challenges. Computers & Education, 107, 13–30.CrossRefGoogle Scholar
  29. Hargreaves, E. (2007). The validity of collaborative assessment for learning. Assessment in Education, 14(2), 185–199.CrossRefGoogle Scholar
  30. Henry, G. T., Thompson, C. L., Fortner, C. K., Zulli, R. A., & Kershaw, D. (2010). The impact of teacher preparation on student learning in North Carolina public schools. Chapel Hill: Carolina Institute for Public Policy. University of North Carolina at Chapel Hill.Google Scholar
  31. Hooker, T. (2015). Assessment for learning: A comparative study of paper-based portfolios and online portfolios. Early Childhood Folio, 19(1), 17–24.CrossRefGoogle Scholar
  32. Huff, K. L., & Sireci, S. G. (2001). Validity issues in computer-based testing. Educational Measurement: Issues and Practice, 20(3), 16–25.CrossRefGoogle Scholar
  33. Hughes, K., & Pate, G. R. (2012). Moving beyond student ratings: A balanced scorecard approach for evaluating teaching performance. Issues in Accounting Education, 28(1), 49–75.CrossRefGoogle Scholar
  34. Hung, M.-L. (2016). Teacher readiness for online learning: Scale development and teacher perceptions. Computers & Education, 94, 120–133.CrossRefGoogle Scholar
  35. Imhof, M., & Picard, C. (2009). Views on using portfolio in teacher education. Teaching and Teacher Education, 25(1), 149–154.CrossRefGoogle Scholar
  36. Kabilan, M. K., & Khan, M. A. (2012). Assessing pre-service English language teachers’ learning using e-portfolios: Benefits, challenges and competencies gained. Computers & Education, 58(4), 1007–1020.CrossRefGoogle Scholar
  37. Kent, C., Laslo, E., & Rafaeli, S. (2016). Interactivity in online discussions and learning outcomes. Computers & Education, 97, 116–128.CrossRefGoogle Scholar
  38. Koh, J. H. L., & Chai, C. S. (2016). Seven design frames that teachers use when considering technological pedagogical content knowledge (TPACK). Computers & Education, 102, 244–257.CrossRefGoogle Scholar
  39. Kreijns, K., Van Acker, F., Vermeulen, M., & Van Buuren, H. (2013). What stimulates teachers to integrate ICT in their pedagogical practices? The use of digital learning materials in education. Computers in Human Behavior, 29(1), 217–225.CrossRefGoogle Scholar
  40. Kwon, K., Liu, Y.-H., & Johnson, L. P. (2014). Group regulation and social-emotional interactions observed in computer supported collaborative learning: Comparison between good vs. poor collaborators. Computers & Education, 78, 185–200.CrossRefGoogle Scholar
  41. Lam, R. (2017). Taking stock of portfolio assessment scholarship: From research to practice. Assessing Writing, 31, 84–97.CrossRefGoogle Scholar
  42. Lasauskienė, J., Rauduvaitė, A., & Barkauskaitė, M. (2015). Development of general competencies within the context of teacher training. Procedia-Social and Behavioral Sciences, 191, 777–782.CrossRefGoogle Scholar
  43. Lee, K., & Brett, C. (2015). Dialogic understanding of teachers’ online transformative learning: A qualitative case study of teacher discussions in a graduate-level online course. Teaching and Teacher Education, 46, 72–83.CrossRefGoogle Scholar
  44. Lin, M., Preston, A., Kharrufa, A., & Kong, Z. (2016). Making L2 learners’ reasoning skills visible: The potential of computer supported collaborative learning environments. Thinking Skills and Creativity, 22, 303–322.CrossRefGoogle Scholar
  45. Liu, S.-H. (2017). Relationship between the factors influencing online help-seeking and self-regulated learning among Taiwanese preservice teachers. Computers in Human Behavior, 72, 38–45.CrossRefGoogle Scholar
  46. Llamas-Nistal, M., Fernández-Iglesias, M. J., González-Tato, J., & Mikic-Fonte, F. A. (2013). Blended e-assessment: Migrating classical exams to the digital world. Computers & Education, 62, 72–87.CrossRefGoogle Scholar
  47. Lucas, E. M., Oliveira, T. C., Farias, K., & Alencar, P. S. (2017). CollabRDL: A language to coordinate collaborative reuse. Journal of Systems and Software, 131, 505–527CrossRefGoogle Scholar
  48. Mansvelder-Longayroux, D. D., Beijaard, D., & Verloop, N. (2007). The portfolio as a tool for stimulating reflection by student teachers. Teaching and Teacher education, 23(1), 47–62.CrossRefGoogle Scholar
  49. Marsh, H. W., Muthén, B., Asparouhov, T., Lüdtke, O., Robitzsch, A., Morin, A. J., & Trautwein, U. (2009). Exploratory structural equation modeling, integrating CFA and EFA: Application to students’ evaluations of university teaching. Structural Equation Modeling: A Multidisciplinary Journal, 16(3), 439–476.CrossRefGoogle Scholar
  50. Mohamadi, Z. (2018a). Comparative effect of online summative and formative assessment on EFL student writing ability. Studies in Educational Evaluation, 59, 29–40.CrossRefGoogle Scholar
  51. Mohamadi, Z. (2018b). Comparative effect of project-based learning and electronic project-based learning on the development and sustained development of english idiom knowledge. Journal of Computing in Higher Education, 2018, 1–23.Google Scholar
  52. Mohamadi, Z., & Malekshahi, N. (2018). Designing and validating a potential formative evaluation inventory for teacher competences. Language Testing in Asia, 8(1), 6.CrossRefGoogle Scholar
  53. Mohammadi, Z. (2017). Interactional complexity development, interactional demonstrators and interaction density in collaborative and e-collaborative writing modalities. Journal of Teaching Language Skills, 36(2), 75–102.Google Scholar
  54. Navidinia, H., Reza Kiani, G., Akbari, R., & Samar, G., R (2015). EFL teacher performance evaluation in Iranian high schools: examining the effectiveness of the status quo and setting the groundwork for developing an alternative model. The International Journal of Humanities, 21(4), 27–53.Google Scholar
  55. Ng, E. M. (2016). Fostering pre-service teachers’ self-regulated learning through self-and peer assessment of wiki projects. Computers & Education, 98, 180–191.CrossRefGoogle Scholar
  56. Peercy, M. M., & Troyan, F. J. (2017). Making transparent the challenges of developing a practice-based pedagogy of teacher education. Teaching and Teacher education, 61, 26–36.CrossRefGoogle Scholar
  57. Picci, P., Calvani, A., & Bonaiuti, G. (2012). The use of digital video annotation in teacher training: The teachers’ perspectives. Procedia-Social and Behavioral Sciences, 69, 600–613.CrossRefGoogle Scholar
  58. Potolea, D. (2008). Asigurarea calităţii programelor de formare continuă a cadrelor didactice–concepţie şi practici. Bucharest: Editura Universităţii din Bucureşti.Google Scholar
  59. Rabiee, A., Nazarian, Z., & Gharibshaeyan, R. (2013). An explanation for internet use obstacles concerning e-learning in Iran. The International Review of Research in Open and Distributed Learning, 14(3), 361–376.CrossRefGoogle Scholar
  60. Renaud, R. D., & Murray, H. G. (2005). Factorial validity of student ratings of instruction. Research in Higher Education, 46(8), 929–953.CrossRefGoogle Scholar
  61. Richards, J. C. (2010). Competence and performance in language teaching. RELC Journal, 41(2), 101–122.CrossRefGoogle Scholar
  62. Santiago, P., & Benavides, F. (2009). Teacher evaluation: A conceptual framework and examples of country practices. Paper for presentation at the OECD Mexico, 1–2.Google Scholar
  63. Sheard, M. K., & Chambers, B. (2014). A case of technology-enhanced formative assessment and achievement in primary grammar: How is quality assurance of formative assessment assured? Studies in Educational Evaluation, 43, 14–23.CrossRefGoogle Scholar
  64. Siozos, P., Palaigeorgiou, G., Triantafyllakos, G., & Despotakis, T. (2009). Computer based testing using “digital ink”: Participatory design of a Tablet PC based assessment application for secondary education. Computers & Education, 52(4), 811–819.CrossRefGoogle Scholar
  65. Skedsmo, G., & Huber, S. G. (2017). Evaluation of educators’ performance—balancing various measures to improve practice. Educational Assessment, Evaluation and Accountability, 2017, 1–4.Google Scholar
  66. Smith, J. S., Szelest, B. P., & Downey, J. P. (2004). Implementing outcomes assessment in an academic affairs support unit. Research in Higher Education, 45(4), 405–427.CrossRefGoogle Scholar
  67. Tarighat, S., & Khodabakhsh, S. (2016). Mobile-assisted language assessment: Assessing speaking. Computers in Human Behavior, 64, 409–413.CrossRefGoogle Scholar
  68. Teo, T. (2015). Comparing pre-service and in-service teachers’ acceptance of technology: Assessment of measurement invariance and latent mean differences. Computers & Education, 83, 22–31.CrossRefGoogle Scholar
  69. Tseng, S.-C., & Tsai, C.-C. (2007). On-line peer assessment and the role of the peer feedback: A study of high school computer course. Computers & Education, 49(4), 1161–1174.CrossRefGoogle Scholar
  70. Turula, A. (2017). Teaching presence in telecollaboration. Keeping an open mind. System, 64, 21–33.CrossRefGoogle Scholar
  71. Vinagre, M. (2016). Developing teachers’ telecollaborative competences in online experiential learning. System, 64, 34–45.CrossRefGoogle Scholar
  72. Wu, H.-L., & Pedersen, S. (2011). Integrating computer-and teacher-based scaffolds in science inquiry. Computers & Education, 57(4), 2352–2363.CrossRefGoogle Scholar
  73. Yücel, ÜA., & Usluel, Y. K. (2016). Knowledge building and the quantity, content and quality of the interaction and participation of students in an online collaborative learning environment. Computers & Education, 97, 31–48.CrossRefGoogle Scholar
  74. Yurdabakan, I., & Erdogan, T. (2009). The effects of portfolio assessment on reading, listening and writing skills of secondary school prep class students. The Journal of International Social Research, 2(9), 526–538.Google Scholar
  75. Zimpher, N., & Howey, K. R. (1987). Adapting supervisory practices to different orientations of teaching competence. Journal of Curriculum and Supervision, 2(2), 101–127.Google Scholar
  76. Zottmann, J. M., Stegmann, K., Strijbos, J.-W., Vogel, F., Wecker, C., & Fischer, F. (2013). Computer-supported collaborative learning with digital video cases in teacher education: The impact of teaching experience on knowledge convergence. Computers in Human Behavior, 29(5), 2100–2108.CrossRefGoogle Scholar

Copyright information

© Education Research Institute, Seoul National University, Seoul, Korea 2018

Authors and Affiliations

  1. 1.English Translation and Teaching Department, Karaj BranchIslamic Azad UniversityKarajIran

Personalised recommendations