Skip to main content
SpringerLink
Log in

Evaluating teachers’ effectiveness in classrooms: an ML-based assessment portfolio

  • Original Article
  • Published:
Social Network Analysis and Mining Aims and scope Submit manuscript

Abstract

Effective teachers are strongly committed to creating a positive learning experience, instinct, and impact for transforming students’ learning. The transforming elements can be defined through related attributes, such as self-efficacy, regular attendance, and cooperative behavior. However, this involves a significant data analysis task to measure teachers’ performance and predict their effectiveness in the education domain. Underpinned by a recognized design perspective of design science research, this study establishes a methodological framework for designing a solution artifact utilizing machine learning algorithms informing the design science research for IS design. We designed a new solution artifact utilizing a case dataset, specifically, a record of the UCI machine learning repository. Researchers can measure teachers’ effectiveness through the proposed innovative technique that elevates distinct resources to configure learning opportunities and relevant monitoring of learning. To evaluate the proposed ML model in measuring teachers’ effectiveness, we validated the prediction by contrasting it with other comparable models. We developed two ML models using K-means and hierarchical algorithms and found that the K-means presented the best outcome in representing three clusters: negative, positive, and neutral feedback, also showing 99% accuracy using the random forest classifier. Therefore, the K-means clustering technique is selected to be the core component of the solution for predicting teachers’ effectiveness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  • Abaidullah AM, Ahmed N, Ali E (2015) Identifying hidden patterns in students’ feedback through cluster analysis. Int J of Comput Theory Eng 7(1):16

    Article  Google Scholar 

  • Abdulhafedh A (2021) Incorporating K-means, hierarchical clustering and PCA in customer segmentation. J City Dev 3(1):12–30

    Google Scholar 

  • Agaoglu M (2016) Predicting instructor performance using data mining techniques in higher education. IEEE Access 4:2379–2387

    Article  Google Scholar 

  • Ahmadi F, Ahmad MS (2013) Data mining in teacher evaluation system using WEKA. Int J Comput Appl 63(10):12–18

    Google Scholar 

  • Ahmed AM, Rizaner A, Ulusoy AH (2016) Using data mining to predict instructor performance. Procedia Comput Sci 102:137–142

    Article  Google Scholar 

  • Anderson LW (2004) Increasing teacher effectiveness. International publication institute for education planning assessment of teacher effectiveness of human resources. UNESCO 41(4):778–820

    Google Scholar 

  • Asanbe M, Osofisan A, William W (2016) Teachers’ performance evaluation in the higher educational institution using data mining technique. Int J Appl Inf Syst (IJAIS) 10:10–15

    Google Scholar 

  • Baradwaj BK, Pal S (2012) Mining educational data to analyse students’ performance. arXiv:1201.3417

  • Bergmann P, Fauser M, Sattlegger D, Steger C (2020) Uninformed students: Student-teacher anomaly detection with discriminative latent embeddings. In: Paper presented at the proceedings of the IEEE/CVF conference on computer vision and pattern recognition 4183–4192

  • Block E, Crochet F, Jones L, Papa T (2012) The importance of teacher’s effectiveness. Creat Educ 3(06):1164

    Article  Google Scholar 

  • Brna P (1999) Artificial intelligence in educational software: has its time come? Br J Educ Technol 30(1):79–81

    Article  Google Scholar 

  • Cheng X, Sun J, Zarifis A (2020) Artificial intelligence and deep learning in educational technology research and practice. Br J Educ Technol 51(5):1653–1656

    Article  Google Scholar 

  • Clark D (1993) Teacher evaluation: a review of the literature with implications for educators, Thesis paper, California State University at Long Beach, Retrieved from https://eric.ed.gov/?id=ED359174. Accessed 19 July 2023

  • Deb AB, Dey L (2017) Outlier detection and removal algorithm in K-means and hierarchical clustering. World J Comput Appl Technol 5(2):24–29

    Article  Google Scholar 

  • Doan S, Schweig JD, Mihaly K (2019) The consistency of composite ratings of teacher effectiveness: evidence from New Mexico. Am Educ Res J 56(6):2116–2146

    Article  Google Scholar 

  • Fahd K, Miah SJ, Ahmed K, Venkatraman S, Miao Y (2021) Integrating design science research and design based research frameworks for developing education support systems. Educ Inf Technol 26:4027–4048

    Article  Google Scholar 

  • Fahd K, Venkatraman S, Miah SJ, Ahmed K (2022) Application of machine learning in higher education to assess student academic performance, at-risk, and attrition: A meta-analysis of literature. Educ Inf Technol 27(1):1–33

    Google Scholar 

  • Goe L, Bell C, Little O (2008) Approaches to evaluating teacher effectiveness: a research synthesis. Evaluation report, National Comprehensive Center for Teacher Quality, p. 1-100. Retrieved from: https://files.eric.ed.gov/fulltext/ED521228.pdf. Accessed 24 July 2023

  • Goldhaber D, Anthony E (2007) Can teacher quality be effectively assessed? national board certification as a signal of effective teaching. Rev Econ Stat 89(1):134–150

    Article  Google Scholar 

  • Gregor S, Hevner AR (2013) Positioning and presenting design science research for maximum impact. MIS Q 37(2):337–355

    Article  Google Scholar 

  • Gupta A, Sharma H, Akhtar A (2021) A comparative analysis of K-means and hierarchical clustering EPRA. Int J Multidiscip Res (IJMR). https://doi.org/10.36713/epra8308

    Article  Google Scholar 

  • Heck RH (2009) Teacher effectiveness and student achievement: investigating a multilevel cross-classified model. J Educ Adm 47(2):227–249

    Article  Google Scholar 

  • Hemaid RK, El-Halees AM (2015) Improving teacher performance using data mining. Int J Adv Res Comput Commun Eng 4(2):407–412

    Article  Google Scholar 

  • Hershberg T, Simon VA, Lea-Kruger B (2004) The revelations of value-added: an assessment model that measures student growth in ways that NCLB fails to do. Sch Adm 61(11):10

    Google Scholar 

  • Hevner AR (2007) A three cycle view of design science research. Scand J Inf Syst 19(2):4

    Google Scholar 

  • Hevner AR, March ST, Park J, Ram S (2004) Design science in information systems research. MIS Q 28(1):75–105

    Article  Google Scholar 

  • Hevner A, Chatterjee S (2010) Design science research in information systems In: Shardr R, Vob S (eds) Design Research in Information Systems Theory and Practice. Springer, p 9–22. Retrieved from https://link.springer.com/chapter/10.1007/978-1-4419-5653-8_2. Accessed 24 July 2023

  • Irshadullah FKHM (2018) A review of the effect of education and good trained teachers on students’ performance. Humanit Soc Sci 25:93–99

    Google Scholar 

  • Järvelä S, Gašević D, Seppänen T, Pechenizkiy M, Kirschner PA (2020) Bridging learning sciences, machine learning and affective computing for understanding cognition and affect in collaborative learning. Br J Educ Technol 51(6):2391–2406

    Article  Google Scholar 

  • Järvinen P (2007) Action research is similar to design science. Qual Quant 41:37–54

    Article  Google Scholar 

  • Kane TJ, McCaffrey DF, Miller T, Staiger DO (2013) Have we identified effective teachers? validating measures of effective teaching using random assignment. In: Paper presented at the research paper. MET Project. Bill & Melinda Gates Foundation

  • Kumar S (2017) A modern data mining method for assessment of teaching assistant in higher educational institutions. Int J Comput Sci Inf Technol 8(3):424–429

    Google Scholar 

  • Lupascu AR, Pânisoară G, Pânisoară I-O (2014) Characteristics of effective teacher. Procedia Soc Behav Sci 127:534–538

    Article  Google Scholar 

  • Mardikyan S, Badur B (2011) Analyzing teaching performance of instructors using data mining techniques. Inform Educ 10(2):245–257

    Article  Google Scholar 

  • Miah SJ (2009) End user as application developer for decision support. In: Proceedings of the American conference on information systems, (AMCIS 2009), Sun Francesco, USA, 142

  • Miah SJ, McGrath GM, Kerr D (2016) Design science research for decision support systems development: recent publication trends in the premier IS journals. Australas J Inf Syst 20:1–14

    Google Scholar 

  • Mohan M, Hull RE (1975). Teaching effectiveness: Its meaning, assessment, and improvement. Englewood Cliffs, NJ: Educational Technology Publications. Retrieved from https://books.google.com.au/books?id=ouEyYjPAMxUC. Accessed 24 July 2023

  • Niemi H, Tirri K (1996) Effectiveness of teacher education. New challenges and approaches to evaluation. Reports from the Department of Teacher Education in Tampere University. ERIC

  • Ogor EN (2007) Student academic performance monitoring and evaluation using data mining techniques. In: Paper presented at the electronics, robotics and automotive mechanics conference (CERMA 2007)

  • Ola A, Pallaniappan S (2013) A data mining model for evaluation of instructors’ performance in higher institutions of learning using machine learning algorithms. Int J Concep Comput Inf Technol 1(1):17–22

  • Onafowora LL (2005) Teacher efficacy issues in the practice of novice teachers. Educ Res Q 28(4):34–43

    Google Scholar 

  • Orozco W, Rodríguez-García MÁ, Fernández A (2019) Teaching effectiveness: an innovative evaluation model. In: International workshop on learning technology for education in cloud. Springer International Publishing, Cham, pp 450–461

    Google Scholar 

  • Pal AK, Pal S (2013) Evaluation of teacher’s performance: a data mining approach. Int J Comput Sci Mob Comput 2(12):359–369

    Google Scholar 

  • Papamitsiou ZK, Economides AA (2014) Learning analytics and educational data mining in practice: a systematic literature review of empirical evidence. J Educ Technol Soc 17(4):49–64

    Google Scholar 

  • Parihar R (2011) Concept of Teacher Effectiveness. New Dehli: Jaypee Brother Publications Policy Implications. USA: University of Missouri-Columbia

  • Peffers K, Tuunanen T, Rothenberger MA, Chatterjee S (2007) A design science research methodology for information systems research. J Manag Inf Syst 24(3):45–77

    Article  Google Scholar 

  • Sanders WL (2000) Value-added assessment from student achievement data: opportunities and hurdles CREATE NATIONAL EVALUATION INSTITUTE July 21, 2000. J Pers Eval Educ 14(4):329–339

    Article  Google Scholar 

  • Shinkfield AJ, Stufflebeam DL (2012) Teacher evaluation: Guide to effective practice (Vol. 41). Springer, Berlin

    Google Scholar 

  • Skourdoumbis A (2014) Teacher effectiveness: making the difference to student achievement? Br J Educ Stud 62(2):111–126

    Article  Google Scholar 

  • Tschannen-Moran M, Hoy AW (2001) Teacher efficacy: capturing an elusive construct. Teach Teach Educ 17(7):783–805

    Article  Google Scholar 

  • Vaishnavi VK (2007) Design science research methods and patterns: innovating information and communication technology. Auerbach Publications, New York

    Book  Google Scholar 

  • Vaishnavi VK, Kuechler W (2007) Introduction to design science research in information and communication technology. In: Vaishnavi VK, Vaishnavi VK, Kuechler W (eds) Design science research methods and patterns: innovating information and communication technology. Taylor & Francis Group, United Kingdom

    Chapter  Google Scholar 

  • Vijayalakshmi V, Panimalar K, Janarthanan S (2020) Predicting the performance of instructors using Machine-learning algorithms. High Technol Lett 26(12):694–705

    Google Scholar 

  • Williams K, Hebert D (2020) Teacher evaluation systems: a literature review on issues and impact. Res Issues Contemp Educ 5(1):42–50

    Google Scholar 

  • Yadav SK, Bharadwaj B, Pal S (2012). Mining education data to predict student’s retention: a comparative study. arXiv:1203.2987

  • Zakaria I, Nor MYBM, Alias B (2021) The effect of teachers’ professionalism on students’ success. Int J Acad Res Bus Soc Sci 11(1):483–500

    Google Scholar 

  • Zawacki-Richter O, Marín VI, Bond M, Gouverneur F (2019) Systematic review of research on artificial intelligence applications in higher education—where are the educators. Int J Educ Technol High Educ. https://doi.org/10.1186/s41239-019-0171-0

    Article  Google Scholar 

  • Zhou H, Lu J, Huang Y, Chen Y (2021) Research on key technology of classroom teaching evaluation based on artificial intelligence. In Journal of Physics: Conference series (Vol. 1757, No. 1, p. 012014). IOP publishing

Download references

Author information

Authors and Affiliations

  1. University of Newcastle, Newcastle, Australia

    Renu Sabharwal & Shah J. Miah

Authors
  1. Renu Sabharwal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shah J. Miah
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Both authors contribute equally to develop the manuscript.

Corresponding author

Correspondence to Shah J. Miah.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix 1: Questions

Appendix 1: Questions

  • instr: instructor’s identifier; values taken from {1, 2, 3}

  • class: course code (descriptor); values taken from {1–13}

  • repeat: number of times the student is taking this course; values are taken from {0, 1, 2, 3, …}

  • attendance: code of the level of attendance; values from {0, 1, 2, 3, 4}

  • difficulty: level of difficulty of the course as perceived by the student; values taken from {1, 2, 3, 4, 5}

  • Q1: the semester course content, teaching method and evaluation system were provided initially.

  • Q2: the course aims and objectives were clearly stated at the beginning of the period.

  • Q3: the course was worth the number of credits assigned to it.

  • Q4: the course was taught according to the syllabus announced on the first day of class.

  • Q5: the class discussions, homework assignments, applications, and studies were satisfactory.

  • Q6: the textbook and other course resources were sufficient and up to date.

  • Q7: the course allowed for fieldwork, applications, laboratory work, discussions, and other studies.

  • Q8: the quizzes, assignments, projects, and exams contributed to the learning.

  • Q9: i greatly enjoyed the class and actively participated during the lectures.

  • Q10: my initial expectations about the course were met at the end of the period or year.

  • Q11: the course was relevant and beneficial to my professional development.

  • Q12: the course helped me look at life and the world from a new perspective.

  • Q13: the instructor’s knowledge was relevant and up to date.

  • Q14: the instructor came prepared for classes.

  • Q15: the instructor taught in accordance with the announced lesson plan.

  • Q16: the instructor was committed to the course and was understandable.

  • Q17: the instructor arrived on time for classes.

  • Q18: the instructor has a smooth and easy-to-follow delivery/speech.

  • Q19: the instructor made effective use of class hours.

  • Q20: the instructor explained the course and was eager to help students.

  • Q21: the instructor demonstrated a positive approach to students.

  • Q22: the instructor was open and respectful of students’ views about the course.

  • Q23: the instructor encouraged participation in the course.

  • Q24: the instructor gave relevant homework assignments/projects and helped/guided students.

  • Q25: the instructor responded to questions about the course inside and outside of the course.

  • Q26: the instructor’s evaluation system (midterm and final questions, projects, assignments) effectively measured the course objectives.

  • Q27: the instructor provided solutions to exams and discussed them with students.

  • Q28: the instructor treated all students in a right and objective manner.

Note: Q1–Q28 are all Likert-type, meaning that the values are taken from {1, 2, 3, 4, 5}.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sabharwal, R., Miah, S.J. Evaluating teachers’ effectiveness in classrooms: an ML-based assessment portfolio. Soc. Netw. Anal. Min. 14, 28 (2024). https://doi.org/10.1007/s13278-023-01195-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s13278-023-01195-5

Keywords

Search

Navigation