Skip to main content
SpringerLink
Log in

Evaluating higher education teaching performance using combined analytic hierarchy process and data envelopment analysis

  • Published:
Journal of the Operational Research Society

Abstract

Evaluating higher education teaching performance is complex as it involves consideration of both objective and subjective criteria. The student evaluation of teaching (SET) is used to improve higher education quality. However, the traditional approaches to considering students’ responses to SET questionnaires for improving teaching quality have several shortcomings. This study proposes an integrated approach to higher education teaching evaluation that combines the analytical hierarchy process (AHP) and data envelopment analysis (DEA). The AHP allows consideration of the varying importance of each criterion of teaching performance, while DEA enables the comparison of tutors on teaching as perceived by students with a view to identifying the scope for improvement by each tutor. The proposed teaching evaluation method is illustrated using data from a higher education institution in Greece.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4

Similar content being viewed by others

References

  • Badri M and Abdulla M (2004). Awards of excellence in institutions of higher education: An AHP approach. International Journal of Educational Management 18(4):224–242.

    Google Scholar 

  • Badri M, Abdulla M, Kamali M and Dodeen H (2006). Identifying potential biasing variables in student evaluation of teaching in a newly accredited business program in the UAE. International Journal of Educational Management 20(1):43–59.

    Google Scholar 

  • Chen J, Hsieh H and Do QH (2015). Evaluating teaching performance based on fuzzy AHP and comprehensive evaluation approach. Applied Soft Computing 28:100–108.

    Article  Google Scholar 

  • Crumbley DL and Fliedner E (2002). Accounting administrators’ perceptions of student evaluation of teaching (SET) information. Quality Assurance in Education 10(4):213–222.

    Article  Google Scholar 

  • Crumbley DL and Reichelt KJ (2009). Teaching effectiveness, impression management, and dysfunctional behavior: Student evaluation of teaching control data. Quality Assurance in Education 17(4):377–392.

    Article  Google Scholar 

  • Crumbley L, Henry BK and Kratchman SH (2001). Students’ perceptions of the evaluation of college teaching. Quality Assurance in Education 9(4):197–207.

    Article  Google Scholar 

  • De Witte K and Rogge N (2011). Accounting for exogenous influences in performance evaluations of teachers. Economics of Education Review 30(4):641–653.

    Article  Google Scholar 

  • Dong P and Dai F (2009). Evaluation for teaching quality based on fuzzy neural network. Proceedings of the First International Workshop on Education Technology and Computer Science 1:112–115.

    Article  Google Scholar 

  • Douglas J, Douglas A and Barnes B (2006). Measuring student satisfaction at a UK university. Quality Assurance in Education 14(3):251–267.

    Article  Google Scholar 

  • DuBois PH and Garson E (1970). A History of Psychological Testing. Allyn and Bacon: Boston.

    Google Scholar 

  • Ferreira Filho AJ, Salomon VA and Marins FA (2007). Measuring the efficiency of outsourcing: An illustrative case study from the aerospace industry. In Loureiro G and Curran R (eds). Complex Systems Concurrent Engineering. Springer: New York, pp 819–826.

  • Fitzpatrick JL, Sanders JR and Worthen BR (2004). Program Evaluation: Alternative Approaches and Practical Guidelines.

  • Frey PW, Leonard DW and Beatty WW (1975). Student ratings of instruction: Validation research. American Educational Research Journal 12(4):435–444.

    Article  Google Scholar 

  • Gray M and Bergmann BR (2003). Student teaching evaluations: Inaccurate, demeaning, misused. Academe 89(5):44–46.

    Article  Google Scholar 

  • Guba EG and Lincoln YS (1981). Effective Evaluation: Improving the Usefulness of Evaluation Results Through Responsive and Naturalistic Approaches. Jossey-Bass.

  • Haladyna T and Hess RK (1994). The detection and correction of bias in student ratings of instruction. Research in Higher Education 35(6):669–687.

    Article  Google Scholar 

  • Hildebrand M (1971). Evaluating University Teaching.

  • Kuzmanovic M, Savic G, Gusavac BA, Makajic-Nikolic D and Panic B (2013). A Conjoint-based approach to student evaluations of teaching performance. Expert Systems with Applications 40(10):4083–4089.

    Article  Google Scholar 

  • Marsh HW (1982). SEEQ: A reliable, valid, and useful instrument for collecting students’evaluations of university teaching. British Journal of Educational Psychology 52(1):77–95.

    Article  Google Scholar 

  • Marsh HW (1983). Multidimensional ratings of teaching effectiveness by students from different academic settings and their relation to student/course/instructor characteristics. Journal of Educational Psychology 75(1):150.

    Article  Google Scholar 

  • Marsh HW (1984). Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential biases, and utility. Journal of Educational Psychology 76(5):707.

    Article  Google Scholar 

  • Marsh HW (1987). Students’ evaluations of university teaching: Research findings, methodological issues, and directions for future research. International Journal of Educational Research 11(3):253–388.

    Article  Google Scholar 

  • Marsh HW and Hocevar D (1991). The multidimensionality of students’ evaluations of teaching effectiveness: The generality of factor structures across academic discipline, instructor level, and course level. Teaching and Teacher Education 7(1): 9–18.

    Article  Google Scholar 

  • Marsh HW and Roche LA (1997). Making students’ evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility. American Psychologist 52(11):1187.

    Article  Google Scholar 

  • O’Hanlon J and Mortensen L (1980). Making teacher evaluation work. The Journal of Higher Education 51(6):664–672.

    Article  Google Scholar 

  • Ramli N, Mohamad D and Sulaiman, NH (2010). Evaluation of teaching performance with outliers data using fuzzy approach. Procedia-Social and Behavioral Sciences 8(1):190–197.

    Article  Google Scholar 

  • Rogge N (2011). Granting teachers the “benefit of the doubt” in performance evaluations. International Journal of Educational Management 25(6):590–614.

    Google Scholar 

  • Saaty T (1996). The Analytic Hierarchy Process, Volume 3 and 4. RWS Publishing: Pittsburgh.

  • Sueyoshi T, Shang J and Chiang W-C (2009). A decision support framework for internal audit prioritization in a rental car company: A combined use between DEA and AHP. European Journal of Operational Research 199(1):219–231.

    Article  Google Scholar 

  • Thanassoulis E (2001). Introduction to the Theory and Application of Data Envelopment Analysis. Kluwer Academic Publishers: Massachusettes.

    Book  Google Scholar 

  • Theofilidis C (1989). The Meta-Evaluation of the Evaluation of Programs (in Greek). New Education Publications: Athens.

    Google Scholar 

  • Tsimboukis K (1979). Measurement and Evaluation in the Educational Sciences (in Greek). Orosimo Publications: Athens.

    Google Scholar 

  • Tsinidou M, Gerogiannis V and Fitsilis P (2010). Evaluation of the factors that determine quality in higher education: An empirical study. Quality Assurance in Education 18(3): 227–244.

    Article  Google Scholar 

  • Uttl B, White CA and Gonzalez DW (2016). Meta-analysis of faculty’s teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation.

  • Warrington WG (1973). Student evaluation of instruction at Michigan State University. In Proceedings: The First Invitational Conference on Faculty Effectiveness as Evaluated by Students, pp 164–182.

  • Worthen BR and Sanders JR (1987). Educational evaluation: Alternative approaches and practical guidelines. Longman Pub Group: New York.

    Google Scholar 

  • Yang T and Kuo C (2003). A hierarchical AHP/DEA methodology for the facilities layout design problem. European Journal of Operational Research 147(1):128–136.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Operations and Information Management Group, Aston Business School, Aston University, Aston Triangle, Birmingham, B4 7ET, UK

    Emmanuel Thanassoulis, Prasanta Kumar Dey & Konstantinos Petridis

  2. Department of Business Administration, University of Macedonia, 156 Egnatia Street, 54636, Thessaloniki, Greece

    Ioannis Goniadis & Andreas C. Georgiou

Authors
  1. Emmanuel Thanassoulis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Prasanta Kumar Dey
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Konstantinos Petridis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ioannis Goniadis
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Andreas C. Georgiou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Prasanta Kumar Dey.

Appendix: The SET questionnaire

Appendix: The SET questionnaire

A usual 5-point Likert scale is used by the student to express his/her level of agreement with the statement

Group 1 General statements

  1. 1.

    The overall performance of the teacher was good.

  2. 2.

    The quality of the course was high.

Group 2 Course Evaluation

  1. 3.

    The organization and the presentation of the course were complete.

  2. 4.

    The subject of the course was interesting and useful for your studies.

  3. 5.

    The course material (books, handouts, slides, exercises, papers, etc.) was satisfactory for the course needs.

Group 3 Teacher Evaluation

  1. 6.

    The tutor was well prepared for the class.

  2. 7.

    The tutor had good communication skills.

  3. 8.

    The tutor encouraged questions and in general the participation in class.

  4. 9.

    Whenever I needed to meet the tutor for discussing questions or problems, he/she was there during his/hers office hours.

  5. 10.

    The tutor was punctual for the classes.

Group 4 Evaluation of Supportive Classes and Supportive Teaching Staff (to be answered only if supportive classes exist).

  1. 11.

    The quality of the supportive classes was high.

  2. 12.

    The overall performance of the supportive teaching staff was good.

Group 5 Other Questions (specific scales)

  1. 13.

    Classes attend frequency (not obligatory attendance)

1 = not at all, 2 = rarely, 3 = often, 4 = very often, 5 = always

  1. 14.

    According to your experience with other courses, you would characterize this course as:

1 = very easy, 2 = easy, 3 = of average difficulty, 4 = difficult, 5 = very difficult

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Thanassoulis, E., Dey, P.K., Petridis, K. et al. Evaluating higher education teaching performance using combined analytic hierarchy process and data envelopment analysis. J Oper Res Soc 68, 431–445 (2017). https://doi.org/10.1057/s41274-016-0165-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1057/s41274-016-0165-4

Keywords

Search

Navigation