Abstract
Student rating instruments are recognised to be valid indicators of effective instruction, providing a valuable tool to improve teaching. However, free-form text comments obtained from the open-ended question component of such surveys are only infrequently analysed comprehensively. We employed an innovative, systematic approach to the analysis of text-based feedback relating to student perceptions of and experiences with a recently developed university program. The automated nature of the semantic analysis tool Leximancer enabled a critical interrogation across units of study, mining the cumulative text for common themes and recurring core concepts. The results of this analysis facilitated the identification of issues that were not apparent from the purely quantitative data, thus providing a deeper understanding of the curriculum and teaching effectiveness that was constructive and detailed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The term “academic staff” is to be understood as “faculty members” in North American parlance.
The word “unit” is to be understood as “course” in North American parlance.
References
Abd-Elrahman, A., Andreu, M., & Abbott, T. (2010). Using text data mining techniques for understanding free-style question answers in course evaluation forms. Research in Higher Education Journal, 9, 12–23.
Alhija, F. N.-A., & Fresko, B. (2009). Student evaluation of instruction: What can be learned from students’ written comments? Studies in Educational Evaluation, 35, 37–44.
Brockx, B., Van Roy, K., & Mortelmans, D. (2012). The student as a commentator: Students’ comments in student evaluations of teaching. Procedia-Social and Behavioral Sciences, 69, 1122–1133.
Brown, A., & Vosper, H. (2013). Development of a blended learning environment to support achievement of graduate outcomes through optimal learning in an undergraduate pharmacy course. Pharmacy, 1, 204–217.
Caron, G., Visentin, S., & Ermondi, G. (2011). Blended-learning for courses in pharmaceutical analysis. Journal of e-Learning and Knowledge Society, 7, 93–102.
Cashin, W. E. (1995). Student ratings of teaching: The research revisited. IDEA Paper No. 32. Manhattan, KS: Kansas State University, Center for Faculty Evaluation and Development.
Chickering, A. W., & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education. AAHE bulletin, 3, 3–7.
Crowsey, M.J., Ramstad, A.R., Gutierrez, D.H., Paladino, G.W., and White, K.P. (2007, April). An evaluation of unstructured text mining software. IEEE Systems and Information Engineering Design Symposium. University of Virginia, Charlottesville, VA. Retrieved from http://www.sys.virginia.edu/sieds07/papers/SIEDS07_0007_FI.pdf
Denson, N., Loveday, T., & Dalton, H. (2010). Student evaluation of courses: What predicts satisfaction? Higher Education Research & Development, 29, 339–356.
Eiszler, C. F. (2002). College students’ evaluations of teaching and grade inflation. Research in Higher Education, 43, 483–501.
Graham, C., Cagiltay, K., Lim, B., Craner, J., & Duffy, T. M. (2001). Seven principles of effective teaching: A practical lens for evaluating online courses. The Technology Source, March//April. Retrieved from http://www.technologysource.org/article/274/?utm_content=buffere64be&utm_source=buffer&utm_medium=twitter&utm_campaign=Buffer
Hodges, L. C., & Stanton, K. (2007). Translating comments on student evaluations into the language of learning. Innovative Higher Education, 31, 279–286.
Hoon, A., Oliver, E., Szpakowska, K., & Newton, P. (2014). Use of the “Stop, Start, Continue” method is associated with the production of constructive qualitative feedback by students in higher education. Assessment & Evaluation in Higher Education. doi:10.1080/02602938.2014.956282
Jordan, D. W. (2011). Re-thinking student written comments in course evaluations: Text mining unstructured data for program and institutional assessment (Doctoral dissertation). California State University, Stanislaus.
Kyndt, E., Berghmans, I., Dochy, F., & Bulckens, L. (2013). “Time is not enough.” Workload in higher education: A student perspective. Higher Education Research & Development, 33, 684–698.
Mandernach, B. J., Donnelli, E., Dailey, A., & Schulte, M. (2005). A faculty evaluation model for online instructors: Mentoring and evaluation in the online classroom. Online Journal of Distance Learning Administration, 8(3). Retrieved from http://www.westga.edu/~distance/ojdla/fall83/mandernach83.htm
Marsh, H. W., & Roche, L. A. (1997). Making students’ evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility. American Psychologist, 52, 1187–1197.
Schindel, T. J., Hughes, C. A., & Sadowski, C. A. (2013). Blended learning: Reflections on teaching experiences across the pharmacy education continuum. Pharmacy, 1, 137–152.
Smith, A. E., & Humphreys, M. S. (2006). Evaluation of unsupervised semantic mapping of natural language with Leximancer concept mapping. Behavior Research Methods, 38, 262–279.
Spooren, P., Brockx, B., & Mortelmans, D. (2013). On the validity of student evaluation of teaching: The state of the art. Review of Educational Research, 83, 598–642.
Thomas, D. A. (2014). Searching for significance in unstructured data: Text mining with Leximancer. European Educational Research Journal, 13, 235–256.
Wood, V., Eccott, L., & Bainbridge, L. (2013). A blended active learning pilot: A way to deliver interprofessional pain management education. Pharmacy, 1, 218–227.
Young, S. (2006). Student views of effective online teaching in higher education. The American Journal of Distance Education, 20, 65–77.
Zabaleta, F. (2007). The use and misuse of student evaluations of teaching. Teaching in Higher Education, 12, 55–76.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Stupans, I., McGuren, T. & Babey, A.M. Student Evaluation of Teaching: A Study Exploring Student Rating Instrument Free-form Text Comments. Innov High Educ 41, 33–42 (2016). https://doi.org/10.1007/s10755-015-9328-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10755-015-9328-5