Skip to main content
SpringerLink
Log in

Maximizing the Value of Student Ratings Through Data Mining

  • Chapter
  • First Online:
Educational Data Mining

Part of the book series: Studies in Computational Intelligence ((SCI,volume 524))

Abstract

Student ratings of instruction are an important means of assessment within universities and have been the focus of much study over the last 50 years. Until very recently it has been difficult to perform meaningful analysis of student narrative comments given that most universities collected them as hand-written notes. This work uses statistical and text mining techniques to analyze a data set consisting of over 1 million student comments that were collected using an online process. The methodology makes use of positive and negative “category vectors” representing instructor characteristics and a domain-specific lexicon. Sentiment analysis is used to detect and gauge attitudes embedded in comments about each category. The methodology is validated using three approaches, two quantitative and one qualitative. While useful to individual instructors and administrators, it is only through data mining that student perceptions of teaching can be analyzed en masse to inform and influence the educational process.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Abbreviations

KWIC:

Keyword in context

MOOC:

Massively open online courses

NAR1:

What are some positive characteristics or strengths of the course and/or instructor?

NAR2:

What are some negative characteristics or weaknesses of the course and/or instructor?

Q10:

How would you rate the difficulty level of this course, compared to other courses you have taken so far at Ole Miss?

Q11:

How would you rate the instructor’s overall performance in this course?

SEEQ:

Students’ evaluation of education quality

References

  1. Feldman, K.A.: Effective college teaching from the students’ and faculty’s view: matched and mismatched priorities. Res. High. Educ. 28(4), 291–329 (1988)

    Article  Google Scholar 

  2. Marsh, H.W., Bailey, M.: Multidimensional students’ evaluations of teaching effectiveness. J. Higher Educ. 64(1), 1–18 (1993)

    Article  Google Scholar 

  3. Cashin, W.E.: Student ratings of teaching: the research revisited. IDEA paper No. 32. center for faculty evaluation and development, Kansas State University, Manhattan, 1995

    Google Scholar 

  4. Rindermann, H.: Generalizability of multidimensional student ratings of university instruction across courses and teachers. Res. High. Educ. 42(4), 377–399 (2001)

    Article  Google Scholar 

  5. Abrami, P.C., d’Apollonia, S., Rosenfield, S.: The dimensionality of student ratings of instruction: what we know and what we do not. In: Perry, R.P., Smart, J.C. (eds.) The Scholarship of Teaching and Learning in Higher Education: An Evidence-Based Perspective, pp. 385–456. Springer, The Netherlands (2007)

    Chapter  Google Scholar 

  6. Jin, W., Ho, H., Srihar, R.: OpinionMiner: a novel machine learning system for web opinion mining and extraction. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1195–1204. ACM, New York (2009)

    Google Scholar 

  7. Kim, S.M., Calvo, R.A.: Sentiment analysis in student experiences of learning. In: Baker, R.S.J.D., Merceron, A., Pavlik Jr., P.I. (eds.) Proceedings of 3rd International Conference on Educational Data Mining, pp. 111–120. International Educational Data Mining Society, Pittsburgh (2010)

    Google Scholar 

  8. Cashin, W.E.: Student ratings of teaching: recommendations for use. IDEA paper No. 22. Technical report, Center for Faculty Evaluation and Development, Kansas State University, Manhattan, 1990

    Google Scholar 

  9. Stowell, J., Addison, W., Smith, J.: Comparison of online and classroom-based student evaluations of instruction. Assess. Eval. High. Educ. 37(4), 465–473 (2012)

    Article  Google Scholar 

  10. Avancini, H., Lavelli, A., Magnini, B., Sebastiani, F., Zonoli, R.: Expanding domain-specific lexicons by term categorization. In: ACM Symposium on Applied Computing, pp. 793–797. ACM, New York (2003)

    Google Scholar 

  11. Sahlgren, M., Coster, R.: Using bag-of-concepts to improve the performance of support vector machines in text categorization. In: Proceedings of 20th International Conference on Computational Linguistics, Geneva, pp. 487–493. Association for Computational Linguistics, Stroudsburg (2004)

    Google Scholar 

  12. Liu, B.: Sentiment analysis and subjectivity. In: Indurkhya, N., Damerau, F.J. (eds.) Handbook of Natural Language Processing, pp. 627–666. CRC Press, Boca Raton (2010). (Chapman and Hall/CRC Machine Learning and Pattern Recognition series)

    Google Scholar 

  13. Liu, B. (ed.): Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data, 2nd edn. Springer, Heidelberg (2011)

    Google Scholar 

  14. Simmons, L., Conlon, S., Mukhopadhyay, S., Yang, J.A.: Computer aided content analysis of online reviews. J. Comput. Inf. Syst. 52(1), 43–55 (2011)

    Google Scholar 

  15. Taboada, M., Brooke, J., Tofiloski, M., Voll, K., Stede, M.: Lexicon-based methods for sentiment analysis. Comput. Linguist. 37(2), 267–307 (2011)

    Article  Google Scholar 

  16. Neviarouskaya, A., Prendinger, H., Ishizuka, M.: SentiFul: a lexicon for sentiment analysis. IEEE Trans. Affect. Comput. 2(1), 22–36 (2011)

    Article  Google Scholar 

  17. Leong, C.K., Lee, Y.H., Mak, W.K.: Mining sentiments in SMS texts for teaching evaluation. Expert Syst. Appl. 39(3), 2584–2589 (2012)

    Article  Google Scholar 

  18. Vialardi, C., Bravo, J., Shafti, L., Ortigosa, A.: Recommendation in higher education using data mining techniques. In: Barnes, T., Desmarais, M., Romero, R., Ventura, S. (eds.) Proceedings of 2nd International Conference on Educational Data Mining, pp. 190–199. International Educational Data Mining Society, Cordoba (2009)

    Google Scholar 

  19. Marsh, H.W.: SEEQ: a reliable, valid, and useful instrument for collecting students’ evaluations of university teaching. Br. J. Educ. Psychol. 52, 77–95 (1982)

    Article  Google Scholar 

  20. Hu., M., Liu, B.: Mining and summarizing customer reviews. In: Proceedings of Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 168–177. ACM, New York (2004)

    Google Scholar 

  21. Luhn, H.P.: Keyword-in-context index for technical literature. Am. Doc. 11(4), 288–295 (1960)

    Article  Google Scholar 

  22. SentiWordNet, http://sentiwordnet.isti.cnr.it/

  23. Big Huge Thesaurus, http://words.bighugelabs.com/

  24. Dey, L., Haque, M.: Opinion mining from noisy text data. Int. J. Doc. Anal. Recogn. 12(3), 205–226 (2009)

    Article  Google Scholar 

  25. Jia, L., Yu, C., Meng, W.: The effect of negation on sentiment analysis and retrieval effectiveness. In: Proceedings of 18th ACM International Conference on Information and Knowledge Management, pp. 1827–1830. ACM, New York (2009)

    Google Scholar 

  26. Qiu, G., Liu, B., Bu, J., Chen, C.: Opinion word expansion and target extraction through double propagation. Comput. Linguist. 37(1), 9–27 (2011)

    Article  Google Scholar 

  27. LingPipe, http://alias-i.com/lingpipe/

  28. Hardy, N.: Online ratings: fact or fiction. New Dir. Teach. Learn. 2003(96), 31–38 (2003)

    Article  Google Scholar 

  29. Elsie, M.: Hood outstanding teaching award at the University of Mississippi, http://www.olemiss.edu/news/hood_award/

  30. Melville, P., Gryc, W., Lawrence, R.: Sentiment analysis of blogs by combining lexical knowledge with text classification. In: Proceedings of 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1275–1284. (2009)

    Google Scholar 

  31. Paltoglou, G., Thelwall, M.: A study of information retrieval weighting schemes for sentiment analysis. In: Proceedings of 48th Annual Meeting of the Association for Computational Linguistics, pp. 1386–1395. Association for Computational Linguistics, Stroudsburg (2010)

    Google Scholar 

  32. Agarwal, B., Mittal, N.: Optimal feature selection for sentiment analysis. In: Gelbukh, A. (ed.) Computational Linguistics and Intelligent Text Processing, vol. 7817, pp. 13–24. LNCSSpringer, Heidelberg (2013)

    Chapter  Google Scholar 

  33. Elberrichi, Z., Rahmoun, A., Bentaalah, M.: Using WordNet for text categorization. Int. Arab. J. Inf. Technol. 5(1), 16–24 (2008)

    Google Scholar 

  34. Campagni, R., Merlini, D., Sprugnoli, R., Verri, M. C.: Comparing examination results and course evaluation: a data mining approach. In: Didamatica 2013, pp. 883–892. Scuola Superiore Sant′Anna, Pisa (2013)

    Google Scholar 

  35. Sap Hana, http://www.saphana.com/

Download references

Acknowledgments

This research was made possible through the support of several University of Mississippi units: The Office of the Provost; The Center for Excellence in Teaching and Learning; and The Office of Information Technology. The authors wish to thank Sarah Hill for her help in coordinating the qualitative assessment process.

Author information

Authors and Affiliations

  1. Office of Information Technology, The University of Mississippi, 300 Powers Hall, University, Oxford, MS, 38677, USA

    Kathryn Gates

  2. Department of Computer and Information Science, The University of Mississippi, 215 Weir Hall, University, Oxford, MS, 38677, USA

    Dawn Wilkins

  3. Department of Management Information Systems, The University of Mississippi, 247 Holman Hall, University, Oxford, MS, 38677, USA

    Sumali Conlon

  4. Center for Excellence in Teaching and Learning, The University of Mississippi, 106 Hill Hall, University, Oxford, MS, 38677, USA

    Susan Mossing

  5. Office of the Provost, The University of Mississippi, 137 Lyceum, University, Oxford, MS, 38677, USA

    Maurice Eftink

Authors
  1. Kathryn Gates
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dawn Wilkins
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sumali Conlon
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Susan Mossing
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Maurice Eftink
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kathryn Gates .

Editor information

Editors and Affiliations

  1. Escuela Superior de Ingeniería Mecánica y Eléctrica, Zacatenco (ESIME-Z), World Outreach Light to the Nations Ministries (WOLNM), Instituto Politécnico Nacional (IPN), Gustavo A. Madero, Mexico City, Distrito Federal, Mexico

    Alejandro Peña-Ayala

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Gates, K., Wilkins, D., Conlon, S., Mossing, S., Eftink, M. (2014). Maximizing the Value of Student Ratings Through Data Mining. In: Peña-Ayala, A. (eds) Educational Data Mining. Studies in Computational Intelligence, vol 524. Springer, Cham. https://doi.org/10.1007/978-3-319-02738-8_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-02738-8_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-02737-1

  • Online ISBN: 978-3-319-02738-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation