Abstract
Automated essay evaluation systems use machine learning models to predict the score for an essay. For such, a training essay set is required which is usually created by human requiring time-consuming effort. Popular choice for scoring is a nearest neighbor model which requires on-line computation of nearest neighbors to a given essay. This is, however, a time-consuming task. In this work, we propose to use locality sensitive hashing that helps to select a small subset of a large set of essays such that it will likely contain the nearest neighbors for a given essay. We provided experiments on real-world data sets provided by Kaggle. According to the experimental results, it is possible to achieve good performance on scoring by using the proposed approach. The proposed approach is efficient with regard to time complexity. Also, it works well in case of a small number of training essays labeled by human and gives comparable results to the case when a large essay sets are used.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Heilman, M., Madnani, N.: The impact of training data on automated short answer scoring performance. In: Tenth Workshop on Innovative Use of NLP for Building Educational Applications, pp. 81–85 (2015)
Brooks, M., Basu, S., Jacobs, C., Vanderwende, L.: Divide and correct: using clusters to grade short answers at scale. In: The First ACM Conference on Learning @ Scale Conference, pp. 89–98. ACM, New York (2014)
Zesch, T., Heilman, M., Cahill, A.: Reducing annotation efforts in supervised short answer scoring. In: Tenth Workshop on Innovative Use of NLP for Building Educational Applications, pp. 124–132 (2015)
Slaney, M., Casey, M.: Locality-sensitive hashing for finding nearest neighbors [Lecture Notes]. IEEE Signal Process. Mag. 25, 128–131 (2008)
Kim, Y.B., Reilly, U.O.: Large-scale physiological waveform retrieval via locality-sensitive hashing, pp. 5829–5833 (2015)
Horbach, A., Palmer, A., Wolska, M.: Finding a tradeoff between accuracy and rater’s workload in grading clustered short answers. In: The 9th Language Resources and Evaluation Conference (LREC), pp. 588–595 (2014)
Basu, S., Jacobs, C., Vanderwende, L.: Powergrading: a clustering approach to amplify human effort for short answer grading. Trans. ACL (2013)
Misikir Tashu, T., Horvath, T.: Pair-wise: automatic essay evaluation using word mover’s distance. In: 10th International Conference on Computer Supported Education, CSEDU, vol. 2, pp. 59–66. SciTePress (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Tashu, T.M., Szabó, D., Horváth, T. (2019). Reducing Annotation Effort in Automatic Essay Evaluation Using Locality Sensitive Hashing. In: Coy, A., Hayashi, Y., Chang, M. (eds) Intelligent Tutoring Systems. ITS 2019. Lecture Notes in Computer Science(), vol 11528. Springer, Cham. https://doi.org/10.1007/978-3-030-22244-4_23
Download citation
DOI: https://doi.org/10.1007/978-3-030-22244-4_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-22243-7
Online ISBN: 978-3-030-22244-4
eBook Packages: Computer ScienceComputer Science (R0)