Skip to main content

Reducing Annotation Effort in Automatic Essay Evaluation Using Locality Sensitive Hashing

Part of the Lecture Notes in Computer Science book series (LNPSE,volume 11528)

Abstract

Automated essay evaluation systems use machine learning models to predict the score for an essay. For such, a training essay set is required which is usually created by human requiring time-consuming effort. Popular choice for scoring is a nearest neighbor model which requires on-line computation of nearest neighbors to a given essay. This is, however, a time-consuming task. In this work, we propose to use locality sensitive hashing that helps to select a small subset of a large set of essays such that it will likely contain the nearest neighbors for a given essay. We provided experiments on real-world data sets provided by Kaggle. According to the experimental results, it is possible to achieve good performance on scoring by using the proposed approach. The proposed approach is efficient with regard to time complexity. Also, it works well in case of a small number of training essays labeled by human and gives comparable results to the case when a large essay sets are used.

Keywords

  • Locality Sensitive Hashing
  • Automatic essay scoring
  • Similarity search

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-22244-4_23
  • Chapter length: 7 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   54.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-22244-4
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   69.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.

Notes

  1. 1.

    https://www.kaggle.com/c/asap-sas

References

  1. Heilman, M., Madnani, N.: The impact of training data on automated short answer scoring performance. In: Tenth Workshop on Innovative Use of NLP for Building Educational Applications, pp. 81–85 (2015)

    Google Scholar 

  2. Brooks, M., Basu, S., Jacobs, C., Vanderwende, L.: Divide and correct: using clusters to grade short answers at scale. In: The First ACM Conference on Learning @ Scale Conference, pp. 89–98. ACM, New York (2014)

    Google Scholar 

  3. Zesch, T., Heilman, M., Cahill, A.: Reducing annotation efforts in supervised short answer scoring. In: Tenth Workshop on Innovative Use of NLP for Building Educational Applications, pp. 124–132 (2015)

    Google Scholar 

  4. Slaney, M., Casey, M.: Locality-sensitive hashing for finding nearest neighbors [Lecture Notes]. IEEE Signal Process. Mag. 25, 128–131 (2008)

    CrossRef  Google Scholar 

  5. Kim, Y.B., Reilly, U.O.: Large-scale physiological waveform retrieval via locality-sensitive hashing, pp. 5829–5833 (2015)

    Google Scholar 

  6. Horbach, A., Palmer, A., Wolska, M.: Finding a tradeoff between accuracy and rater’s workload in grading clustered short answers. In: The 9th Language Resources and Evaluation Conference (LREC), pp. 588–595 (2014)

    Google Scholar 

  7. Basu, S., Jacobs, C., Vanderwende, L.: Powergrading: a clustering approach to amplify human effort for short answer grading. Trans. ACL (2013)

    Google Scholar 

  8. Misikir Tashu, T., Horvath, T.: Pair-wise: automatic essay evaluation using word mover’s distance. In: 10th International Conference on Computer Supported Education, CSEDU, vol. 2, pp. 59–66. SciTePress (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Tsegaye Misikir Tashu .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Tashu, T.M., Szabó, D., Horváth, T. (2019). Reducing Annotation Effort in Automatic Essay Evaluation Using Locality Sensitive Hashing. In: Coy, A., Hayashi, Y., Chang, M. (eds) Intelligent Tutoring Systems. ITS 2019. Lecture Notes in Computer Science(), vol 11528. Springer, Cham. https://doi.org/10.1007/978-3-030-22244-4_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-22244-4_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-22243-7

  • Online ISBN: 978-3-030-22244-4

  • eBook Packages: Computer ScienceComputer Science (R0)