Skip to main content
SpringerLink
Log in

A Study of Distributed Semantic Representations for Automated Essay Scoring

  • Conference paper
  • First Online:
Knowledge Science, Engineering and Management (KSEM 2017)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10412))

Abstract

Automated essay scoring (AES) applies machine learning and NLP techniques to automatically rate essays written in an educational setting, by which the workload of human raters is considerably reduced. Current AES systems utilize common text features such as essay length, tf-idf weight, and the number of grammar errors to learn a scoring function. Despite the effectiveness brought by those common features, the semantics within the essay text is not well considered. To this end, this paper presents a study of the usefulness of the distributed semantic representations to AES. Novel features based on word or paragraph embeddings are combined with the common text features in order to improve the effectiveness of the AES systems. Evaluation results show that the use of the distributed semantic representations are beneficial for the task of AES.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.languagetool.org.

  2. 2.

    http://nlp.stanford.edu/software/corenlp.shtml.

  3. 3.

    https://drive.google.com/file/d/0B7XkCwpI5KDYNlNUTTlSS21pQmM/edit?usp=sharing.

  4. 4.

    https://www.kaggle.com/c/asap-aes/data.

  5. 5.

    https://www.kaggle.com/c/asap-aes/details/evaluation.

  6. 6.

    http://svmlight.joachims.org/.

References

  1. Attali, Y., Burstein, J.: Automated essay scoring with e-rater. J. Technol. Learn. Assess. 4(3), 7–15 (2006)

    Google Scholar 

  2. Dikli, S.: An overview of automated scoring of essays. J. Technol. Learn. Assess. 5(1), 5–21 (2006)

    Google Scholar 

  3. Williamson, D.M.: A framework for implementing automated scoring. In: Annual Meeting of the American Educational Research Association and the National Council on Measurement in Education, San Diego, CA (2009)

    Google Scholar 

  4. Chen, H., He, B., Luo, T., Li, B.: A ranked-based learning approach to automated essay scoring. In: 2012 Second International Conference on Cloud and Green Computing (CGC), pp. 448–455 (2012)

    Google Scholar 

  5. Yongwei, Y., Buckendahl, C.W., Juszkiewicz, P.J., et al.: A review of strategies for validating computer-automated scoring. Appl. Measur. Educ. 15(4), 391–412 (2002)

    Article  Google Scholar 

  6. Dumais, S.T.: Latent semantic analysis. Annu. Rev. Inf. Sci. Technol. 38(1), 188–230 (2004)

    Article  Google Scholar 

  7. Foltz, P.W., Laham, D., Landauer, T.K.: Automated essay scoring: applications to educational technology. In: World Conference on Educational Multimedia, Hypermedia and Telecommunications, vol. 1999(1), pp. 939–944 (1999)

    Google Scholar 

  8. Mesnil, G., Mikolov, T., Ranzato, M., Bengio, Y.: Ensemble of generative and discriminative techniques for sentiment analysis of movie reviews. CoRR, abs/1412.5335 (2006)

    Google Scholar 

  9. Socher, R., Lin, C.C., Ng, A.Y., Manning, C.: Parsing natural scenes and natural language with recursive neural networks. In: ICML, pp. 129–136 (2011)

    Google Scholar 

  10. Zanzotto, F.M., Korkontzelos, I., Fallucchi, F., Manandhar, S.: Estimating linear models for compositional distributional semantics. In: COLING, pp. 1263–1271 (2010)

    Google Scholar 

  11. Wang, S., Manning, C.D.: Baselines and bigrams: simple, good sentiment and topic classification. In: ACL (2), pp. 90–94 (2012)

    Google Scholar 

  12. Lai, S., Xu, L., Liu, K., Zhao, J.: Recurrent convolutional neural networks for text classification. In: AAAI, pp. 2267–2273 (2015)

    Google Scholar 

  13. Le, Q.V., Mikolov, T.: Distributed representations of sentences and documents. ICML, vol. 32, pp. 1188–1196 (2014)

    Google Scholar 

  14. Chen, H., He, B.: Automated essay scoring by maximizing human-machine agreement. In: EMNLP, pp. 1741–1752 (2013)

    Google Scholar 

  15. Shermis, M.D., Burstein, J.C.: Automated Essay Scoring: A Cross-Disciplinary Perspective. Routledge, Abingdon (2003)

    Google Scholar 

  16. Briscoe, T., Medlock, B., Andersen, Ø.: Automated assessment of ESOL free text examinations. University of Cambridge Computer Laboratory Technical reports, vol. 790 (2010)

    Google Scholar 

  17. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)

    MATH  Google Scholar 

  18. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space In: Proceedings of Workshop at ICLR (2013)

    Google Scholar 

  19. Shermis, M.D., Burstein, J.: Handbook of Automated Essay Evaluation: Current Applications and New Directions. Routledge, Abingdon (2013)

    Google Scholar 

  20. Yannakoudakis, H., Briscoe, T., Medlock, B.: A new dataset and method for automatically grading ESOL texts. In: ACL, pp. 180–189 (2011)

    Google Scholar 

  21. Lawrence, I., Lin, K.: A concordance correlation coefficient to evaluate reproducibility. Biometrics 45, 255–268 (1989)

    Article  MATH  Google Scholar 

  22. Croux, C., Dehon, C.: Influence functions of the Spearman and Kendall correlation measures. Stat. Methods Appl. 19(4), 497–515 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  23. Hyndman, R.J., Koehler, A.B.: Another look at measures of forecast accuracy. Int. J. Forecast. 22(4), 679–688 (2006)

    Article  Google Scholar 

  24. Altman, N.S.: An introduction to kernel and nearest-neighbor nonparametric regression. Am. Stat. 46(3), 175–185 (1992)

    MathSciNet  Google Scholar 

Download references

Acknowledgments

This work is supported by the National Natural Science Foundation of China (61472391).

Author information

Authors and Affiliations

  1. School of Computer and Control Engineering, University of Chinese Academy of Sciences, Geese-resting Lake Campus, Beijing, 101408, China

    Cancan Jin, Ben He & Jungang Xu

Authors
  1. Cancan Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ben He
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jungang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Ben He or Jungang Xu .

Editor information

Editors and Affiliations

  1. Deakin University, Burwood, Victoria, Australia

    Gang Li

  2. University of Arizona, Tucson, Arizona, USA

    Yong Ge

  3. Southwest University, Chongqing, China

    Zili Zhang

  4. Peking University, Beijing, China

    Zhi Jin

  5. University of Technology Sydney, Sydney, New South Wales, Australia

    Michael Blumenstein

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Jin, C., He, B., Xu, J. (2017). A Study of Distributed Semantic Representations for Automated Essay Scoring. In: Li, G., Ge, Y., Zhang, Z., Jin, Z., Blumenstein, M. (eds) Knowledge Science, Engineering and Management. KSEM 2017. Lecture Notes in Computer Science(), vol 10412. Springer, Cham. https://doi.org/10.1007/978-3-319-63558-3_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-63558-3_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-63557-6

  • Online ISBN: 978-3-319-63558-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation