The Automated Grading of R Code Snippets: Preliminary Results in a Course of Health Informatics

  • Anna Maria Angelone
  • Pierpaolo VittoriniEmail author
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1007)


Automated grading tools either execute the submitted code against sample data or statically analyse it. The paper presents a tool for the automated grading of R code snippets, submitted by students learning Health Informatics in the degree course of Medicine and Surgery of the University of L’Aquila (Italy). The tool performs a static analysis of the R commands, with the respective output, as well as of the sentences written in natural language. The paper details the problem in general and through examples. Then, it describes the proposed solution and reports on the comparison between the automated grading and the human one. Finally, the paper ends by describing the foreseen use of the tool and the needed improvements.


Summative assessment Formative assessment Automated grading 


  1. 1.
    Bartko, J.J.: The intraclass correlation coefficient as a measure of reliability. Psychol. Rep. 19(1), 3–11 (1966)CrossRefGoogle Scholar
  2. 2.
    Bernardi, A., Innamorati, C., Padovani, C., Romanelli, R., Saggino, A., Tommasi, M., Vittorini, P.: On the design and development of an assessment system with adaptive capabilities. In: Methodologies and Intelligent Systems for Technology Enhanced Learning, pp. 190–199. Springer, Cham (2019)Google Scholar
  3. 3.
    Burrows, S., Gurevych, I., Stein, B.: The eras and trends of automatic short answer grading. Int. J. Artif. Intell. Educ. 25(1), 60–117 (2015)CrossRefGoogle Scholar
  4. 4.
    Choy, M., Lam, S., Poon, C.K., Wang, F.L., Yu, Y.T., Yuen, L.: Design and implementation of an automated system for assessment of computer programming assignments. In: Advances in Web Based Learning, ICWL 2007, pp. 584–596. Springer, Heidelberg (2007)Google Scholar
  5. 5.
    Cicchetti, D.V.: Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol. Assess. 6(4), 284–290 (1994)CrossRefGoogle Scholar
  6. 6.
    Cohen, J.: A coefficient of agreement for nominal scales. Educ. Psychol. Measur. 20(1), 37–46 (1960)CrossRefGoogle Scholar
  7. 7.
    Edwards, S.H., Perez-Quinones, M.A., Edwards, S.H., Perez-Quinones, M.A.: Web-CAT: automatically grading programming assignments. In: Proceedings of the 13th Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE 2008, vol. 40, p. 328. ACM Press, New York (2008)Google Scholar
  8. 8.
    Georgouli, K., Guerreiro, P.: Incorporating an automatic judge into blended learning programming activities. In: Advances in Web-Based Learning, ICWL 2010, pp. 81–90. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  9. 9.
    Gomaa, W.H., Fahmy, A.A.: A survey of text similarity approaches. Int. J. Comput. Appl. 68(13), 13–18 (2013)Google Scholar
  10. 10.
    Harlen, W., James, M.: Assessment and learning: differences and relationships between formative and summative assessment. Assess. Educ. Princ. Policy Pract. 4(3), 365–379 (1997)Google Scholar
  11. 11.
    Joy, M., Griffiths, N., Boyatt, R.: The boss online submission and assessment system. J. Educ. Resour. Comput. 5(3), 2–es (2005)CrossRefGoogle Scholar
  12. 12.
    Knight, P., Yorke, M.: Society for Research into Higher Education: Assessment Learning and Employability. Society for Research into Higher Education & Open University Press, Maidenhead (2003)Google Scholar
  13. 13.
    Levenshtein, V.I.: Binary codes capable of correcting deletions, insertions and reversals. In: Soviet Physics Doklady, vol. 10, p. 707 (1966)Google Scholar
  14. 14.
    R Core Team: R: A Language and Environment for Statistical Computing (2018)Google Scholar
  15. 15.
    Souza, D.M., Felizardo, K.R., Barbosa, E.F.: A systematic literature review of assessment tools for programming assignments. In: 2016 IEEE 29th International Conference on Software Engineering Education and Training (CSEET), pp. 147–156. IEEE, April 2016Google Scholar
  16. 16.
    Sultan, M.A., Salazar, C., Sumner, T.: Fast and easy short answer grading with high accuracy. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1070–1075. Association for Computational Linguistics, Stroudsburg (2016)Google Scholar
  17. 17.
    Weisberg, S.: Applied Linear Regression. Wiley, Hoboken (2013)zbMATHGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Department of Life, Health and Environmental SciencesUniversity of L’AquilaL’AquilaItaly

Personalised recommendations