Scale-Driven Automatic Hint Generation for Coding Style

  • Rohan Roy Choudhury
  • Hezheng Yin
  • Armando Fox
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9684)


While the use of autograders for code correctness is widespread, less effort has focused on automating feedback for good programming style: the tasteful use of language features and idioms to produce code that is not only correct, but also concise, elegant, and revealing of design intent. We present a system that can provide real-time actionable code style feedback to students in large introductory computer science classes. We demonstrate that in a randomized controlled trial, 70 % of students using our system achieved the best style solution to a coding problem in less than an hour, while only 13 % of students in the control group achieved the same. Students using our system also showed a statistically-significant greater improvement in code style than students in the control group.


Coding style Autograding Automatic hint generation MOOCs 


  1. 1.
    Ericsson, K., Krampe, R., Tesch-Römer, C.: The role of deliberate practice in the acquisition of expert performance. Psychol. Rev. 100(3), 363–406 (1993)CrossRefGoogle Scholar
  2. 2.
    Fast, E., Steffee, D., Wang, L., Brandt, J., Bernstein, M.: Emergent, crowd-scale programming practice in the IDE. In: SIGCHI Conference on Human Factors in Computing Systems. Toronto (2014)Google Scholar
  3. 3.
    Fitzpatrick, J.: Applying the ABC metric to C, C++, and Java. In: More C++ Gems, pp. 245–264. Cambridge University Press, New York (2000)Google Scholar
  4. 4.
    Fowler, M., Beck, K., Brant, J., Opdyke, W., Roberts, D.: Refactoring: Improving the Design of Existing Code. Addison-Wesley, Boston (1999)Google Scholar
  5. 5.
    Fox, A., Patterson, D.: Engineering Software as a Service. Strawberry Canyon LLC, San Francisco (2014)Google Scholar
  6. 6.
    Fox, A., Patterson, D., Joseph, S., McCulloch, P.: MAGIC: Massive automated grading in the cloud. In: CHANGEE (Facing the challenges of assessing 21st century skills in the newly emerging educational ecosystem) workshop at EC-TEL 2015, Toledo, Spain (2015)Google Scholar
  7. 7.
    Gamma, E., Helm, R., Johnson, R., Vlissides, J.: Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley, Boston (1994)MATHGoogle Scholar
  8. 8.
    Glassman, E., Singh, R., Miller, R.: Feature engineering for clustering student solutions. In: 1st ACM Conference on Learning at Scale, Atlanta (2014)Google Scholar
  9. 9.
    Guskey, T.R.: Closing achievement gaps: revisiting Benjamin S. Blooms, “Learning for Mastery”. J. Adv. Acad. 19(1), 8–31 (2007)Google Scholar
  10. 10.
    Huang, J., Piech, C., Nguyen, A., Guibas, L.: Syntactic and functional variability of a million code submissions in a machine learning MOOC. In: International Conference on Artificial Intelligence in Education (AIED), Memphis (2013)Google Scholar
  11. 11.
    Johnson, S.: Lint, a C program checker. Technical report 65, Bell Labs (1977)Google Scholar
  12. 12.
    Kan, S.H.: Metrics and Models in Software Quality Engineering, 2nd edn. Addison-Wesley, Boston (2002)MATHGoogle Scholar
  13. 13.
    Kulkarni, C.E., Bernstein, M.S., Klemmer, S.R.: PeerStudio: rapid peer feedback emphasizes revision and improves performance. In: 2nd ACM Conference on Learning at Scale. Vancouver (2015)Google Scholar
  14. 14.
    Lazar, T., Bratko, I.: Data-driven program synthesis for hint generation in programming tutors. In: Trausan-Matu, S., Boyer, K.E., Crosby, M., Panourgia, K. (eds.) ITS 2014. LNCS, vol. 8474, pp. 306–311. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  15. 15.
    Van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(2579–2605), 85 (2008)MATHGoogle Scholar
  16. 16.
    Martin, R.C.: Clean Code: A Handbook of Agile Software Craftsmanship. Prentice Hall, Upper Saddle River (2008)Google Scholar
  17. 17.
    Moghadam, J., Roy Choudhury, R., Yin, H., Fox, A.: AutoStyle: toward coding style feedback at scale. In: 2nd ACM Conference on Learning at Scale, Vancouver (2015)Google Scholar
  18. 18.
    Nguyen, A., Piech, C., Huang, J., Guibas, L.: Codewebs: scalable code search for MOOCs. In: 23rd International Conference on world wide web, Seoul (2014)Google Scholar
  19. 19.
    Rivers, K., Koedinger, K.R.: Data-driven hint generation in vast solution spaces: a self-improving python programming tutor. Int. J. Artif. Intell. Educ., 1–28 (2015).
  20. 20.
    Shute, V.J.: Focus on formative feedback. Rev. Educ. Res. 78(1), 153–189 (2008)CrossRefGoogle Scholar
  21. 21.
    Yin, H., Moghadam, J., Fox, A.: Clustering student programming assignments to multiply instructor leverage. In: 2nd ACM Conference on Learning at Scale, Vancouver (2015)Google Scholar
  22. 22.
    Zhang, K., Shasha, D.: Simple fast algorithms for the editing distance between trees and related problems. SIAM J. Comput. 18(6), 1245–1262 (1989)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.University of CaliforniaBerkeleyUSA

Personalised recommendations