An Observational Study of How Experienced Programmers Annotate Program Code

  • Craig J. SutherlandEmail author
  • Andrew Luxton-Reilly
  • Beryl Plimmer
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9297)


This study investigates how and why experienced programmers annotate program code. Research has shown that marking up prose with a pen is an invaluable aid to reading for understanding. However program code is very different from prose: there are no studies on how programmers annotate code while reading. We asked experienced programmers to read code printed on paper and observed their annotation practices. We found the main reasons for annotating code are to assist with navigation and to record information for later use. Furthermore, we found annotation practices that are hard to replicate in current standard Integrated Development Environments. This suggests that support for digital ink annotations in programming tools may be useful for comprehending program code.


Freeform annotation Reading code Understanding code Observational study 



We would like to thank all the participants in our study for time they gave us.


  1. 1.
    Sillito, J., De Voider, K., Fisher, B., Murphy, G.: Managing software change tasks: an exploratory study. In: International Symposium on Empirical Software Engineering 2005, pp. 23–32 (2005)Google Scholar
  2. 2.
    Singer, J., Lethbridge, T., Vinson, N., Anquetil, N.: An examination of software engineering work practices. In: CASCON First Decade High Impact Papers, pp. 174–188. IBM Corporation, Toronto (2010)Google Scholar
  3. 3.
    Sellen, A.J., Harper, R.H.R.: The Myth of the Paperless Office. MIT Press, Cambridge (2002)Google Scholar
  4. 4.
    Maalej, W., Tiarks, R., Roehm, T., Koschke, R.: On the comprehension of program comprehension. ACM Trans. Softw. Eng. Methodol. 23, 1–37 (2014)CrossRefGoogle Scholar
  5. 5.
    Fowler, R.L., Barker, A.S.: Effectiveness of highlighting for retention of text material. J. Appl. Psychol. 59, 358 (1974)CrossRefGoogle Scholar
  6. 6.
    Hynd, C.R., Simpson, M.L., Chase, N.D.: Studying narrative text: The effects of annotating vs. journal writing on test performance. Reading Res. Instruction 29, 44–54 (1989)CrossRefGoogle Scholar
  7. 7.
    Simpson, M.L., Nist, S.L.: Textbook annotation: an effective and efficient study strategy for college students. J. Reading 34, 122–129 (1990)Google Scholar
  8. 8.
    Wolfe, J.L., Neuwirth, C.M.: From the margins to the center: the future of annotation. J. Bus. Technical Commun. 15, 333–371 (2001)CrossRefGoogle Scholar
  9. 9.
    Ball, E., Franks, H., Jenkins, J., McGrath, M., Leigh, J.: Annotation is a valuable tool to enhance learning and assessment in student essays. Nurse Educ. Today 29, 284–291 (2009)CrossRefGoogle Scholar
  10. 10.
    O’Hara, K., Sellen, A.: A comparison of reading paper and on-line documents. In: Proceedings of CHI 1997, pp. 335–342. ACM, New York (1997)Google Scholar
  11. 11.
    Morris, M.R., Brush, A.B., Meyers, B.R.: Reading revisited: evaluating the usability of digital display surfaces for active reading tasks. In: Proceedings of TABLETOP 2007, pp. 79–86. IEEE (2007)Google Scholar
  12. 12.
    Adler, A., Gujar, A., Harrison, B.L., O’Hara, K., Sellen, A.: A diary study of work-related reading: design implications for digital reading devices. In: Proceedings of CHI 1998, pp. 241–248. ACM, New York (1998)Google Scholar
  13. 13.
    Hong, M., Piper, A.M., Weibel, N., Olberding, S., Hollan, J.: Microanalysis of active reading behavior to inform design of interactive desktop workspaces. In: TABLETOP 2012, pp. 215–224. ACM, New York (2012)Google Scholar
  14. 14.
    Jackel, B.: Item differential in computer based and paper based versions of a high stakes tertiary entrance test: diagrams and the problem of annotation. In: Dwyer, T., Purchase, H., Delaney, A. (eds.) Diagrams 2014. LNCS, vol. 8578, pp. 71–77. Springer, Heidelberg (2014)Google Scholar
  15. 15.
    Johnson, M., Nadas, R.: Marginalised behaviour: digital annotations, spatial encoding and the implications for reading comprehension. Learn. Media Technol. 34, 323–336 (2009)CrossRefGoogle Scholar
  16. 16.
    Glover, I., Xu, Z., Hardaker, G.: Online annotation – Research and practices. Comput. Educ. 49, 1308–1320 (2007)CrossRefGoogle Scholar
  17. 17.
    Crisp, V., Johnson, M.: The use of annotations in examination marking: opening a window into markers’ minds. Br. Educ. Res. J. 33, 943–961 (2007)CrossRefGoogle Scholar
  18. 18.
    Priest, R., Plimmer, B.: RCA: experiences with an IDE annotation tool. In: Proceedings of CHINZ 2006, pp. 53–60. ACM, New York (2006)Google Scholar
  19. 19.
    Bragdon, A., Reiss, S.P., Zeleznik, R., Karumuri, S., Cheung, W., Kaplan, J., Coleman, C., Adeputra, F., LaViola Jr., J.J.: Code bubbles: rethinking the user interface paradigm of integrated development environments. In: Proceedings of ACM/IEEE International Conference on Software Engineering, pp. 455–464. ACM, New York (2010)Google Scholar
  20. 20.
    DeLine, R., Bragdon, A., Rowan, K., Jacobsen, J., Reiss, S.P.: Debugger canvas: industrial experience with the code bubbles paradigm. In: Proceedings of the 2012 International Conference on Software Engineering, pp. 1064–1073. IEEE Press (2012)Google Scholar
  21. 21.
    Sutherland, C.J., Plimmer, B.: VsInk: integrating digital ink with program code in visual studio. In: Proceedings of AUIC 2013, pp. 13–22. Australian Computer Society, Incorporation (2013)Google Scholar
  22. 22.
    Lichtschlag, L., Spychalski, L., Bochers, J.: CodeGraffiti: using hand-drawn sketches connected to code bases in navigation tasks. In: Proceedings of VL/HCC 2014, pp. 65–68. IEEE (2014)Google Scholar
  23. 23.
    Chen, X., Plimmer, B.: CodeAnnotator: digital ink annotation within Eclipse. In: Proceedings of AusCHI 2007, pp. 211–214. ACM, New York (2007)Google Scholar
  24. 24.
    Lichtschlag, L., Borchers, J.: CodeGraffiti: communication by sketching for pair programmers. In: Adjunct Proceedings of ACM Symposium on User Interface Software and Technology, pp. 439–440. ACM, New Ork (2010)Google Scholar
  25. 25.
    Plimmer, B.: A comparative evaluation of annotation software for grading programming assignments. In: Proceedings of AUIC 2010, pp. 14–22. Australian Computer Society, Incorporation (2010)Google Scholar
  26. 26.
    Marshall, C.C.: Annotation: from paper books to the digital library. In: Proceedings of ACM International Conference on Digital Libraries, pp. 131–140. ACM, New York (1997)Google Scholar
  27. 27.
    Marshall, C.C., Brush, A.J.B.: Exploring the relationship between personal and public annotations. In: Proceedings of Digital Libraries, 2004, pp. 349–357. ACM Press, New York (2004)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2015

Authors and Affiliations

  • Craig J. Sutherland
    • 1
    Email author
  • Andrew Luxton-Reilly
    • 1
  • Beryl Plimmer
    • 1
  1. 1.University of AucklandAucklandNew Zealand

Personalised recommendations