Advertisement

Structuring Documentation to Support State Search: A Laboratory Experiment about Protocol Programming

  • Joshua Sunshine
  • James D. Herbsleb
  • Jonathan Aldrich
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8586)

Abstract

Application Programming Interfaces (APIs) often define object protocols. Objects with protocols have a finite number of states and in each state a different set of method calls is valid. Many researchers have developed protocol verification tools because protocols are notoriously difficult to follow correctly. However, recent research suggests that a major challenge for API protocol programmers is effectively searching the state space. Verification is an ineffective guide for this kind of search. In this paper we instead propose Plaiddoc, which is like Javadoc except it organizes methods by state instead of by class and it includes explicit state transitions, state-based type specifications, and rich state relationships. We compare Plaiddoc to a Javadoc control in a between-subjects laboratory experiment. We find that Plaiddoc participants complete state search tasks in significantly less time and with significantly fewer errors than Javadoc participants.

Keywords

Completion Time State Search Task Completion Time Documentation Format Method Summary 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Beckman, N.E., Kim, D., Aldrich, J.: An empirical study of object protocols in the wild. In: Mezini, M. (ed.) ECOOP 2011. LNCS, vol. 6813, pp. 2–26. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  2. 2.
    Beckman, N.E., Nori, A.V.: Probabilistic, modular and scalable inference of typestate specifications. In: Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2011, pp. 211–221. ACM, New York (2011)Google Scholar
  3. 3.
    Bierhoff, K., Beckman, N.E., Aldrich, J.: Practical API protocol checking with access permissions. In: Drossopoulou, S. (ed.) ECOOP 2009. LNCS, vol. 5653, pp. 195–219. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  4. 4.
    Bortolozzo, M., Centenaro, M., Focardi, R., Steel, G.: Attacking and fixing PKCS#11 security tokens. In: Proceedings of the 17th ACM Conference on Computer and Communications Security, CCS 2010, pp. 260–269. ACM, New York (2010)Google Scholar
  5. 5.
    Chandler, P., Sweller, J.: Cognitive load theory and the format of instruction. Cognition and Instruction 8(4), 293–332 (1991)CrossRefGoogle Scholar
  6. 6.
    Chi, M.T., Bassok, M., Lewis, M.W., Reimann, P., Glaser, R.: Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science 13(2), 145–182 (1989)CrossRefGoogle Scholar
  7. 7.
    Chong, S., Liu, J., Myers, A.C., Qi, X., Vikram, K., Zheng, L., Zheng, X.: Secure web applications via automatic partitioning. In: Proceedings of Twenty-first ACM SIGOPS Symposium on Operating Systems Principles, SOSP 2007, pp. 31–44. ACM, New York (2007)CrossRefGoogle Scholar
  8. 8.
    de Caso, G., Braberman, V., Garbervetsky, D., Uchitel, S.: Program abstractions for behaviour validation. In: Proceedings of the 33rd International Conference on Software Engineering, ICSE 2011, pp. 381–390. ACM, New York (2011)Google Scholar
  9. 9.
    Dekel, U., Herbsleb, J.D.: Improving API documentation usability with knowledge pushing. In: Proceedings of the 31st International Conference on Software Engineering, ICSE 2009, pp. 320–330 (2009)Google Scholar
  10. 10.
    Dwyer, M.B., Kinneer, A., Elbaum, S.: Adaptive online program analysis. In: Proceedings of the 29th international conference on Software Engineering, ICSE 2007, pp. 220–229. IEEE Computer Society, Washington, DC (2007)Google Scholar
  11. 11.
    Ellis, B., Stylos, J., Myers, B.: The factory pattern in API design: A usability evaluation. In: Proceedings of the 29th international conference on Software Engineering, ICSE 2007, pp. 302–312 (2007)Google Scholar
  12. 12.
    Fetterman, D.M.: Ibsen’s baths: Reactivity and insensitivity (a misapplication of the treatment-control design in a national evaluation). Educational Evaluation and Policy Analysis 4(3), 261–279 (1982)CrossRefGoogle Scholar
  13. 13.
    Foster, J.S., Terauchi, T., Aiken, A.: Flow-sensitive type qualifiers. In: Proceedings of the ACM SIGPLAN 2002 Conference on Programming language design and implementation, PLDI 2002, pp. 1–12. ACM, New York (2002)CrossRefGoogle Scholar
  14. 14.
    Georgiev, M., Iyengar, S., Jana, S., Anubhai, R., Boneh, D., Shmatikov, V.: The most dangerous code in the world: Validating SSL certificates in non-browser software. In: Proceedings of the 2012 ACM Conference on Computer and Communications Security, CCS 2012, pp. 38–49. ACM, New York (2012)CrossRefGoogle Scholar
  15. 15.
    Green, T.R.G., Petre, M.: Usability analysis of visual programming environments: a ‘cognitive dimensions’ framework. Journal of Visual Languages & Computing 7(2), 131–174 (1996)CrossRefGoogle Scholar
  16. 16.
    Hanenberg, S.: An experiment about static and dynamic type systems: Doubts about the positive impact of static type systems on development time. In: Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications, OOPSLA 2010, pp. 22–35. ACM, New York (2010)Google Scholar
  17. 17.
    Hanenberg, S., Kleinschmager, S., Robbes, R., Tanter, É., Stefik, A.: An empirical study on the impact of static typing on software maintainability. Empirical Software Engineering, 1–48 (2013)Google Scholar
  18. 18.
    Holmes, R., Walker, R.J., Murphy, G.C.: Strathcona example recommendation tool. In: Proceedings of the 10th European Software Engineering Conference Held Jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering, ESEC/FSE-13, pp. 237–240. ACM, New York (2005)Google Scholar
  19. 19.
    Jaspan, C.N.: Proper Plugin Protocols. PhD thesis, Carnegie Mellon University. Technical Report: CMU-ISR-11-116 (December 2011)Google Scholar
  20. 20.
    John, B.E., Kieras, D.E.: The GOMS family of user interface analysis techniques: Comparison and contrast. ACM Trans. Comput.-Hum. Interact. 3(4), 320–351 (1996)CrossRefGoogle Scholar
  21. 21.
    Larkin, J.H., Simon, H.A.: Why a diagram is (sometimes) worth ten thousand words. Cognitive Science 11(1), 65–100 (1987)CrossRefGoogle Scholar
  22. 22.
    Mayer, C., Hanenberg, S., Robbes, R., Tanter, É., Stefik, A.: An empirical study of the influence of static type systems on the usability of undocumented software. In: Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications, pp. 683–702. ACM (2012)Google Scholar
  23. 23.
    Neal, L.R.: A system for example-based programming. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 1989, pp. 63–68. ACM, New York (1989)Google Scholar
  24. 24.
    Robillard, M.P., Bodden, E., Kawrykow, D., Mezini, M., Ratchford, T.: Automated api property inference techniques. IEEE Transactions on Software Engineering 39(5), 613–637 (2013)CrossRefGoogle Scholar
  25. 25.
    Rosenthal, R., Rosnow, R.L.: Essential of Behavioiural Research: Methods and Data Analysis, 3rd edn. McGraw-Hill Higher Education, New York (2008)Google Scholar
  26. 26.
    Sabelfeld, A., Myers, A.: Language-based information-flow security. IEEE Journal on Selected Areas in Communications 21(1), 5–19 (2003)CrossRefGoogle Scholar
  27. 27.
    Shadish, W.R., Cook, T.D., Campbell, D.T.: Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Wadsworth Cengage Learning (2002)Google Scholar
  28. 28.
    Somorovsky, J., Mayer, A., Schwenk, J., Kampmann, M., Jensen, M.: On breaking SAML: Be whoever you want to be. In: Proceedings of the 21st USENIX Conference on Security Symposium, Security, vol. 12, p. 21 (2012)Google Scholar
  29. 29.
    Stuchlik, A., Hanenberg, S.: Static vs. dynamic type systems: An empirical study about the relationship between type casts and development time. In: Proceedings of the 7th Symposium on Dynamic Languages, DLS 2011, pp. 97–106. ACM, New York (2011)Google Scholar
  30. 30.
    Stylos, J., Myers, B.A.: The implications of method placement on API learnability. In: Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of Software Engineering, SIGSOFT 2008/FSE-16, pp. 105–112. ACM, New York (2008)CrossRefGoogle Scholar
  31. 31.
    Sunshine, J.: Protocol Programmability. PhD thesis, Carnegie Mellon University (December 2013)Google Scholar
  32. 32.
    Sunshine, J., Naden, K., Stork, S., Aldrich, J., Tanter, E.: First-class state change in plaid. In: Proceedings of the 2011 ACM International Conference on Object Oriented Programming Systems Languages and Applications, OOPSLA 2011, pp. 713–732. ACM, New York (2011)CrossRefGoogle Scholar
  33. 33.
    Sutherland, D.F., Scherlis, W.L.: Composable thread coloring. In: Proceedings of the 15th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP 2010, pp. 233–244. ACM, New York (2010)Google Scholar
  34. 34.
    Ullmer, B., Ishii, H.: Emerging frameworks for tangible user interfaces. IBM Systems Journal 39(3.4), 915–931 (2000)CrossRefGoogle Scholar
  35. 35.
    Walther, B.J., Ross, A.S.: The effect on behavior of being in a control group. Basic and Applied Social Psychology 3(4), 259–266 (1982)CrossRefGoogle Scholar
  36. 36.
    Whaley, J., Martin, M.C., Lam, M.S.: Automatic extraction of object-oriented component interfaces. In: Proceedings of the 2002 ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2002, pp. 218–228. ACM, New York (2002)Google Scholar
  37. 37.
    Ye, Y., Fischer, G., Reeves, B.: Integrating active information delivery and reuse repository systems. In: Proceedings of the 8th ACM SIGSOFT International Symposium on Foundations of Software Engineering: Twenty-first Century Applications, SIGSOFT 2000/FSE-8, pp. 60–68. ACM, New York (2000)Google Scholar
  38. 38.
    Zhong, H., Xie, T., Zhang, L., Pei, J., Mei, H.: MAPO: Mining and recommending API usage patterns. In: Drossopoulou, S. (ed.) ECOOP 2009. LNCS, vol. 5653, pp. 318–343. Springer, Heidelberg (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Joshua Sunshine
    • 1
  • James D. Herbsleb
    • 1
  • Jonathan Aldrich
    • 1
  1. 1.Institute for Software Research, School of Computer ScienceCarnegie Mellon UniversityUSA

Personalised recommendations