Empirical Software Engineering

, Volume 16, Issue 6, pp 703–732 | Cite as

A field study of API learning obstacles

  • Martin P. Robillard
  • Robert DeLine


Large APIs can be hard to learn, and this can lead to decreased programmer productivity. But what makes APIs hard to learn? We conducted a mixed approach, multi-phased study of the obstacles faced by Microsoft developers learning a wide variety of new APIs. The study involved a combination of surveys and in-person interviews, and collected the opinions and experiences of over 440 professional developers. We found that some of the most severe obstacles faced by developers learning new APIs pertained to the documentation and other learning resources. We report on the obstacles developers face when learning new APIs, with a special focus on obstacles related to API documentation. Our qualitative analysis elicited five important factors to consider when designing API documentation: documentation of intent; code examples; matching APIs with scenarios; penetrability of the API; and format and presentation. We analyzed how these factors can be interpreted to prioritize API documentation development efforts


Application programming interfaces Software libraries Programming Documentation 



We thank the anonymous participants in our study for generously contributing their time to this project, and the members of the Human Interactions in Programming group at Microsoft Research for feedback and discussions on the work reported here. Additional thanks to E. Duala-Ekoko, U. Farooq, G. Murphy, A. Ying, T. Ratchford, and the anonymous reviewers for feedback on the paper. We are also grateful to Jose Correa of the McGill University Statistical Consulting Service for his assistance with a statistical test.


  1. Beaton J, Jeong SY, Xie Y, Stylos J, Myers BA (2008) Usability challenges for enterprise service-oriented architecture APIs. In: Proc. IEEE symp. visual languages and human-centric computing, pp 193–196Google Scholar
  2. Berglund E (2003) Designing electronic reference documentation for software component libraries. J Syst Softw 68(1):65–75MathSciNetCrossRefGoogle Scholar
  3. Bloch J (2006) How to design a good API and why it matters. In: Companion to the 21st ACM SIGPLAN symposium on object-oriented programming systems, languages, and applications, pp 505–506Google Scholar
  4. Bore C, Bore S (2005) Profiling software API usability for consumer electronics. In: Digest of int’l conf. on consumer electronics, pp 155–156Google Scholar
  5. Brandt J, Guo PJ, Lewenstein J, Dontcheva M, Klemmer SR (2009) Two studies of opportunistic programming: interleaving web foraging, learning, and writing code. In: Proc. 27th int’l conf. on human factors in computing systems, pp 1589–1598Google Scholar
  6. Clarke S (2004) Measuring API usability. Dr Dobb’s Journal Special Windows/NET SupplementGoogle Scholar
  7. Corbin J, Strauss A (2007) Basics of qualitative research: techniques and procedures for developing grounded theory. Sage PublicationsGoogle Scholar
  8. Creswell JW (2007) Qualitative inquiry and research design: choosing among five approaches. Sage PublicationsGoogle Scholar
  9. Cwalina K, Abrams B (2009) Framework design guidelines: conventions, idioms, and patterns for reusable .NET Libraries, 2nd edn. Addison-WesleyGoogle Scholar
  10. des Rivières J (2004) Eclipse APIs: lines in the sand. EclipseCon Technical Talk.
  11. Ellis B, Stylos J, Myers B (2007) The factory pattern in API design: a usability evaluation. In: Proc. 29th int’l conf. on software engineering, pp 302–312. doi: 10.1109/ICSE.2007.85
  12. Feilkas M, Ratiu D (2008) Ensuring well-behaved usage of APIs through syntactic constraints. In: Proc. 16th int’l conf. on program comprehension, pp 248–253Google Scholar
  13. Fowler M (2002) Public versus published interfaces. IEEE Softw 19(2):18–19CrossRefGoogle Scholar
  14. Holmes R, Walker RJ, Murphy GC (2006) Approximate structural context matching: an approach to recommend relevant examples. IEEE Trans Softw Eng 32(12):952–970CrossRefGoogle Scholar
  15. Hou D (2008) Investigating the effects of framework design knowledge in example-based framework learning. In: Proc. 24th int’l conf. on software maintenance, pp 37–46Google Scholar
  16. Hou D, Wong K, Hoover JH (2005) What can programmer questions tell us about frameworks? In: Proc. 13th int’l workshop on program comprehension, pp 87–96Google Scholar
  17. Jeong SY, Xie Y, Beaton J, Myers BA, Stylos J, Ehret R, Karstens J, Efeoglu A, Busse DK (2009) Improving documentation for eSOA APIs through user studies. In: Proc. 2nd int’l symp. on end-user development. LNCS, vol 5435. Springer, pp 86–105Google Scholar
  18. Knowles MS, Holton III EF, Swanson RA (2005) The adult learner, 6th edn. Butterworth-HeinemannGoogle Scholar
  19. Ko AJ, Myers BA, Aung HH (2004) Six learning barriers in end-user programming systems. In: Proc. symp. on visual languages and human centric computing, pp 199–206Google Scholar
  20. Ko AJ, DeLine R, Venolia G (2007) Information needs in collocated software development teams. In: Proc. 29th int’l conf. on software engineering, pp 344–353Google Scholar
  21. Larman C (2001) Protected variation: the importance of being closed. IEEE Softw 18(3):89–91CrossRefGoogle Scholar
  22. Lutters WG, Seaman CB (2007) Revealing actual documentation usage in software maintenance through war stories. Inf Softw Technol 49:576–587CrossRefGoogle Scholar
  23. McLellan SG, Roesler AW, Tempest JT, Spinuzzi CI (1998) Building more usable APIs. IEEE Softw 15(3):78–86CrossRefGoogle Scholar
  24. Nykaza J, Messinger R, Boehme F, Norman CL, Mace M, Gordon M (2002) What programmers really want: results of a needs assessment for SDK documentation. In: Proc. 20th annual ACM SIGDOC int’l conf. on computer documentation, pp 133–141Google Scholar
  25. Olson M, Hergenhahn B (2008) Introduction to the theories of learning, 8th edn. Prentice HallGoogle Scholar
  26. Parnas DL (1972) On the criteria to be used in decomposing systems into modules. Commun ACM 15(12):1053–1058CrossRefGoogle Scholar
  27. Robillard MP (2009) What makes APIs hard to learn? The answers of developers. IEEE Softw (November/December):27–34Google Scholar
  28. Rosson MB, Carroll JM (1996) The reuse of uses in Smalltalk programming. ACM Trans Comput-Hum Interact 3(3):219–253. doi: 10.1145/234526.234530 CrossRefGoogle Scholar
  29. Seaman CB (2002) The information gathering strategies of software maintainers. In: Proc. int’l conf. on software maintenance, pp 141–149Google Scholar
  30. Shull F, Lanubile F, Basili VR (2000) Investigating reading techniques for object-oriented framework learning. IEEE Trans Softw Eng 26(11):1101–1118CrossRefGoogle Scholar
  31. Sillito J, Murphy GC, Volder KD (2008) Asking and answering questions during a programming change task. IEEE Trans Softw Eng 34(4):434–451CrossRefGoogle Scholar
  32. Stylos J, Clarke S (2007) Usability implications of requiring parameters in objects’ constructors. In: Proc. 29th int’l conf. on software engineering, pp 529–539Google Scholar
  33. Stylos J, Myers BA (2007) Mapping the space of API design decisions. In: Proc. symp. on visual languages and human-centric computing, pp 50–60Google Scholar
  34. Stylos J, Myers BA (2008) Implications of method placement on api learnability. In: Proc. 16th ACM SIGSOFT int’l symp. on the foundations of software engineering, pp 105–112Google Scholar
  35. Stylos J, Graf B, Busse DK, Ziegler C, Karstens REJ (2008) A case study of API redesign for improved usability. In: Proc. symp. on visual languages and human-centric computing, pp 189–192Google Scholar
  36. Tulach J (2008) Practical API Design: confessions of a Java framework architect. APressGoogle Scholar
  37. Weiss RS (1994) Learning from strangers: the art and method of qualitative interview studies. The Free PressGoogle Scholar
  38. Westfall PH, Tobias RD, Rom D, Wolfinger RD, Hochberg Y (1999) Multiple comparisons and multiple tests using the SAS system. SAS Institute Inc., Cary, NCGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.School of Computer ScienceMcGill UniversityMontréalCanada
  2. 2.Microsoft ResearchOne Microsoft WayRedmondUSA

Personalised recommendations