Making End User Development More Natural



When end users approach a development task, they bring with them a set of techniques, expressions, and knowledge, which can be leveraged in order to make the process easier. The Natural Programming Project has been working for over twenty years to better understand how end users think about their tasks, and to develop new ways for users to express those tasks that will be more “natural,” by which we mean closer to the way they think. Our chapter in the previous book covered the first 10 years of this research; and here we summarize the most recent 10 years. This includes studies on barriers that impede EUD, and a new tool that helps with the understanding and debugging barriers by showing developers why their program has its current behavior. We also describe a tool that we created to help EUDs input, process, and transform data in the context of spreadsheets and web pages. Interaction designers are a class of EUDs that may need to program interactive behaviors, so we studied how they naturally express those behaviors, and then built a spreadsheet-like tool to allow them to author new behaviors. Another spreadsheet tool we created helps EUDs access web service data without writing code, and extends the familiar spreadsheet to support analyzing the acquired web-based hierarchical data and programming data-driven GUI applications. Finally, EUDs often need to engage in exploratory programming, where the goals and tasks are not well-formed in advance. We describe new tools to help users selectively undo past actions, along with on-going research to help EUDs create more efficient behaviors on smartphones and facilitate variations when performing data analysis.


Spreadsheets exploratory programming data analysis the Natural Programming Group 



This article grows out of over 20 years of work by the Natural Programming group by more than 50 students, staff and postdocs in addition to the authors, and we thank them all for their contributions. The work summarized here has been funded at least by SAP, Adobe, IBM, Microsoft, Yahoo! and multiple NSF grants including CNS-1423054, IIS-1314356, IIS-1116724, IIS-0329090, CCF-0811610, IIS-0757511, and CCR-0324770. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect those of any of the sponsors.


  1. Amershi, S., Cakmak, M., Knox, W. B., Kulesza, T. (2014). Power to the people: the role of humans in interactive machine learning. AI Magazine, 35(4), 105–120.CrossRefGoogle Scholar
  2. Berlage, T. (1994). A selective undo mechanism for graphical user interfaces based on command objects. ACM Transactions on Computer Human Interaction. ACM Transactions on Computer Human Interaction, 1(3), 269–294.CrossRefGoogle Scholar
  3. Buchen, L. (2009). Robot makes scientific discovery all by itself. Wired UK Online.
  4. Chang, K., & Myers, B.A. (2014a, October 5–8). Creating interactive web data applications with spreadsheets. In UIST’14: ACM Symposium on User Interface Software and Technology (pp. 87–96). Honolulu, Hawaii.Google Scholar
  5. Chang, K., & Myers, B.A. (2014b, July 28–August 1). A spreadsheet model for using web service data. In VL/HCC’14: IEEE Symposium on Visual Languages and Human-Centric Computing (pp. 169–176). Melbourne, Australia.Google Scholar
  6. Chang, K., & Myers, B.A. (2016, May 7–12). Using and exploring hierarchical data in spreadsheets. In Proceedings CHI’2016: Human Factors in Computing Systems (pp. 2497–2507). San Jose, CA.Google Scholar
  7. Clark, L. (2012). Google’s artificial brain learns to find cat videos. Wired UK Online.
  8. Cypher, A., Halbert, D. C., Kurlander, D., Lieberman, H., Maulsby, D., Myers, B. A., Turransky, A. (1993). Watch what I do: programming by demonstration. Cambridge, MA: MIT Press.Google Scholar
  9. Fisher, II, M., & Rothermel, G. (2004). The EUSES spreadsheet corpus: a shared re-source for supporting experimentation with spreadsheet dependability mechanisms. Lincoln: University of Nebraska. Technical Report 04-12-03.Google Scholar
  10. Green, T. R. G., & Petre, M. (1996). Usability analysis of visual programming environments: a ‘cognitive dimensions’ framework. Journal of Visual Languages and Computing, 7(2), 131–174.CrossRefGoogle Scholar
  11. Grigoreanu, V., Fernandez, R., Inkpen, K., Robertson, G. (2009, September 20–24). What designers want: needs of interactive application designers. In IEEE Symposium on Visual Languages and Human-Centric Computing, VL/HCC’09 (pp. 139–146). Corvallis, Oregon.Google Scholar
  12. Henley, A.Z., & Fleming, S.D. (2016, September 4–8). Yestercode: improving code-change support in visual dataflow programming environments. In: VL/HCC’16: IEEE Symposium on Visual Languages and Human-Centric Computing. Cambridge.Google Scholar
  13. Hill, C., Bellamy, R., Erickson, T., Burnett, M. (2016). Trials and tribula-tions of developers of intelligent systems: a field study. In VL/HCC’2016: IEEE Symposium on Visual Lan-guages and Human-Centric Computing (pp. 162–170). Denver, CO.Google Scholar
  14. Kery, M.B., Horvath, A., Myers, B.A. (2017, May 6–11). Variolite: supporting exploratory programming by data scientists. In Proceedings CHI’2017: Human Factors in Computing Systems (pp. 1265–1276). Denver, CO.Google Scholar
  15. Ko, A. J., Abraham, R., Beckwith, L., Blackwell, A., Burnett, M., Erwig, M., et al. (April, 2011). The state of the art in end-user software engineering. ACM Computing Surveys, 43(3), Article 21 44 pages.CrossRefGoogle Scholar
  16. Ko, A.J., & Myers, B.A. (2004, April 24–29). Designing the whyline, a debugging interface for asking why and why not questions about runtime failures. In Proceedings CHI’2004: Human Factors in Computing Systems (pp. 151–158). Vienna, Austria.Google Scholar
  17. Ko, A. J., & Myers, B. A. (2005, February). A framework and methodology for studying the causes of software errors in programming systems. Journal of Visual Languages and Computing, 16(1), 41–84.CrossRefGoogle Scholar
  18. Ko, A.J., & Myers, B.A. (2009, April 4–9). Finding causes of program output with the java whyline. In CHI’2009: Human Factors in Computing Systems (pp. 1569–1578). Boston, MA.Google Scholar
  19. Ko, A. J., & Myers, B. A. (2010, August). Extracting and answering why and why not questions about java program output. ACM Transactions on Software Engineering and Methodology (TOSEM), 20(2), Article 4 36 pages.CrossRefGoogle Scholar
  20. Ko, A.J., Myers, B.A., Aung, H.H. (2004, September 26–29). Six learning barriers in end-user programming systems. In IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) (pp. 199–206). Rome, Italy.Google Scholar
  21. Ko, A. J., Myers, B. A., Coblenz, M., Aung, H. H. (2006, December). An exploratory study of how developers seek, relate, and collect relevant information during software maintenance tasks. IEEE Transactions on Software Engineering, 33(12), 971–987.CrossRefGoogle Scholar
  22. Kuttal, S.K., Sarma, A., Rothermel, G. (2011). History repeats itself more easily when you log it: versioning for mashup. In IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) (pp. 69–72). Pittsburgh, PA.Google Scholar
  23. Li, T., Azaria, A., Myers, B. (2017, May 6–11). SUGILITE: creating multimodal smartphone automation by demonstration. In Proceedings CHI’2017: Human Factors in Computing Systems (pp. 6038–6049). Denver, CO.Google Scholar
  24. Lin, J., Wong, J., Nichols, J., Cypher, A., Lau, T. A. (2009). End-user programming of mashups with vegemite. Proceedings of the 14th International Conference on Intelligent User Interfaces (pp. 97–106). Sanibel Island, FL: ACM.Google Scholar
  25. Loksa, D., Ko, A. J., Jernigan, W., Oleson, A., Mendez, C. J., Burnett, M. M. (2016). Programming, problem solving, and self-awareness: effects of explicit guidance. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (pp. 1449–1461). Santa Clara, CA: ACM.CrossRefGoogle Scholar
  26. Meyerovich, L. A., Guha, A., Baskin, J., Cooper, G. H., Greenberg, M., Bromfield, A., et al. (2009). Flapjax: a programming language for Ajax applications. SIGPLAN Notices (Proc. OOPSLA’2009), 44(10), pp. 1–20. 1640091.CrossRefGoogle Scholar
  27. Muncy, J. (2016). Making AI play lots of videogames could be huge (No, Seriously). Wired UK Online.
  28. Myers, B.A. (1991, November). Separating application code from toolkits: eliminating the spaghetti of call-backs. In UIST’91: ACM SIGGRAPH Symposium on User Interface Software and Technology (pp. 211–220). Hilton Head, SC.Google Scholar
  29. Myers, B.A. (1998, April). Scripting graphical applications by demonstration. In SIGCHI’98: Human Factors in Computing Systems (pp. 534–541). Los Angeles, CA.Google Scholar
  30. Myers, B. A., Hudson, S. E., Pausch, R. (2000, March). Past, present and future of user interface software tools. ACM Transactions on Computer Human Interaction, 7(1), 3–28.CrossRefGoogle Scholar
  31. Myers, B. A., Ko, A. J., LaToza, T. D., Yoon, Y. S. (2016, July). Programmer are users too: human centered methods to improve software development. IEEE Computer, 49(7), 44–52.CrossRefGoogle Scholar
  32. Myers, B., McDaniel, R., Wolber, D. (2000, March). Programming by example: intelligence in demonstrational interfaces. Communications of the ACM, 43(3), pp. 82–89.CrossRefGoogle Scholar
  33. Myers, B.A., Park, S.Y., Nakano, Y., Mueller, G., Ko, A. (2008, September 15–18). How designers design and program interactive behaviors. In 2008 IEEE Symposium on Visual Languages and Human-Centric Computing, VL/HCC’08 (pp. 185–188). Herrsching am Ammersee, Germany.Google Scholar
  34. Norman, D. A. (1988). The design of everyday things. New York: Doubleday.Google Scholar
  35. Oney, S., Myers, B.A., Brandt, J. (2012, October 7–10). ConstraintJS: programming interactive behaviors for the web by integrating constraints and states. In UIST’2012: ACM Symposium on User Interface Software and Technology (pp. 229–238). Cambridge, MA.Google Scholar
  36. Oney, S., Myers, B.A., Brandt, J. (2014, October 5–8). InterState: a language and environment for expressing interface behavior. In ACM Symposium on User Interface Software and Technology, UIST’14 (pp. 263–272). Honolulu, Hawaii.Google Scholar
  37. O’Toole, J. (2014, February 28). Mobile apps overtake PC Internet usage in U.S. CNN Money.
  38. Ozenc, K., Kim, M., Zimmerman, J., Oney, S., Myers, B. (2010, April 10–15). How to support designers in getting hold of the immaterial material of software. In CHI’2010: Human Factors in Computing Systems (pp. 2513–2522). Atlanta, GA.Google Scholar
  39. Pane, J. F., & Myers, B. A. (2006). More natural programming languages and environments. H. Lieberman, F. Paterno, V. Wulf (Eds.). End-User development (pp. 31–50). Dordrecht: Springer.CrossRefGoogle Scholar
  40. Park, S., Myers, B., Ko, A. (2008, September 15–18). Designers’ natural descriptions of interactive behaviors. In 2008 IEEE Symposium on Visual Languages and Human-Centric Computing, VL/HCC’08 (pp. 185–188). Herrsching am Ammersee, Germany.Google Scholar
  41. Patel, K. D. (2013). Lowering the barrier to applying machine learning. Seattle, WA: University of Washington. PhD Dissertation.Google Scholar
  42. Scaffidi, C., Myers, B., Shaw, M. (2008, May 10–18). Topes: reusable abstractions for validating data. In ICSE’08: International Conference on Software Engineering (pp. 1–10). Leipzig, Germany.Google Scholar
  43. Sheil, B. (1983, February). Environments for exploratory programming. In Datamation. Reprinted in in “Papers on Interlisp-D,” Sheil, B.A. and Masinter, L.M., eds., Xerox PARC Tech Report CIS-5.Google Scholar
  44. Smith, A. (2015, April 1). U.S. smartphone use in 2015. Pew Research Center.
  45. Sun, M., Chen, Y.N., Rudnicky, A.I. (2016, March 10). Learning user intentions spanning multiple domains. In Proceedings of IUI 2016 Workshop on Interacting with Smart Objects (SmartObjects 2016). Sonoma, California.Google Scholar
  46. Yang, H., Pupons-Wickham, D., Chiticariu, L., Li, Y., Nguyen, B., Carreno-Fuentes, A. (2013). I can do text analytics!: designing development tools for novice developers. In CHI’2013: Human Factors in Computing Systems (pp. 1599–1608). Paris, France.Google Scholar
  47. Yoon, Y.S., Koo, S., Myers, B.A. (2013, September 15–19). Visualization of fine-grained code change history. In IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC’13) (pp. 119–126). San Jose, CA.Google Scholar
  48. Yoon, Y.S., & Myers, B.A. (2012, June 2). An exploratory study of backtracking strategies used by developers. In Cooperative and Human Aspects of Software Engineering (CHASE’2012), An ICSE 2012 Workshop (pp. 138–144). Zurich, Switzerland.Google Scholar
  49. Yoon, Y.S., & Myers, B.A. (2014, 28 July–1 August). A longitudinal study of programmers’ backtracking. In IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC’14) (pp. 101–108). Melbourne, Australia.Google Scholar
  50. Yoon, Y.S., & Myers, B.A. (2015, May 16–24). Supporting selective undo in a code editor. In 37th International Conference on Software Engineering, ICSE 2015 (vol. 1; pp. 223–233). Florence, Italy.Google Scholar
  51. Zang, N., Rosson, M.B., Nasser, V. (2008). Mashups: who? what? why? In CHI’08 Extended Abstracts on Human Factors in Computing Systems (pp. 3171–3176). New York, NY.Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Carnegie Mellon UniversityPittsburghUnited States
  2. 2.University of WashingtonSeattleUnited States
  3. 3.Oregon State UniversityCorvallisUnited States
  4. 4.University of MichiganAnn ArborUnited States
  5. 5.GoogleMountain ViewUnited States
  6. 6.IBMArmonkUnited States

Personalised recommendations