Skip to main content

Systematic evaluation of design choices for software development tools

Abstract

Most design and evaluation of software tools is based on the intuition and experience of the designers. Software tool designers consider themselves typical users of the tools that they build and tend to subjectively evaluate their products rather than objectively evaluate them using established usability methods. This subjective approach is inadequate if the quality of software tools is to improve and the use of more systematic methods is advocated. This paper summarises a sequence of studies that show how user interface design choices for software development tools can be evaluated using established usability engineering techniques. The techniques used included guideline review, predictive modelling and experimental studies with users.

This is a preview of subscription content, access via your institution.

References

  1. Broom, B., Welsh, J., Wildman, L.: UQ2 Editor User Manual. Technical Report. Department of Computer Science. University of Queensland. Brisbane, Australia 1989

    Google Scholar 

  2. Card, S.K., Moran, T.P., Newell, A.: The Keystroke-Level Model for User Performance Time with Interactive Systems. Communications of the ACM 23(7): 396–410, 1980

    Article  Google Scholar 

  3. Card, S.K., Moran, T.P., Newell, A.: The Psychology of Human-Computer Interaction. Hillsdale, NJ: Lawrence Erlbaum Associates, 1983

    Google Scholar 

  4. Eberts, R.E.: User Interface Design. Englewood Cliffs, NJ: Prentice-Hall, 1994

    MATH  Google Scholar 

  5. Embley D.W., Nagy, G.: Behavioral Aspects of Text Editors. ACM Computing Surveys 13(1): 33–70, 1981

    Article  Google Scholar 

  6. Gray, W.D., John, B.E., Atwood, M.E.: Project Ernestine: Validating a GOMS analysis for predicting and explaining real-world performance. Human-Computer Interaction, 8(3): 237–309, 1993

    Article  Google Scholar 

  7. Green, T.R.G.: Why software engineers don’t listen to what psychologists don’t tell them anyway. In: Gilmore, D.J., Winder, R.L., Detiénne, F. (eds.): User-Centred Requirements for Software Engineering Environments. pp. 323–333. Berlin, Heidelberg, New York: Springer-Verlag, 1994

    Chapter  Google Scholar 

  8. Jeffries, R., Miller, J.R., Wharton, C., Uyeda, K.M.: User interface evaluation in the real world: A comparison of four techniques. In: Robertson, S.P., Olson, G.M., Olson, J.S. (eds.): Human Factors in Computing Systems - CHI’91, pp. 119–124, 1991

    Google Scholar 

  9. Jesshope, C.R., Crawley, M.J., Lovegrove, G.L.: An intelligent Pascal editor for a graphical oriented workstation. Software — Practice and Experience. 15(11): 1103–1119, 1985

    Article  Google Scholar 

  10. Khwaja, A.A., Urban, J.E.: Syntax-directed editing environments: Issues and features. In: Deaton, E., Berghel, G.H., Hedrick, G. (eds.): Applied Computing: States of the Art and Practice — 1993, New York, NY: ACM Press 1993, pp. 230–237; Also in Proceedings of the 1993 ACM SIGAPP Symposium, Indianapolis, IN, 14–16 February

    Google Scholar 

  11. Löwgren, J., Laurén, U.: Supporting the use of guidelines and style guides in professional user interface design. Interacting with Computers. 5(4): 385–396, 1993

    Article  Google Scholar 

  12. Mack, R.L., Nielsen, J.: Executive summary. In: Nielsen, J., Mack, R.L. (eds.): Usability Inspection Methods. New York: John Wiley 1994, p. 123

    Google Scholar 

  13. Minör, S.: On structure-oriented editing. PhD thesis, Department of Computer Science. Lund University, Sweden 1990

    Google Scholar 

  14. Myers, B.A.: Challenges of HCI design and implementation. Interactions. 1(1): 73–83, 1994

    Article  Google Scholar 

  15. Neal, L., Szwillus, G.: Report on the CHI’90 workshop on structure editors. SIGCHI Bulletin. 22(2): 49–53, 1990

    Article  Google Scholar 

  16. Nielsen, J.: The usability engineering life cycle. Computer. 25(3): 12–22, 1992

    Article  Google Scholar 

  17. Olson, J.R., Olson, G.M.: The growth of cognitive modeling in human-computer interaction since GOMS. Human-Computer Interaction. 5(23): 221–265, 1990

    Article  Google Scholar 

  18. Peck, V.A., John, B.E.: Browser-Soar: A computational model of a highly interactive task. In: Bauersfeld, P., Bennett, J., Lynch, G. (eds.): Human Factors in Computing Systems - CHI’92 1992, pp. 165–172

  19. Perlman, G., Moorhead, T.: NaviTextSAM Software UserManual. Northern Lights Software Corporation. Westford, MA 1988

    Google Scholar 

  20. Reps, T.W., Teitelbaum, T.: The Synthesizer Generator. SIGPLAN Notices, 19(5): 42–48, 1984; Also in: Henderson, P. (ed.): Proceedings of ACM SIGSOFT/SIGPLAN Software Engineering Symposium on Practical Software Development Environments. Pittsburgh, May 1984

    Article  Google Scholar 

  21. Smith, S.L., Mosier, J.N.: Design guidelines for user-system interface software. Technical Report ESD-TR-84-190. Mitre Corporation, Bedford, MA 1984, NTIS AD A154 907

    Google Scholar 

  22. Smith, S.L., Mosier, J.N.: The user interface to computerbased information systems: A survey of current software design practice. Behaviour and Information Technology. 3(4): 195–203, 1984

    Article  Google Scholar 

  23. Smith, S.L., Mosier, J.N.: Guidelines for designing user interface software. Technical Report ESD-TR-86-278. Mitre Corporation, Bedford, MA 1986; Also published as NTIS AD A177 198

    Google Scholar 

  24. Soloway, E.: A cognitively-based methodology for designing languages, environments and methodologies. SIGPLAN Notices. 19(5): 193–196, 1984

    Article  Google Scholar 

  25. Sommerville, I.: Software Engineering. Wokingham, England: Addison-Wesley, fifth edition 1995

    Google Scholar 

  26. Taylor, R.N., Coutaz, J.: Workshop on software engineering and computer-human interaction: Joint research issues. In: Proceedings of the Sixteenth International Conference on Software Engineering 1994, pp. 356–357

    Google Scholar 

  27. Thimbleby, H.: Failure in the technical user-interface design process. Computers and Graphics. 9(3): 187–193, 1990

    Article  Google Scholar 

  28. Toleman, M.A.: The Design of the User Interface for Software Development Tools. PhD thesis, Department of Computer Science, University of Queensland, Brisbane, Australia, December 1996

    Google Scholar 

  29. Toleman, M.A., Welsh, J.: Retrospective application of user interface guidelines: A case study of a language-based editor. In: Hammond, J.H., Hall, R.R., Kaplan, I. (eds.): People Before Technology - OZCHI’91 1991, pp. 33–38

    Google Scholar 

  30. Toleman M.A., Welsh, J.: An evaluation of editing paradigms. In: Howard, S., Leung, Y.K. (eds.): Harmony Through Working Together — OZCHI’94 1994, pp. 73–78; Also available in extended version as Technical Report 94-5. Software Verification Research Centre, University of Queensland, Brisbane, Australia

    Google Scholar 

  31. Toleman, M.A., Welsh, J.: An empirical investigation of language-based editing paradigms. In: Hasan, H., Nicastri, C. (eds.): HCI: A Light into the Future - OZCHI’95 1995, pp. 163–168; Also available as Technical Report 95-45, Software Verification Research Centre, University of Queensland, Brisbane, Australia

    Google Scholar 

  32. Toleman, M.A., Welsh, J.: Can design choices for languagebased editors be analysed with Keystroke-Level Models? In: Sasse, M.A., Cunningham, R.J., Winder, R.L. (eds.): People and Computers XI - Proceedings of HCI’96, London 1996, pp. 97–112; Berlin, Heidelberg, New York: Springer-Verlag. Presented at HCI’96, London, 20–23 August. Also available as Technical Report 96-36, Software Verification Research Centre, University of Queensland, Brisbane, Australia

    Google Scholar 

  33. Toleman, M.A., Welsh, J., Chapman, A.J.: An empirical investigation of menu design in language-based editors. ACM SIGSOFT Software Engineering Notes. 17(5): 41–46, 1992; Also in: Weber, H. (ed.): ACM SIGSOFT’92: Fifth Symposium on Software Development Environments. Washington D.C. 9-11 December

    Article  Google Scholar 

  34. Welsh, J., Broom, B., Kiong, D.: A design rationale for a language-based editor. Software — Practice and Experience. 21(9): 923–948, 1991

    Article  Google Scholar 

  35. Welsh, J., Han, J.: Software documents: Concepts and tools. Software — Concepts and Tools. 15(1): 12–25, 1994

    Google Scholar 

  36. Welsh, J., Rose, G.A., Lloyd, M.: An adaptive program editor. The Australian Computer Journal. 18(2): 67–74, 1986

    Google Scholar 

  37. Whittle, B.R., Gautier, R.J., Ratcliffe, M.: Trends in structure-oriented environments. International Journal of Software Engineering and Knowledge Engineering. 4(1): 123–157, 1994

    Article  Google Scholar 

  38. Zelkowitz, M.V., Kowalchack, B., Itkin, D., Herman, L.: Experiences building a syntax-directed editor. Software Engineering Journal. 4(6): 294–300, 1989

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Mark A. Toleman.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Toleman, M.A., Welsh, J. Systematic evaluation of design choices for software development tools. Software - Concepts & Tools 19, 109–121 (1998). https://doi.org/10.1007/s003780050014

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s003780050014

Key words

  • Guideline review
  • Experimental studies
  • Predictive models
  • Software development tools
  • Language-based editors