Abstract
Most design and evaluation of software tools is based on the intuition and experience of the designers. Software tool designers consider themselves typical users of the tools that they build and tend to subjectively evaluate their products rather than objectively evaluate them using established usability methods. This subjective approach is inadequate if the quality of software tools is to improve and the use of more systematic methods is advocated. This paper summarises a sequence of studies that show how user interface design choices for software development tools can be evaluated using established usability engineering techniques. The techniques used included guideline review, predictive modelling and experimental studies with users.
This is a preview of subscription content, access via your institution.
References
Broom, B., Welsh, J., Wildman, L.: UQ2 Editor User Manual. Technical Report. Department of Computer Science. University of Queensland. Brisbane, Australia 1989
Card, S.K., Moran, T.P., Newell, A.: The Keystroke-Level Model for User Performance Time with Interactive Systems. Communications of the ACM 23(7): 396–410, 1980
Card, S.K., Moran, T.P., Newell, A.: The Psychology of Human-Computer Interaction. Hillsdale, NJ: Lawrence Erlbaum Associates, 1983
Eberts, R.E.: User Interface Design. Englewood Cliffs, NJ: Prentice-Hall, 1994
Embley D.W., Nagy, G.: Behavioral Aspects of Text Editors. ACM Computing Surveys 13(1): 33–70, 1981
Gray, W.D., John, B.E., Atwood, M.E.: Project Ernestine: Validating a GOMS analysis for predicting and explaining real-world performance. Human-Computer Interaction, 8(3): 237–309, 1993
Green, T.R.G.: Why software engineers don’t listen to what psychologists don’t tell them anyway. In: Gilmore, D.J., Winder, R.L., Detiénne, F. (eds.): User-Centred Requirements for Software Engineering Environments. pp. 323–333. Berlin, Heidelberg, New York: Springer-Verlag, 1994
Jeffries, R., Miller, J.R., Wharton, C., Uyeda, K.M.: User interface evaluation in the real world: A comparison of four techniques. In: Robertson, S.P., Olson, G.M., Olson, J.S. (eds.): Human Factors in Computing Systems - CHI’91, pp. 119–124, 1991
Jesshope, C.R., Crawley, M.J., Lovegrove, G.L.: An intelligent Pascal editor for a graphical oriented workstation. Software — Practice and Experience. 15(11): 1103–1119, 1985
Khwaja, A.A., Urban, J.E.: Syntax-directed editing environments: Issues and features. In: Deaton, E., Berghel, G.H., Hedrick, G. (eds.): Applied Computing: States of the Art and Practice — 1993, New York, NY: ACM Press 1993, pp. 230–237; Also in Proceedings of the 1993 ACM SIGAPP Symposium, Indianapolis, IN, 14–16 February
Löwgren, J., Laurén, U.: Supporting the use of guidelines and style guides in professional user interface design. Interacting with Computers. 5(4): 385–396, 1993
Mack, R.L., Nielsen, J.: Executive summary. In: Nielsen, J., Mack, R.L. (eds.): Usability Inspection Methods. New York: John Wiley 1994, p. 123
Minör, S.: On structure-oriented editing. PhD thesis, Department of Computer Science. Lund University, Sweden 1990
Myers, B.A.: Challenges of HCI design and implementation. Interactions. 1(1): 73–83, 1994
Neal, L., Szwillus, G.: Report on the CHI’90 workshop on structure editors. SIGCHI Bulletin. 22(2): 49–53, 1990
Nielsen, J.: The usability engineering life cycle. Computer. 25(3): 12–22, 1992
Olson, J.R., Olson, G.M.: The growth of cognitive modeling in human-computer interaction since GOMS. Human-Computer Interaction. 5(23): 221–265, 1990
Peck, V.A., John, B.E.: Browser-Soar: A computational model of a highly interactive task. In: Bauersfeld, P., Bennett, J., Lynch, G. (eds.): Human Factors in Computing Systems - CHI’92 1992, pp. 165–172
Perlman, G., Moorhead, T.: NaviTextSAM Software UserManual. Northern Lights Software Corporation. Westford, MA 1988
Reps, T.W., Teitelbaum, T.: The Synthesizer Generator. SIGPLAN Notices, 19(5): 42–48, 1984; Also in: Henderson, P. (ed.): Proceedings of ACM SIGSOFT/SIGPLAN Software Engineering Symposium on Practical Software Development Environments. Pittsburgh, May 1984
Smith, S.L., Mosier, J.N.: Design guidelines for user-system interface software. Technical Report ESD-TR-84-190. Mitre Corporation, Bedford, MA 1984, NTIS AD A154 907
Smith, S.L., Mosier, J.N.: The user interface to computerbased information systems: A survey of current software design practice. Behaviour and Information Technology. 3(4): 195–203, 1984
Smith, S.L., Mosier, J.N.: Guidelines for designing user interface software. Technical Report ESD-TR-86-278. Mitre Corporation, Bedford, MA 1986; Also published as NTIS AD A177 198
Soloway, E.: A cognitively-based methodology for designing languages, environments and methodologies. SIGPLAN Notices. 19(5): 193–196, 1984
Sommerville, I.: Software Engineering. Wokingham, England: Addison-Wesley, fifth edition 1995
Taylor, R.N., Coutaz, J.: Workshop on software engineering and computer-human interaction: Joint research issues. In: Proceedings of the Sixteenth International Conference on Software Engineering 1994, pp. 356–357
Thimbleby, H.: Failure in the technical user-interface design process. Computers and Graphics. 9(3): 187–193, 1990
Toleman, M.A.: The Design of the User Interface for Software Development Tools. PhD thesis, Department of Computer Science, University of Queensland, Brisbane, Australia, December 1996
Toleman, M.A., Welsh, J.: Retrospective application of user interface guidelines: A case study of a language-based editor. In: Hammond, J.H., Hall, R.R., Kaplan, I. (eds.): People Before Technology - OZCHI’91 1991, pp. 33–38
Toleman M.A., Welsh, J.: An evaluation of editing paradigms. In: Howard, S., Leung, Y.K. (eds.): Harmony Through Working Together — OZCHI’94 1994, pp. 73–78; Also available in extended version as Technical Report 94-5. Software Verification Research Centre, University of Queensland, Brisbane, Australia
Toleman, M.A., Welsh, J.: An empirical investigation of language-based editing paradigms. In: Hasan, H., Nicastri, C. (eds.): HCI: A Light into the Future - OZCHI’95 1995, pp. 163–168; Also available as Technical Report 95-45, Software Verification Research Centre, University of Queensland, Brisbane, Australia
Toleman, M.A., Welsh, J.: Can design choices for languagebased editors be analysed with Keystroke-Level Models? In: Sasse, M.A., Cunningham, R.J., Winder, R.L. (eds.): People and Computers XI - Proceedings of HCI’96, London 1996, pp. 97–112; Berlin, Heidelberg, New York: Springer-Verlag. Presented at HCI’96, London, 20–23 August. Also available as Technical Report 96-36, Software Verification Research Centre, University of Queensland, Brisbane, Australia
Toleman, M.A., Welsh, J., Chapman, A.J.: An empirical investigation of menu design in language-based editors. ACM SIGSOFT Software Engineering Notes. 17(5): 41–46, 1992; Also in: Weber, H. (ed.): ACM SIGSOFT’92: Fifth Symposium on Software Development Environments. Washington D.C. 9-11 December
Welsh, J., Broom, B., Kiong, D.: A design rationale for a language-based editor. Software — Practice and Experience. 21(9): 923–948, 1991
Welsh, J., Han, J.: Software documents: Concepts and tools. Software — Concepts and Tools. 15(1): 12–25, 1994
Welsh, J., Rose, G.A., Lloyd, M.: An adaptive program editor. The Australian Computer Journal. 18(2): 67–74, 1986
Whittle, B.R., Gautier, R.J., Ratcliffe, M.: Trends in structure-oriented environments. International Journal of Software Engineering and Knowledge Engineering. 4(1): 123–157, 1994
Zelkowitz, M.V., Kowalchack, B., Itkin, D., Herman, L.: Experiences building a syntax-directed editor. Software Engineering Journal. 4(6): 294–300, 1989
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Toleman, M.A., Welsh, J. Systematic evaluation of design choices for software development tools. Software - Concepts & Tools 19, 109–121 (1998). https://doi.org/10.1007/s003780050014
Published:
Issue Date:
DOI: https://doi.org/10.1007/s003780050014
Key words
- Guideline review
- Experimental studies
- Predictive models
- Software development tools
- Language-based editors