Can the System Be Improved?
The preceding three chapters have focused on formative evaluation, the assessment of the system as part of the overall iterative design process (as described in Chapter 5). The main point of this evaluation is to find out about the design of the system during its development, in order to address those aspects that need modification and revision. This will allow changes to be made before product release. In contrast, summative evaluation takes place when the system or product has moved from being experimental to operational, and is usually carried out towards the end of the design life cycle. Consequently, the broad difference between formative and summative evaluation concerns the nature of the purpose. Formative evaluation (as implied by the term) is carried out to help the system designer refine and ‘form’ the design, while summative evaluation is concerned with the overall performance of the system. Hewett (1986) defines summative evaluation as follows: ‘Summative evaluation involves assessing the impact, usability and effectiveness of the system - the overall performance of User and system.’ Implicit here is the suggestion that the system or product is close to being launched.
KeywordsSummative Evaluation Automatic Speech Recognition International Standard Organisation Verbal Protocol Slide Show
Unable to display preview. Download preview PDF.
- The bulk of the references on summative methods are those on usability. Often, books on this topic approach usability by considering other methods that may be engaged in conducting a usability assessment, e.g. heuristic evaluation, experimental studies.Google Scholar
- Jakob Nielsen, Usability Engineering, Academic Press, 1993. (See’ selected References’, Chapter 6.)Google Scholar
- Michael E. Wiklund (ed.), Usability in Practice: How Companies Develop User-friendly Products, Academic Press, London, 1994.Google Scholar
- Seventeen leading companies (mainly in the USA) outline their programmes to ensure usability, providing a unique insight into what organisations are doing in terms of usability in the 1990s.Google Scholar
- Jeffrey Rubin, Handbook of Usability Testing, Wiley, New York, 1994.Google Scholar
- This book assumes no prior knowledge of usability testing on the part of the reader. It takes you through step-by-step how to plan, design and conduct effective usability tests. It also points out common problems, with the use of reallife examples and case histories.Google Scholar
- Patrick W. Jordan, An Introduction to Usability, Taylor and Francis, London, 1998.Google Scholar
- This book, as the title suggests, provides a broad introduction to the topic, covering concepts and definitions of usability, principles of design, and advantages and disadvantages of methods for usability evaluation.Google Scholar