Abstract
The rate of introduction of new technology into safety critical domains continues to increase. Improvements in evaluation methods are needed to keep pace with the rapid development of these technologies. A significant challenge in improving evaluation is developing efficient methods for collecting and characterizing knowledge of the domain and context of the work being performed. Traditional methods of incorporating domain and context knowledge into an evaluation rely upon expert user testing, but these methods are expensive and resource intensive. This paper will describe three new methods for evaluating the applicability of a user interface within a safety-critical domain (specifically aerospace work domains), and consider how these methods may be incorporated into current evaluation processes.
Chapter PDF
Similar content being viewed by others
References
Ballard, D.H., Hayhoe, M.M.: Modeling the role of task in the control of gaze. Visual Cognition 17(6-7), 1185–1204 (2009)
Billman, D., Feary, M., Schreckengost, D., Sherry, L.: Needs analysis: the case of flexible constraints and mutable boundaries. In: CHI 2010 Extended Abstracts on Human Factors in Computing Systems (CHI EA 2010), pp. 4597–4612. ACM, New York (2010), http://doi.acm.org/10.1145/1753846.1754201 , doi:10.1145/1753846.1754201
Billman, D., Arsintescucu, L., Feary, M., Lee, J., Smith, A., Tiwary, R.: Benefits of matching domain structure for planning software: the right stuff. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2011), pp. 2521–2530. ACM, New York (2011), http://doi.acm.org/10.1145/1978942.1979311 , doi:10.1145/1978942.1979311
Billman, D., Arsintescu, L., Feary, M., Lee, J., Schreckenghost, D., Tiwary, R.: Product-Based Needs Analysis and Case Study of Attitude Determination and Control Operator (ADCO) Planning Work (in preparation as NASA Technical Memorandum)
Caulton, D.A.: Relaxing the homogeneity assumption in usability testing. Behaviour & Information Technology 20(1), 1–7 (2001)
Eng, K., Lewis, R.L., Tollinger, I., Chu, A., Howes, A., Vera, A.: Generating automated predictions of behavior strategically adapted to specific performance objectives. In: Proceedings of the Sigchi Conference on Human Factors in Computing Systems, pp. 621–630 (2006)
Engbert, R., Nuthmann, A., Richter, E.M., Kliegl, R.: SWIFT: a dynamical model of saccade generation during reading. Psychological Review 112(4), 777 (2005)
European Aviation Safety Agency, Certification Specification 25.1302 and Acceptable Means of Compliance 25.1302 Installed Systems and Equipment for Use by the Flight Crew (2007)
Faulkner, L.: Beyond the five-user assumption: Benefits of increased sample sizes in usability testing. Behavior Research Methods 35(3), 379–383 (2003)
Feary, M.: Formal Identification of Automation Surprise Vulnerabilities in Design. Doctorate Dissertation, Cranfield University (2005)
Federal Aviation Administration AVS Workplan for Nextgen. Federal Aviation Administration, USA (2012)
Howes, A., Lewis, R.L., Vera, A.: Rational adaptation under task and processing constraints: Implications for testing theories of cognition and action. Psychological Review (2009)
Lewis, R., Shvartsman, M., Singh, S.: The adaptive nature of eye-movements in linguistic tasks: How payoff and architecture shape speed-accuracy tradeoffs. Topics in Cognitive Science (to appear)
Macefield, R.: How To Specify the Participant Group Size for Usability Studies: A Practitioner’s Guide. Journal of Usability Studies 5(1), 34–45 (2009)
Molich, R., Chattratichart, J., Hinkle, V., Jensen, J.J., Kirakowski, J., Sauro, J., Sharon, T., Traynor, B.: Rent a Car in Just 0, 60, 240 or 1,217 Seconds?-Comparative Usability Measurement, CUE-8. Journal of Usability Studies 6(1), 8–24 (2010)
Norris, D.: Putting it all together: A unified account of word recognition and reaction-time distributions. Psychological Review 116(1), 207 (2009)
Payne, S., Howes, A.: Adaptive Interaction: A utility maximisation approach to understanding human interaction with technology. Morgan Claypool lecture (in press)
Reichle, E.D., Rayner, K., Pollatsek, A.: The EZ Reader model of eye-movement control in reading: Comparisons to other models. Behavioral and Brain Sciences 26(4), 445–476 (2003)
Rothkopf, C.A., Ballard, D.H., Hayhoe, M.M.: Task and context determine where you look. Journal of Vision 7(14) (2007)
Salverda, A.P., Brown, M., Tanenhaus, M.K.: A goal-based perspective on eye movements in visual world studies. Acta Psychologica 137(2), 172–180 (2011)
Sherry, L., Medina-Mora, M., John, B., Teo, L., Polson, P., Blackmon, M., Koch, M., Feary, M.: System Design and Analysis: Tools for Automation Interaction Design and Evaluation Methods. Final Report NASA NRA NNX07AO67A (2010)
Wagenmakers, E.J., Ratcliff, R., Gomez, P., McKoon, G.: A diffusion model account of criterion shifts in the lexical decision task. Journal of Memory and Language 58(1), 140–159 (2008)
Wharton, C., Bradford, J., Jeffries, J., Franzke, M.: Applying Cognitive Walkthroughs to more Complex User Interfaces: Experiences, Issues and Recommendations. In: CHI 1992, pp. 381–388 (1992)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Feary, M. et al. (2013). Linking Context to Evaluation in the Design of Safety Critical Interfaces. In: Kurosu, M. (eds) Human-Computer Interaction. Human-Centred Design Approaches, Methods, Tools, and Environments. HCI 2013. Lecture Notes in Computer Science, vol 8004. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39232-0_22
Download citation
DOI: https://doi.org/10.1007/978-3-642-39232-0_22
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39231-3
Online ISBN: 978-3-642-39232-0
eBook Packages: Computer ScienceComputer Science (R0)