Skip to main content
SpringerLink
Log in

Impact of response delay and training on user performance with text-based and graphical user interfaces for engineering design

  • Original Paper
  • Published:
Research in Engineering Design Aims and scope Submit manuscript

Abstract

With recent computer advances, visualization techniques are becoming more prevalently used as decision support tools for parametric design and engineering optimization. Despite the apparent advantages of visualization techniques, we have found little evidence in the engineering design literature that assesses the impact of fast graphical design interfaces on the efficiency and effectiveness of engineering design decisions. In this paper, we present experimental results from an I-beam design problem where the importance of rapid feedback is investigated by incorporating time delays in the software response to “mimic” computationally expensive design analyses. Design efficiency is measured by recording the completion time for solving the design problem, and design effectiveness is measured by calculating the error between a submitted design and the known optimum. The impact of graphical feedback is examined by comparing user performance on three different design interfaces to determine if their functionality and graphical capabilities mediate the impact of response delays in the software or the amount of training needed. Experimental results indicate that, on average, error increased by 280% and completion time increased by 33% when a delay of 1.5 s was present, and the perceived workload significantly increased as well. Meanwhile, user performance improved and perceived workload decreased as the “richness” of the design interface increased. The combination of a rich interface with a fast response time, therefore, will lead to the “best” interface as one might surmise, but our work provides the first empirical evidence of the effect of response delay on user performance within a realistic engineering design setting. Implications for interface development for engineering design are explored within the context of our findings, as are suggestions for future work.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  • Card SK, Moran TP, Newell A (1983) The psychology of human-computer interaction. Lawrence Erlbaum, Hillsdale, NJ

    Google Scholar 

  • Carignan CR, Akin DL (2001) Advanced operator interface design for complex space telerobots. Auton Robots 11(1):49–58

    Article  MATH  Google Scholar 

  • Cohen J (1988) Statistical power analysis for the behavioral sciences. Lawrence Erlbaum Associates, Hillsdale, NJ

    MATH  Google Scholar 

  • Dahl DW, Chattopadhyay A, Gorn GJ (2001) The importance of visualisation in concept design. Des Stud 22(1):5–26

    Google Scholar 

  • Day PN, Holt PO, Russell GT (1999) Modeling the effects of delayed visual feedback in real-time operator control loops: a cognitive perspective. In: Proceedings of the XVIII European annual conference on human decision making and manual control, Group D Publications Ltd., Loughborough, UK, pp 70–79

  • Diehl E, Sterman JD (1995) Effects of feedback complexity on dynamic decision making. Organ Behav Hum Decis Process 62(2):198–215

    Article  Google Scholar 

  • Dobson MW, Pengelly M, Sime J-A, Albaladejo SA, Garcia EV, Gonzales F, Maseda J (2001) Situated learning with co-operative agent simulations in team training. Comput Hum Behav 17(5–6):547–573

    Article  Google Scholar 

  • Eddy J, Lewis K (2002) Multidimensional design visualization in multiobjective optimization. In: Proceedings of the 9th AIAA/ISSMO symposium on multidisciplinary analysis and optimization, Atlanta, GA, AIAA, AIAA-2002-5621

  • Eddy WF, Mockus A (1995) Dynamic visualization in modeling and optimization of ill-defined problems: case studies and generalizations. Technical Report, Department of Statistics, Carnegie Mellon University, Pittsburgh, PA

    Google Scholar 

  • Evans PT, Vance JM, Dark VJ (1999) Assessing the effectiveness of traditional and virtual reality interfaces in spherical mechanism design. ASME J Mech Des 121(4):507–514

    Google Scholar 

  • Foley JD, Wallace JD (1974) The art of natural graphic man-machine conversation. Proc IEEE 4:462–471

    Article  Google Scholar 

  • Frecker M, Simpson TW, Goldberg JH, Barton RR, Holewinski B, Stump G (2001) Integrating design research into the classroom: experiments in two graduate courses. In: proceedings of the 2001 annual asee conference, Albuquerque, NM, ASEE

  • Goodman T, Spence R (1978) The effect of system response time on interactive computer-aided design. Comp Graph 12:100–104

    Article  Google Scholar 

  • Haftka R, Gürdal Z (1992) Elements of structural optimization, 3rd Revised and Expanded edn. Kluwer Academic Publishers, Boston, MA

    Google Scholar 

  • Hart SG, Staveland LE (1988) Development of NASA-TLX (Task Load Index): results of experimental and theoretical research. In: Hancock PA, Meshkati N (eds) Human mental workload. North Holland, Amsterdam, pp 139–183

  • Henderson T (2001) Latency and user behaviour on a multiplayer game server. In: Proceedings of the 3rd international COST264 workshop on networked group communication, Lecture Notes In Computer Science, vol 2233. Springer, New York, pp 1–13

  • Hirschi NW, Frey DD (2002) Cognition and complexity: an experiment on the effect of coupling in parameter design. Res Eng Des 13(3):123–131

    Google Scholar 

  • Jay C, Hubbold R (2004) Does performance in the face of delayed sensory feedback change with practice?. In: Proceedings of the Eurohaptics conference, Springer, New York, pp 530–533

  • Jay C, Hubbold R (2006) Quantifying the effects of latency on sensory feedback in distributed virtual environments. In: Proceedings of the virtual images seminar 2006, College de France, Paris, pp 9–16

  • Jayaram S, Vance JM, Gadh R, Jayaram U, Srinivasan H (2001) Assessment of VR technology and its applications to engineering problems. ASME J Comput Inf Sci Eng 1(1):72–83

    Article  Google Scholar 

  • Jones CV (1994) Visualization and optimization. ORSA Journal of Computing 6(3):221–257

    MATH  Google Scholar 

  • Jost K (1998) Chryslers clean sheet V6. Automot Eng Int 106(1):79–81

    MathSciNet  Google Scholar 

  • Kelsick J, Vance JM, Buhr L, Moller C (2004) Discrete event simulation implemented in a virtual environment. ASME J Mech Des 125(3):428–433

    Article  Google Scholar 

  • Kesavadas T, Sudhir A (2000) Computational steering in simulation of manufacturing systems. In: Proceedings of the 2000 IEEE international conference on robotics and automation, San Francisco, CA, IEEE 3:2654–2658

  • Kihonge JN, Vance JM, Larochelle PM (2002) Spatial mechanism design in virtual reality with networking. ASME J Mech Des 124(3):435–440

    Article  Google Scholar 

  • Kirlik A, Walker N, Fisk AD, Nagel K (1996) Supporting perception in the service of dynamic decision making. Hum Factors 38(2):288–299

    Article  Google Scholar 

  • Kodiyalam S, Yang R-J, Gu L (2004) High performance computing and surrogate modeling for rapid visualization with multidisciplinary optimization. AIAA J 42(11):2347–2354

    Google Scholar 

  • La Viola JJ Jr (2000) A discussion of cybersickness in virtual environments. ACM SIGCHI Bull 32(1):47–56

    Article  Google Scholar 

  • Lewis K, Bloebaum CL (2000) The use of visualization to aid multidisciplinary decision making in design. In: Proceedings of the NSF grantees conference, Vancouver, British Columbia, Canada, National Science Foundation, Arlington, VA

  • Ligetti C, Simpson TW (2005) Metamodel-driven design optimization using integrative graphical design interfaces: results from a job shop manufacturing simulation experiment. ASME J Comput Inf Sci Eng 5(1):8–17

    Article  Google Scholar 

  • Ligetti C, Simpson TW, Frecker M, Barton RR, Stump G (2003) Assessing the impact of graphical design interfaces on design efficiency and effectiveness. ASME J Comput Inf Sci Eng 3(2):144–154

    Article  Google Scholar 

  • Liu L, Watson B, Miyazaki M (1999) VR for the elderly: quantitative and qualitative differences in performance with a driving simulator. CyberPsychol Behav 2(6):567–576

    Article  Google Scholar 

  • MacKenzie IS, Ware C (1993) Lag as a determinate of human performance in interactive systems. In: Proceedings of the SIGCHI conference on human factors in computing systems, ACM, New York, pp 488–493

  • Maxfield J, Juster NP, Dew PM, Taylor S, Fitchie M, Ion WJ, Zhao J, Thompson M (2000) Predicting product cosmetic quality using virtual environments. ASME Design Engineering Technical Conferences—Computers and Information in Engineering, Baltimore, MD, ASME, Paper No. DETC2000/CIE-14591

  • Messac A, Chen X (2000) Visualizing the optimization process in real-time using physical programming. Eng Optimiz 32(6):721–747

    Article  Google Scholar 

  • Mulder JD, van Liere R, van Wijk JJ (1998) Computational steering in the CAVE. Future Generation Comput Syst 14(3):199–207

    Article  Google Scholar 

  • National Research Council (1998) Visionary manufacturing challenges for 2020. Committee on Visionary manufacturing challenges, national research council. National Academy Press, Washington, DC

    Google Scholar 

  • Ng WY (1991) Generalized computer-aided design system: a multiobjective approach. Comput Aided Des 23(8):548–553

    Article  MATH  Google Scholar 

  • Osio IG, Amon CH (1996) An engineering design methodology with multistage bayesian surrogates and optimal sampling. Res Eng Des 8(4):189–206

    Article  Google Scholar 

  • Ottensmeyer MP, Hu J, Thompson JM, Ren J, Sheridan TB (2000) Investigations into performance of minimally invasive Telesurgery with Feedback time delays. Presence 9(4):369–382

    Article  Google Scholar 

  • Pantel L, Wolf LC (2002) Network issues for video and games: on the impact of delay on real-time multiplayer games. In: Proceedings of the I2th international workshop on network and operating systems support for digital audio and video, Miami, FL, ACM, pp 23–29

  • Pausch R, Crea T, Conway M (1992) A literature survey for virtual environments: military flight simulator visual systems and simulator sickness. Presence 1(3):344–363

    Google Scholar 

  • Rothrock L, Barron K, Simpson TW, Frecker M, Barton RR, Ligetti C (2006) Applying the proximity compatibility and the control-display compatibility principles to engineering design interfaces. Hum Factors Ergon Manuf 16(1):61–81

    Article  Google Scholar 

  • Sabbagh K (1996) Twenty-first century jet: the making and marketing of the boeing 777. Scribner, New York, NY

    Google Scholar 

  • Sheridan TB (1993) Space teleoperation through time delay: review and prognosis. IEEE Trans Rob Autom 9(5):592–606

    Article  Google Scholar 

  • Simpson TW, Iyer P, Barron K, Rothrock L, Frecker M, Barton RR, Meckesheimer M (2005) Metamodel-driven interfaces for engineering design: impact of delay and problem size on user performance. In: Proceedings of the 1st AIAA multidisciplinary design optimization specialist conference, Austin, TX, AIAA, AIAA-2005-2060

  • Simpson TW, Meckesheimer M (2004) Evaluation of a graphical design interface for design space visualization. In: Proceedings of the 45th AIAA/ASME/ASCE/AHS/ASC structures, structural dynamics & materials conference, Palm Springs, CA, AIAA, AIAA-2004-1683

  • Simpson TW, Peplinski J, Koch PN, Allen JK (2001) Metamodels for computer-based engineering design: survey and recommendations. Eng Comput 17(2):129–150

    Article  MATH  Google Scholar 

  • Stump G, Yukish M, Simpson TW, Harris EN (2003) Design space visualization and its application to a design by shopping paradigm. ASME Design Engineering Technical Conferences—Design Automation Conference, Chicago, IL, ASME, Paper No. DETC2003/DAC-48785

  • Stump GM, Yukish M, Simpson TW, Bennett L (2002) Multidimensional visualization and its application to a design by shopping paradigm. In: Proceedings of the 9th AIAA/ISSMO symposium on multidisciplinary analysis and optimization, Atlanta, GA, AIAA, AIAA-2002-5622

  • Sturman DJ, Zeltzer D, Pieper S (1989) Hands-on interaction with virtual environments. In: Proceedings of the 1989 ACM SIGGRAPH symposium on user interface software and technology, Williamsburg, VA, pp 19–24

  • Waern Y (1989) Cognitive aspects of computer supported tasks. Wiley, New York

    Google Scholar 

  • Ware C, Balakrishnan R (1994) Reaching for objects in VR displays: lag and frame rate. ACM Trans Comput Hum Int 1(4):331–356

    Article  Google Scholar 

  • Watson B, Walker N, Hodges LF, Worden A (1997) Managing level of detail through peripheral degradation: effects on search performance in head-mounted display. ACM Trans Comput Hum Int 4:323–346

    Article  Google Scholar 

  • Wickman C, Soderberg R (2003) Increased concurrency between industrial and engineering design using CAT technology combined with virtual reality. Concurrent Eng Res Appl 11(1):7–15

    Article  Google Scholar 

  • Wierwille WW, Eggemeier FT (1993) Recommendations for mental workload measurement in a test and evaluation environment. Hum Factors 35(2):262–282

    Google Scholar 

  • Winer EH, Bloebaum CL (2001) Visual design steering for optimization solution improvement. Struct Optimiz 22(3):219–229

    Article  Google Scholar 

  • Winer EH, Bloebaum CL (2002a) Development of visual design steering as an aid in large-scale multidisciplinary design optimization. Part I: method development. Struct Multidisciplinary Optimiz 23(6):412–424

    Article  Google Scholar 

  • Winer EH, Bloebaum CL (2002b) Development of visual design steering as an aid in large-scale multidisciplinary design optimization. Part II: method validation. Struct Multidisciplinary Optimiz 23(6):425–435

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the National Science Foundation under Grant No. DMI-0084918, and Kim Barron was supported by a NSF REU (Research Experiences for Undergraduates) Supplement to this grant. Any opinions, findings, and conclusions or recommendations presented in this paper are those of the authors and do not necessarily reflect the views of the National Science Foundation.

Author information

Authors and Affiliations

  1. Mechanical and Industrial Engineering and Engineering Design, Penn State University, 329 Leonhard Building, University Park, PA, 16802, USA

    Timothy W. Simpson

  2. Department of Industrial and Manufacturing Engineering, Penn State University, University Park, PA, 16802, USA

    Kimberly Barron, Ling Rothrock & Chris Ligetti

  3. Mechanical Engineering, Penn State University, University Park, PA, 16802, USA

    Mary Frecker

  4. Supply Chain and Information Systems, Penn State University, University Park, PA, 16802, USA

    Russell R. Barton

Authors
  1. Timothy W. Simpson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kimberly Barron
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ling Rothrock
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mary Frecker
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Russell R. Barton
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chris Ligetti
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Timothy W. Simpson.

Appendix

Appendix

Table 3.

Table 3 Summary of results of secondary hypothesis testing
Full size table

Rights and permissions

Reprints and permissions

About this article

Cite this article

Simpson, T.W., Barron, K., Rothrock, L. et al. Impact of response delay and training on user performance with text-based and graphical user interfaces for engineering design. Res Eng Design 18, 49–65 (2007). https://doi.org/10.1007/s00163-007-0033-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00163-007-0033-y

Keywords

Search

Navigation