Skip to main content
Book cover

Human-in-the-Loop Simulations pp 131–152Cite as

Trade Space Exploration: Assessing the Benefits of Putting Designers “Back-in-the-Loop” during Engineering Optimization

Abstract

Trade space exploration is a promising decision-making paradigm that provides a visual and more intuitive means for formulating, adjusting, and ultimately solving engineering design optimization problems. This is achieved by combining multi-dimensional data visualization techniques with visual steering commands to allow designers to “steer” the optimization process while searching for the best, or Pareto optimal, designs. After introducing the trade space exploration paradigm and visual steering capabilities that we developed, we compare the performance of different combinations of visual steering commands implemented by two users to a multi-objective genetic algorithm executed “blindly” on the same problem with no human intervention. The results indicate that the visual steering commands—regardless of the order and combination in which they are invoked—provide a 4–7× increase in the number of Pareto solutions obtained for a given number of function evaluations when the human is “in-the-loop” during the optimization process. As such, this study provides empirical evidence of the benefits of interactive visualization-based strategies to support engineering design optimization and decision-making. Future work is also discussed.

Keywords

  • Function Evaluation
  • Pareto Front
  • Differential Evolution Algorithm
  • Pareto Solution
  • User Trial

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-0-85729-883-6_7
  • Chapter length: 22 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   129.00
Price excludes VAT (USA)
  • ISBN: 978-0-85729-883-6
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   169.00
Price excludes VAT (USA)
Hardcover Book
USD   169.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 7.1
Fig. 7.2
Fig. 7.3
Fig. 7.4
Fig. 7.5
Fig. 7.6
Fig. 7.7
Fig. 7.8
Fig. 7.9
Fig. 7.10
Fig. 7.11
Fig. 7.12

References

  • Balling R (1999) Design by shopping: a new paradigm? In: Proceedings of the third world congress of structural and multidisciplinary optimization (WCSMO-3), University at Buffalo, Buffalo, New York, pp 295–297

    Google Scholar 

  • Becker RA, Cleveland WS (1987) Brushing scatterplots. Technometrics 29(1):127–142

    CrossRef  MathSciNet  Google Scholar 

  • Buja A, McDonald JA, Michalak J, Stuetzle W (1991) Interactive data visualization using focusing and linking. In: Proceedings of the IEEE Conference on Visualization '91, IEEE Computer Society Press, San Diego, CA, pp 156–163

    Google Scholar 

  • Donndelinger J, Ferguson S, Lewis K (2006) Exploring mass trade-offs in preliminary vehicle design using pareto sets. 11th AIAA/ISSMO symposium on multidisciplinary analysis and optimization, Portsmouth, VA, AIAA-2006-7056

    Google Scholar 

  • Dym CL, Wood WH, Scott MJ (2006) On the legitimacy of pairwise comparisons. Decision Making in Engineering Design, ASME, 135–143

    Google Scholar 

  • Ferguson S, Gurnani A, Donndelinger J, Lewis, K (2005a) A study of convergence and mapping in multiobjective optimization problems. ASME design engineering technical conferences & Computers and information in engineering conference, Long Beach, CA, ASME, Paper No. DETC2005/CIE-84852

    Google Scholar 

  • Ferguson S, Gurnani A, Donndelinger J, Lewis K (2005b) An approach to feasibility assessment in preliminary design. ASME design engineering technical conferences—design automation conference, Long Beach, CA, ASME, Paper No. DETC2005/CIE-84853

    Google Scholar 

  • Kesavadas T, Sudhir A (2000) Computational steering in simulation of manufacturing systems. In: Proceedings of the 2000 IEEE International Conference on Robotics and Automation. IEEE, San Francisco, CA, pp 2654–2658

    Google Scholar 

  • Kollat JB, Reed PM (2005a) Comparing state-of-the-art evolutionary multi-objective algorithms for long-term groundwater monitoring design. Adv Water Resour 29(6):792–807

    CrossRef  Google Scholar 

  • Kollat JB, Reed PM (2005b) The value of online adaptive search: aperformance comparison of NSGAII, e-NSGAII, and eMOEA. Lecture Notes in Computer Science, Springer, Berlin, p 3410

    Google Scholar 

  • Madar J, Abonyi J, Szeifert F (2005) Interactive particle swarm optimization. International Conference on Intelligent Systems Design and Applications, IEEE, Wroclaw, Poland, pp 314–319

    Google Scholar 

  • Messac A, Chen X (2000) Visualizing the optimization process in real-time using physical programming. Eng Optim 32(6):721–747

    CrossRef  Google Scholar 

  • Michalek J, Papalambros P (2002) Interactive design optimization of architectural layouts. Eng Optim 34(5):485–501

    CrossRef  Google Scholar 

  • Miettinen K, Makela MM (2006) Synchronous approach in interactive multiobjective optimization. Eur J Oper Res 170:909–922

    CrossRef  MATH  Google Scholar 

  • North C (2006) Toward measuring visualization insight. IEEE Comp Graph Appl 26(3):6–9

    CrossRef  Google Scholar 

  • Okabe T, Jin Y, Sendhoff B (2003) A critical survey of performance indices for multi-objective optimisation. Evol Comput 2:878–885

    Google Scholar 

  • Price K, Storn R, Lampinen J (2005) Differential evolution: a practical approach to global optimization. Springer, Berlin

    MATH  Google Scholar 

  • Robic T, Filipic B (2005) DEMO: Differential evolution for multiobjective optimization. Third international conference on evolutionary multi-criterion optimization, Guanajuato, Mexico, Springer, pp 520–533

    Google Scholar 

  • Scott SD, Lesh N, Klau GW (2002) Investigating Human--Computer optimization. CHI’02, Minneapolis MN 4(1):155–162

    Google Scholar 

  • Shanteau J (1992) Competence in experts: the role of task characteristics. Organ Behavior and Human Decis 53(2):252–266

    CrossRef  Google Scholar 

  • Simpson TW, Donndelinger JA, Yukish M, Stump G (2007) Visual steering commands for trade space exploration: user-guided sampling with example. In: Proceedings of the ASME 2007 international design engineering technical conferences & computers and information in engineering conference, Las Vegas, NV, DETC2007/DAC-34684

    Google Scholar 

  • Stump G, Yukish M, Simpson TW (2004a) The advanced trade space visualizer: an engineering decision-making tool. 10th AIAA/ISSMO multidisciplinary analysis and optimization conference, Albany, NY, AIAA-2004-4568

    Google Scholar 

  • Stump G, Yukish M, Simpson TW, Harris EN, O’Hara JJ (2004b) Trade space exploration of satellite datasets using a design by shopping paradigm. IEEE Aerospace Conference, IEEE, Big Sky, MT

    Google Scholar 

  • Tang Y, Reed PM, Wagener T (2005) How effective and efficient are mulitobjective evolutionary algorithms at hydrologic model calibration. Hydrol Earth Syst Sci Discuss 2:2465–2520

    CrossRef  Google Scholar 

  • Wang GG, Shan S (2007) Review of metamodeling techniques in support of engineering design optimization. ASME J Mech Des 129(4):370–380

    CrossRef  MathSciNet  Google Scholar 

  • Wilson TD, Schooler JW (1991) Thinking too much: introspection can reduce the quality of preferences and decisions. J Pers Soc Psychol 60(2):181–192

    CrossRef  Google Scholar 

  • Winer EH, Bloebaum CL (2001) Visual design steering for optimization solution improvement. Struct Optim 22(3):219–229

    CrossRef  Google Scholar 

  • Winer EH, Bloebaum CL (2002) Development of visual design steering as an aid in large-scale multidisciplinary design optimization Part I: method development. Struct Multidiscipl Optim 23(6):412–424

    CrossRef  Google Scholar 

  • Wright H, Brodlie K, David T (2000) Navigating high-dimensional spaces to support design steering. Proceedings of IEEE visualization 2000, IEEE Computer Society Press, Salt Lake City, UT, pp 291–296

    Google Scholar 

  • Wu J, Azarm S (2001) Metrics for quality assessment of a multiobjective design optimization solution set. ASME J Mech Des 123(1):18–25

    CrossRef  Google Scholar 

  • Zitzler E (1999) Evolutionary algorithms for multiobjective: methods and applications. Swiss Federal Institute of Technology, Zurich, Switzerland

    Google Scholar 

Download references

Acknowledgments

We thank Dr. Scott Ferguson for his assistance with the vehicle model and the results from the exhaustive MOGA. This work has been supported by the National Science Foundation under Grant No. CMMI-0620948. Any opinions, findings, and conclusions or recommendations presented in this chapter are those of the authors and do not necessarily reflect the views of the National Science Foundation.

Author information

Authors and Affiliations

  1. Mechanical & Nuclear Engineering and Industrial & Manufacturing Engineering, The Pennsylvania State University, University Park, PA, 16802, USA

    Timothy W. Simpson, Dan Carlsen, Matthew Malone & Joshua Kollat

Authors
  1. Timothy W. Simpson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dan Carlsen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Matthew Malone
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Joshua Kollat
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Timothy W. Simpson .

Editor information

Editors and Affiliations

  1. , The Harold and Inge Marcus Department of, The Pennsylvania State University, Leonhard Building 210, State College, 16802, Pennsylvania, USA

    Ling Rothrock

  2. , 405 Russ Engineering Center, Wright State University, Col. Glenn Hwy 3640, Dayton, 45435, Ohio, USA

    S. Narayanan

Appendix

Appendix

7.1.1 Exploration Strategy for Trial 1 (Total Points = 5,025)

  • Basic Sampler: 100 runs.

  • Brush objectives 1–5: minimize Obj1 (−100), maximize Objs2–5 (100).

  • Point attractors: 10 possible pair-wise point attractors for Objs1–5 set at the current limits of the scatter plot window (on Objs [1 & 2], [3 & 4], [5 & 1], [2 & 3], [4 & 5], [1 & 3], [2 & 4], [3 & 5], [4 & 1], [5 & 2]).

  • Pareto Sampler.

7.1.2 Exploration Strategy for Trial 2 (Total Points = 5,075)

  • Basic sampler: 500 runs.

  • Brush Objs1–5: minimize Obj1 (−100), maximize Objs2–5 (100).

  • Pareto Sampler.

  • Line attractors (1D point attractor): one for each Obj1–5 set at the current limit of the scatter plot window (minimum of window for Obj1 and maximum of window for Objs2–5).

  • Preference Sampler.

  • Point attractors: set at current limits of the scatter plot window (on Objs [2 & 5], [2 & 4]).

  • Point attractors: set at the current limits of the scatter plot window, generation size changed to 15 (on Objs [3 & 2], [3 & 4], [1 & 5], [2 & 5]).

  • Point attractor: set at the current limits of the scatter plot window (on Objs [3 & 5]).

7.1.3 Exploration Strategy for Trial 3 (Total Points = 5525)

  • Basic Sampler: 500 runs.

  • Brush Objs1–5: minimize Obj 1 (−100), maximize Objs2–5 (100).

  • Point attractors: set at the current limits of the glyph plot (on Objs [1, 2, & 3], [1, 2, & 4], [1, 2, & 5], [1, 3, & 4], [1, 3, & 5], [1, 4, & 5], [2, 3, & 4], [2, 3, & 5], [2, 4, & 5], [3, 4, & 5]).

  • Pareto Sampler.

7.1.4 Exploration Strategy for Trial 4 (Total Points = 5375)

  • Basic Sampler: 100 runs.

  • Brush Objs1–5: minimize Obj1 (−100), maximize Objs2–5 (100).

  • Line attractors: set at the current limits of the scatter plot window (on Objs1–5).

  • Pareto Sampler.

  • Point attractors: set at the current limits of the scatter plot window, generation size changed to 15 and population limit changed to 250 (on Objs [1 & 2], [1 & 3], [1 & 4], [1 & 5], [2 & 3], [2 & 4], [2 & 5], [3 & 4], [3 & 5], [4 & 5]).

  • Line attractor (1D point attractor): set Obj3 at current limit of the scatter plot window.

  • Point attractors: set at current limits of the scatter plot window (on Objs [3 & 4], [4 & 5]).

7.1.5 Exploration Strategy for Trial 5 (Total points: 10,375)

  • Basic Sampler: 100 runs.

  • Brush Objs1–5: minimize Obj1 (−100), maximize Objs2–5 (100).

  • Point attractors: set at the current limits of the scatter plot window ±5% for minimizing or maximizing, respectively (on Objs [1 & 2], [2 & 3], [3 & 4], [4 & 5], [5 & 1], [1 & 3], [3 & 5], [5 & 2], [2 & 4], [4 & 1]).

  • Preference Sampler.

  • Point attractors: these specific values were used to fill in the Pareto front ([Obj1 = 0.9, Obj2 = 1.102], [Obj1 = 0.645, Obj2 = 0.872], [Obj2 = 1.144, Obj3 = 0.988]).

  • Line attractors (1D point attractors): these specific values were used to fill in the Pareto front ([Obj4 = 1.124], [Obj5 = 1.191]).

  • Brush (preference): minimize ConVio (−100).

  • Preference Sampler.

  • Pareto Sampler.

  • Line attractors (1D point attractors): one for each Obj1–5 set at the feasible limit of the objective in the scatter window (minimum for Obj1 and maximum for Objs2–5).

  • Pareto Sampler.

7.1.6 Exploration Strategy for Trial 6 (Total points: 10,375)

  • Basic Sampler: 250 runs.

  • Brush Objs1–5 and ConVio: minimize Obj1 and ConVio (−100), maximize Objs2–5 (100).

  • Preference Sampler: generation size changed to 50 and population limit changed to 1,000.

  • Pareto Sampler: generation size changed to 50 and population limit changed to 1,000.

  • Point attractors: set at the current limits of the scatter plot window (on [ConVio & Obj1], [ConVio & Obj2], [ConVio & Obj3], [ConVio & Obj4], [ConVio & Obj5]).

  • Pareto Sampler Generation size changed to 50 and population limit changed to 1,000.

  • Point attractors: these specific values were used to fill in the Pareto front ([ConVio = 0, Obj1 = 1.043, Obj2 = 1.2], [ConVio = 0, Obj1 = 0.755, Obj3 = 1.026], [ConVio = 0, Obj1 = 0.911, Obj4 = 1.121], [ConVio = 0, Obj1 = 0.729, Obj2 = 1.153], [ConVio = 0, Obj2 = 1.126, Obj3 = 0.993], [ConVio = 0, Obj2 = 1.186, Obj4 = 1.099], [ConVio = 0, Obj2 = 1.154, Obj5 = 1.052], [ConVio = 0, Obj3 = 1.018, Obj4 = 1.123], [ConVio = 0, Obj3 = 1.003, Obj5 = 1.137], [ConVio = 0, Obj4 = 1.121, Obj5 = 1.105], [ConVio = 0, Obj3 = 0.923, Obj5 = 0.993], [ConVio = 0, Obj2 = 1.207, Obj5 = 0.853]).

  • Preference Sampler.

  • Pareto Sampler.

  • Point attractors: use these specific values to fill in the Pareto front ([Obj1 = 0.802, Obj2 = 0.851, Obj3 = 1.007], [Obj3 = 1.003, Obj2 = 0.854], [Obj1 = 1.073, Obj2 = 1.19], [Obj4 = 0.995, Obj5 = 0.824], [Obj3 = 0.955, Obj4 = 1.119]).

  • Pareto Sampler: population limit changed to 250.

7.1.7 Exploration Strategy for Trial 7 (Total points = 10,125)

  • Basic Sampler: 25 runs.

  • Brush Objs 1–5 and ConVio: minimize Obj1 and ConVio (−100), maximize Objs2–5 (100).

  • Pareto Sampler: generation size changed to 50 and population limit changed to 1,000.

  • Preference Sampler: generation size changed to 50 and population limit changed to 1,000.

  • Repeat Pareto and Preference Samplers in above order with the same settings four more times.

  • Pareto Sampler: generation size changed to 50 and population limit changed to 1,000.

7.1.8 Exploration Strategy for Trial 8 (Total points = 10,275)

  • Basic Sampler: 25 runs.

  • Brush Objs 1–5 and ConVio: minimize Obj1 and ConVio (−100), maximize Objs2–5 (100).

  • Pareto Sampler: generation size changed to 50, population limit changed to 1,000, and selection strategy changed to Rand1Bin.

  • Preference Sampler: generation size changed to 50, population limit changed to 1,000, and selection strategy changed to Rand1Bin.

  • Repeat Pareto and Preference Samplers in above order with the same settings four more times.

Rights and permissions

Reprints and Permissions

Copyright information

© 2011 Springer-Verlag London Limited

About this chapter

Cite this chapter

Simpson, T.W., Carlsen, D., Malone, M., Kollat, J. (2011). Trade Space Exploration: Assessing the Benefits of Putting Designers “Back-in-the-Loop” during Engineering Optimization. In: Rothrock, L., Narayanan, S. (eds) Human-in-the-Loop Simulations. Springer, London. https://doi.org/10.1007/978-0-85729-883-6_7

Download citation

  • DOI: https://doi.org/10.1007/978-0-85729-883-6_7

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-0-85729-882-9

  • Online ISBN: 978-0-85729-883-6

  • eBook Packages: Computer ScienceComputer Science (R0)