Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Industrial issues and solutions to statistical model improvement: a case study of an automobile steering column

  • 61 Accesses


Statistical model improvement consists of model calibration, validation, and refinement techniques. It aims to increase the accuracy of computational models. Although engineers in industrial fields are expanding the use of computational models in the process of product development, many field engineers still hesitate to perform statistical model improvement due to its practical aspects. Therefore, this paper describes research aimed at addressing three practical issues that hinder statistical model improvement in industrial fields: (1) lack of experimental data for quantifying uncertainties of true responses, (2) numerical input variables for propagating uncertainties of the computational model, and (3) model form uncertainties in the computational model. Issues 1 and 2 deal with difficulties in uncertainty quantification of experimental and computational responses. Issue 3 focuses on model form uncertainties, which are due to the excessive simplification of computational modeling; simplification is employed to reduce the calculation cost. Furthermore, the paper outlines solutions to address these three issues, specifically: (1) kernel density estimation with estimated bounded data, (2–1) variance-based variable screening, (2–2) surrogate modeling, and (3) a model refinement approach. By examining the computational model of an automobile steering column, these techniques are shown to demonstrate efficient statistical model improvement. This case study shows that the suggested approaches can actively reduce the burden in statistical model improvement and increase the accuracy of computational modeling, thereby encouraging its use in industry.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10


  1. Aeschliman D, Oberkampf W, Blottner F A Proposed methodology for computational fluid dynamics code verification, calibration, and validation. In: Instrumentation in Aerospace Simulation Facilities, 1995. ICIASF'95 Record., International Congress on, 1995. IEEE, pp 27/21–2713

  2. Anderson AE, Ellis BJ, Weiss JA (2007) Verification, validation and sensitivity studies in computational biomechanics. Comput Methods Biomech Biomed Eng 10:171–184. https://doi.org/10.1080/10255840601160484

  3. Arendt PD, Apley DW, Chen W (2012) Quantification of model uncertainty: calibration, model discrepancy, and identifiability. J Mech Des 134:100908

  4. Armstrong M (1984) Problems with universal kriging. J Int Assoc Math Geol 16:101–108

  5. Bayarri MJ, Berger JO, Paulo R, Sacks J, Cafeo JA, Cavendish J, Lin CH, Tu J (2007) A framework for validation of computer models. Technometrics 49:138–154

  6. Buljak V, Pandey S (2015) Material model calibration through indentation test and stochastic inverse analysis. arXiv Preprint arXiv:150703487

  7. Campbell K (2006) Statistical calibration of computer simulations. Reliab Eng Syst Saf 91:1358–1363

  8. Campolongo F, Cariboni J, Saltelli A (2007) An effective screening design for sensitivity analysis of large models. Environ Model Softw 22:1509–1518

  9. Cho H, Bae S, Choi K, Lamb D, Yang R-J (2014) An efficient variable screening method for effective surrogate models for reliability-based design optimization. Struct Multidiscip Optim 50:717–738

  10. Clarke SM, Griebsch JH, Simpson TW (2005) Analysis of support vector regression for approximation of complex engineering analyses. J Mech Des 127:1077–1087

  11. Deng J, Xie L, Chen L, Khatibisepehr S, Huang B, Xu F, Espejo A (2013) Development and industrial application of soft sensors with on-line Bayesian model updating strategy. J Process Control 23:317–325

  12. Dieter GE (1991) Engineering design: a materials and processing approach vol 2. McGraw-Hill, New York

  13. Fang S-E, Ren W-X, Perera R (2012) A stochastic model updating method for parameter variability quantification based on response surface models and Monte Carlo simulation. Mech Syst Signal Process 33:83–96

  14. Ferson S, Oberkampf WL, Ginzburg L (2009) Validation of imprecise probability models. Int J Reliab Qual Saf Eng 3:3–22

  15. Forman EH, Gass SI (2001) The analytic hierarchy process—an exposition. Oper Res 49:469–486

  16. Frey HC, Patil SR (2002) Identification and review of sensitivity analysis methods. Risk Anal 22:553–578

  17. Gholizadeh S (2013) Structural optimization for frequency constraints. Metaheuristic applications in structures and infrastructures:389

  18. Hamby D (1994) A review of techniques for parameter sensitivity analysis of environmental models. Environ Monit Assess 32:135–154

  19. Hess PE, Bruchman D, Assakkaf IA, Ayyub BM (2002) Uncertainties in material and geometric strength and load variables. Nav Eng J 114:139–166

  20. Hu Z, Mahadevan S (2016) Global sensitivity analysis-enhanced surrogate (GSAS) modeling for reliability analysis. Struct Multidiscip Optim 53:501–521

  21. Hurtado J, Barbat AH (1998) Monte Carlo techniques in computational stochastic mechanics. Arch Comput Methods Eng 5:3

  22. Iooss B, Lemaître P (2015) A review on global sensitivity analysis methods. In: Uncertainty management in simulation-optimization of complex systems. Springer, Boston, MA, pp 101–122

  23. Jung BC, Yoon H, Oh H, Lee G, Yoo M, Youn BD, Huh YC (2016) Hierarchical model calibration for designing piezoelectric energy harvester in the presence of variability in material properties and geometry. Struct Multidiscip Optim 53:161–173

  24. Kang Y-J, Hong J, Lim O, Noh Y (2017) Reliability analysis using parametric and nonparametric input modeling methods. J Comput Struct Eng Inst Korea 30:87–94

  25. Kang Y-J, Noh Y, Lim O-K (2018) Kernel density estimation with bounded data. Struct Multidiscip Optim 57:95–113

  26. Kang K, Qin C, Lee B, Lee I (2019a) Modified screening-based Kriging method with cross validation and application to engineering design. Appl Math Model 70:626–642

  27. Kang Y-J, Noh Y, Lim O-K (2019b) Integrated statistical modeling method: part I—statistical simulations for symmetric distributions. Struct Multidiscip Optim 60(5):1719–1740

  28. Kennedy MC, O'Hagan A (2001) Bayesian calibration of computer models. J R Stat Soc: B (Stat Methodol) 63:425–464

  29. Krige DG (1951) A statistical approach to some basic mine valuation problems on the Witwatersrand. J South Afr Inst Min Metall 52:119–139

  30. Lee I, Choi K, Zhao L (2011) Sampling-based RBDO using the stochastic sensitivity analysis and dynamic Kriging method. Struct Multidiscip Optim 44:299–317

  31. Lee D, Kim NH, Kim H-S (2016) Validation and updating in a large automotive vibro-acoustic model using a P-box in the frequency domain. Struct Multidiscip Optim 54:1485–1508

  32. Lee G, Yi G, Youn BD (2018) Special issue: a comprehensive study on enhanced optimization-based model calibration using gradient information. Struct Multidiscip Optim 57:2005–2025

  33. Lee G, Kim W, Oh H, Youn BD, Kim NH (2019a) Review of statistical model calibration and validation—from the perspective of uncertainty structures. Struct Multidiscip Optim 60:1619–1644

  34. Lee G, Son H, Youn BD (2019b) Sequential optimization and uncertainty propagation method for efficient optimization-based model calibration. Structural and Multidisciplinary Optimization 60:1355–1372

  35. Lee K, Cho H, Lee I (2019c) Variable selection using Gaussian process regression-based metrics for high-dimensional model approximation with limited data. Struct Multidiscip Optim 59:1439–1454

  36. Liu Y, Chen W, Arendt P, Huang H-Z (2011) Toward a better understanding of model validation metrics. J Mech Des 133:071005

  37. Lophaven SN, Nielsen HB, Sondergaard J, Dace A (2002a) A matlab kriging toolbox. Technical University of Denmark, Kongens Lyngby, Technical Report No IMMTR-2002 12

  38. Mahadevan S, Haldar A (2000) Probability, reliability and statistical method in engineering design. John Wiley & Sons, Inc., Hoboken, NJ, USA

  39. Manfren M, Aste N, Moshksar R (2013) Calibration and uncertainty analysis for computer models–a meta-model based approach for integrated building energy simulation. Appl Energy 103:627–641

  40. Marrel A, Iooss B, Van Dorpe F, Volkova E (2008) An efficient methodology for modeling complex computer codes with Gaussian processes. Comput Stat Data Anal 52:4731–4744

  41. Moon M-Y, Choi K, Gaul N, Lamb D (2019a) Treating epistemic uncertainty using bootstrapping selection of input distribution model for confidence-based reliability assessment. J Mech Des 141:031402

  42. Moon M-Y, Choi K, Lamb D (2019b) Target output distribution and distribution of bias for statistical model validation given a limited number of test data. Struct Multidiscip Optim 60:1327–1353

  43. Mousaviraad SM, He W, Diez M, Stern F (2013) Framework for convergence and validation of stochastic uncertainty quantification and relationship to deterministic verification and validation. Int J Uncertain Quantif 3(5):371–395

  44. Oberkampf WL, Roy CJ (2010) Verification and validation in scientific computing. Cambridge University Press, UK

  45. Oberkampf WL, Trucano TG (2002) Verification and validation in computational fluid dynamics. Prog Aerosp Sci 38:209–272

  46. Oh H, Kim J, Son H, Youn BD, Jung BC (2016) A systematic approach for model refinement considering blind and recognized uncertainties in engineered product development. Struct Multidiscip Optim 54:1527–1541

  47. Oh H, Choi H, Jung JH, Youn BD (2019) A robust and convex metric for unconstrained optimization in statistical model calibration—probability residual (PR). Struct Multidiscip Optim 60:1171–1187

  48. Park C, Choi J-H, Haftka RT (2016) Teaching a verification and validation course using simulations and experiments with paper helicopters. J Verific, Valid Uncertainty Quantif 1:031002

  49. Pettit CL (2004) Uncertainty quantification in aeroelasticity: recent results and research challenges. J Aircr 41:1217–1229

  50. Qiu N, Park C, Gao Y, Fang J, Sun G, Kim NH (2018) Sensitivity-based parameter calibration and model validation under model error. J Mech Des 140:011403

  51. Rahman S, Xu H (2004) A univariate dimension-reduction method for multi-dimensional integration in stochastic mechanics. Probab Eng Mech 19:393–408

  52. Rebba R, Mahadevan S, Huang S (2006) Validation and error estimation of computational models. Reliab Eng Syst Saf 91:1390–1397

  53. Roy CJ, Oberkampf WL A complete framework for verification, validation, and uncertainty quantification in scientific computing. In: 48th AIAA Aerospace Sciences Meeting Including the New Horizons Forum and Aerospace Exposition, 2010. pp 4–7

  54. Roy A, Manna R, Chakraborty S (2019) Support vector regression based metamodeling for structural reliability analysis. Probab Eng Mech 55:78–89

  55. Schuëller GI, Jensen HA (2008) Computational methods in optimization considering uncertainties–an overview. Comput Methods Appl Mech Eng 198:2–13

  56. Stein A, Corsten L (1991) Universal kriging and cokriging as a regression procedure. Biometrics:575–587

  57. Total Materia (2019). https://www.totalmateria.com/. Accessed 18 September 2019

  58. Trucano TG, Pilch M, Oberkampf WL (2002) General concepts for experimental validation of ASCI code applications. Sandia National Labs., Albuquerque, NM (US); Sandia National Labs., Livermore, CA (US),

  59. Wang GG (2003) Adaptive response surface method using inherited Latin hypercube design points. J Mech Des 125:210–220

  60. Wang S, Chen W, Tsui K-L (2009) Bayesian validation of computer models. Technometrics 51:439–451

  61. Wang C, Matthies HG, Xu M, Li Y (2018) Epistemic uncertainty-based model validation via interval propagation and parameter calibration. Comput Methods Appl Mech Eng 342:161–176

  62. Wu X, Kozlowski T, Meidani H (2018) Kriging-based inverse uncertainty quantification of nuclear fuel performance code BISON fission gas release model using time series measurement data. Reliab Eng Syst Saf 169:422–436. https://doi.org/10.1016/j.ress.2017.09.029

  63. Xi Z, Fu Y, Yang R (2013) Model bias characterization in the design space under uncertainty. Int J Perform Eng 9:433–444

  64. Xiong Y, Chen W, Tsui K-L, Apley DW (2009) A better understanding of model updating strategies in validating engineering models. Comput Methods Appl Mech Eng 198:1327–1337. https://doi.org/10.1016/j.cma.2008.11.023

  65. Youn BD, Xi Z, Wang P (2008) Eigenvector dimension reduction (EDR) method for sensitivity-free probability analysis. Struct Multidiscip Optim 37:13–28

  66. Youn BD, Jung BC, Xi Z, Kim SB, Lee W (2011) A hierarchical framework for statistical model calibration in engineering product development. Comput Methods Appl Mech Eng 200:1421–1431

  67. Zhao L, Choi K, Lee I (2011) Metamodeling method using dynamic kriging for design optimization. AIAA J 49:2034–2046

  68. Zimmerman D, Pavlik C, Ruggles A, Armstrong MP (1999) An experimental comparison of ordinary and universal kriging and inverse distance weighting. Math Geol 31:375–390

Download references


This work was supported by the R&D project (R17GA08) of Korea Electric Power Corporation (KEPCO) and a grant (17TLRP-C135446-01, Development of Hybrid Electric Vehicle Conversion Kit for Diesel Delivery Trucks and its Commercialization for Parcel Services) from the Transportation & Logistics Research Program (TLRP) funded by the Ministry of Land, Infrastructure and Transport of the Korean government.

Author information

Correspondence to Byeng D. Youn or Ikjin Lee or Yoojeong Noh.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Replication of results

For readers interested in the specific process of statistical model improvement, Section 4 explains the basic principle of overall techniques. Kang’s paper can give comprehensive information about KDE-ebd (Kang et al. 2018). Equation (3) shows the main idea of code implementation for variable screening. The authors used the DACE MATLAB toolbox for universal Kriging (Lophaven et al. 2002). Section 4.1.3 includes the specific code implementation of the SQP optimization for statistical model calibration. Equations (12), (13), and (14) give the area metric with u-pooling for statistical model validation. The experimental data of natural frequency and the finite element model of the automobile steering column geometry is proprietary and protected.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Responsible editor: KK Choi

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Son, H., Lee, G., Kang, K. et al. Industrial issues and solutions to statistical model improvement: a case study of an automobile steering column. Struct Multidisc Optim (2020). https://doi.org/10.1007/s00158-020-02526-2

Download citation


  • Statistical model improvement
  • Uncertainty characterization
  • Uncertainty propagation
  • Model refinement
  • Statistical model calibration
  • Statistical validation
  • Automobile steering column