Structural and Multidisciplinary Optimization

, Volume 48, Issue 1, pp 127–148

Robust design optimization by polynomial dimensional decomposition

Authors

  • Xuchun Ren
    • Department of Mechanical & Industrial EngineeringThe University of Iowa
    • Department of Mechanical & Industrial EngineeringThe University of Iowa
Research Paper

DOI: 10.1007/s00158-013-0883-z

Cite this article as:
Ren, X. & Rahman, S. Struct Multidisc Optim (2013) 48: 127. doi:10.1007/s00158-013-0883-z

Abstract

This paper introduces four new methods for robust design optimization (RDO) of complex engineering systems. The methods involve polynomial dimensional decomposition (PDD) of a high-dimensional stochastic response for statistical moment analysis, a novel integration of PDD and score functions for calculating the second-moment sensitivities with respect to the design variables, and standard gradient-based optimization algorithms. New closed-form formulae are presented for the design sensitivities that are simultaneously determined along with the moments. The methods depend on how statistical moment and sensitivity analyses are dovetailed with an optimization algorithm, encompassing direct, single-step, sequential, and multi-point single-step design processes. Numerical results indicate that the proposed methods provide accurate and computationally efficient optimal solutions of RDO problems, including an industrial-scale lever arm design.

Keywords

Design under uncertaintyANOVA dimensional decompositionOrthogonal polynomialsScore functionsOptimization

1 Introduction

Robust design optimization (RDO) constitutes a mathematical framework for solving design problems in the presence of uncertainty, manifested by statistical descriptions of the objective and/or constraint functions (Taguchi 1993; Chen et al. 1996; Du and Chen 2000; Mourelatos and Liang 2006; Zaman et al. 2011; Park et al. 2006). Aimed at improving product quality, it minimizes the propagation of input uncertainty to output responses of interest, leading to an insensitive design. RDO, pioneered by Taguchi (1993), is being increasingly viewed as an enabling technology for design of aerospace, civil, and automotive structures subject to uncertainty (Chen et al. 1996; Du and Chen 2000; Mourelatos and Liang 2006; Zaman et al. 2011; Park et al. 2006).

The objective or constraint functions in RDO often involve second-moment properties, such as means and standard deviations, of stochastic responses, describing the statistical performance of a given design. Therefore, solving an RDO problem draws in uncertainty quantification of random responses and its coupling with gradient-based optimization algorithms, consequently demanding a greater computational effort than that required by a deterministic design optimization. There exist three principal concerns or shortcomings when conducting RDO with existing approaches or techniques. First, the commonly used stochastic methods, including the Taylor series or perturbation expansions (Huang and Du 2007), point estimate method (Huang and Du 2007), polynomial chaos expansion (PCE) (Wang and Kim 2006), tensor-product quadrature rule (Lee et al. 2009), and dimension-reduction methods (Lee et al. 2008, 2009) may not be adequate or applicable for uncertainty quantification of many large-scale practical problems. For instance, the Taylor series expansion and point estimate methods, although simple and inexpensive, begin to break down when the input-output mapping is highly nonlinear or the input uncertainty is arbitrarily large. Furthermore, truly high-dimensional problems are all but impossible to solve using the PCE and tensor-product quadrature rule due to the curse of dimensionality. The dimension-reduction methods, developed by the author’s group (Rahman and Xu 2004; Xu and Rahman 2004), including a modification (Youn et al. 2008), alleviate the curse of dimensionality to some extent, but they are rooted in the referential dimensional decomposition, resulting in sub-optimal approximations of a multivariate function (Rahman 2011, 2012). Second, many of the aforementioned methods invoke finite-difference techniques to calculate design sensitivities of the statistical moments. They demand repeated stochastic analyses for nominal and perturbed values of design parameters and are, therefore, expensive and unwieldy. Although some methods, such as Taylor series expansions, also provide the design sensitivities economically, the sensitivities are either inaccurate or unreliable because they inherit errors from the affiliated second-moment analysis. Therefore, alternative stochastic methods should be explored for calculating the statistical moments and design sensitivities as accurately as possible and simultaneously, but without the computational burden of crude Monte Carlo simulation (MCS). Third, existing methods for solving RDO problems permit the objective and constraint functions and their sensitivities to be calculated only at a fixed design, requiring new statistical moment and sensitivity analyses at every design iteration until convergence is attained. Consequently, the current RDO methods, entailing expensive finite-element analysis (FEA) or similar numerical calculations, are computationally intensive, if not prohibitive, when confronted with a large number of design or random variables. New or significantly improved design paradigms, possibly requiring a single or a few stochastic simulations, are needed for solving the entire RDO problem. Further complications may arise when an RDO problem is formulated in conjunction with a multi-point approximation (Toropov et al. 1993)—a setting frequently encountered when tackling a practical optimization problem with a large design space. In which case, one must integrate stochastic analysis, design sensitivity analysis, and optimization algorithms on a local subregion of the entire design space.

This paper presents four new methods for robust design optimization of complex engineering systems. The methods are based on: (1) polynomial dimensional decomposition (PDD) of a high-dimensional stochastic response for statistical moment analysis; (2) a novel integration of PDD and score functions for calculating the second-moment sensitivities with respect to design variables; and (3) standard gradient-based optimization algorithms, encompassing direct, single-step, sequential, and multi-point single-step design processes. Section 2 formally defines a general RDO problem, including a concomitant mathematical statement. Section 3 starts with a brief exposition of the analysis-of-variance (ANOVA) dimensional decomposition and explains how it leads up to PDD approximations, resulting in explicit formulae for the first two moments of a generic stochastic response. The calculation of the expansion coefficients by dimension-reduction integration is also briefly described. Section 4 defines score functions and unveils new closed-form formulae for the design sensitivities of the first two moments, determined from a single stochastic analysis. Section 5 introduces four new design methods and explains how the stochastic analysis and design sensitivities from a PDD approximation are integrated with a gradient-based optimization algorithm in each method. Section 6 presents four numerical examples involving mathematical functions or solid-mechanics problems, contrasting the accuracy, convergence properties, and computational efforts of the proposed RDO methods. It is followed by Section 7, which discusses the efficiency and applicability of all four methods. Finally, the conclusions are drawn in Section 8.

2 Robust design optimization

Let \(\mathbb {N}\), \(\mathbb {N}_{0}\), \(\mathbb {R}\), and \(\mathbb {R}_{0}^{+}\) represent the sets of positive integer (natural), non-negative integer, real, and non-negative real numbers, respectively. For \(k\in \mathbb {N}\), denote by \(\mathbb {R}^{k}\) the k-dimensional Euclidean space and by \(\mathbb {N}_{0}^{k}\) the k-dimensional multi-index space. These standard notations will be used throughout the paper.

Consider a measurable space \((\Omega ,\mathcal {F})\), where \(\Omega \) is a sample space and \(\mathcal {F}\) is a \(\sigma \)-field on \(\Omega \). Defined over \((\Omega ,\mathcal {F})\), let \(\{P_{\mathbf {d}}:\mathcal {F}\to [0,1]\}\) be a family of probability measures, where for \(M\in \mathbb {N}\) and \(N\in \mathbb {N}\), \(\mathbf {d}=(d_{1},\ldots ,d_{M})\in \mathcal {D}\) is an \(\mathbb {R}^{M}\)-valued design vector with non-empty closed set \(\mathcal {D}\subseteq \mathbb {R}^{M}\) and \(\mathbf {X}:=(X_{1},\ldots ,X_{N}):(\Omega ,\mathcal {F})\to (\mathbb {R}^{N},\mathcal {B}^{N})\) be an \(\mathbb {R}^{N}\)-valued input random vector with \(\mathcal {B}^{N}\) representing the Borel \(\sigma \)-field on \(\mathbb {R}^{N}\), describing the statistical uncertainties in loads, material properties, and geometry of a complex mechanical system. The probability law of \(\mathbf {X}\) is completely defined by a family of the joint probability density functions \(\{f_{\mathbf {X}}(\mathbf {x};\mathbf {d}),\:\mathbf {x}\in \mathbb {R}^{N},\:\mathbf {d}\in \mathcal {D}\}\) that are associated with probability measures \(\{P_{\mathbf {d}},\:\mathbf {d}\in \mathcal {D}\}\), so that the probability triple \((\Omega ,\mathcal {F},P_{\mathbf {d}})\) of \(\mathbf {X}\) depends on \(\mathbf {d}\). A design variable \(d_{k}\) can be any distribution parameter or a statistic–for instance, the mean or standard deviation–of \(X_{i}\).

Let \(y_{l}(\mathbf {X})\), \(l=0,1,2,\ldots ,K\), be a collection of \(K+1\) real-valued, square-integrable, measurable transformations on \((\Omega ,\mathcal {F})\), describing relevant geometry (e.g., length, area, volume, mass) and performance functions of a complex system. It is assumed that \(y_{l}:(\mathbb {R}^{N},\mathcal {B}^{N})\to (\mathbb {R},\mathcal {B})\) is not an explicit function of \(\mathbf {d}\), although \(y_{l}\) implicitly depends on \(\mathbf {d}\) via the probability law of \(\mathbf {X}\). This is not a major limitation, as most RDO problems involve means and/or standard deviations of random variables as design variables. Nonetheless, the mathematical formulation of a general RDO problem involving an objective function \(c_{0}:\mathbb {R}^{M}\to \mathbb {R}\) and constraint functions \(c_{l}:\mathbb {R}^{M}\to \mathbb {R}\), where \(l=1,\ldots ,K\) and \(1\le K<\infty \), requires one to
$$ \begin{array}{rll} \min_{\mathbf{d}\in\mathcal{D}\subseteq\mathbb{R}^{M}} && c_{0}(\mathbf{d}):=g_{0}\left({ \mathbb{E}_{\mathbf{d}}}\left[y_{0}(\mathbf{X})\right],\mathrm{var}_{\mathbf{d}}\left[y_{0}(\mathbf{X})\right]\right),\\ \mathrm{subject\: to\;\;\;} && c_{l}(\mathbf{d}):=g_{l}\left({ \mathbb{E}_{\mathbf{d}}}\left[y_{l}(\mathbf{X})\right],\mathrm{var}_{\mathbf{d}}\left[y_{l}(\mathbf{X})\right]\right)\\ && \le0,\; l=1,\ldots,K,\\ && d_{k,L}\le d_{k}\le d_{k,U},\: k=1,\ldots,M, \end{array} $$
(1)
where \({ \mathbb {E}_{\mathbf {d}}}[y_{l}(\mathbf {X})]:=\int _{\mathbb {R}^{N}}y_{l}(\mathbf {x})f_{\mathbf {X}}(\mathbf {x};\mathbf {d})d\mathbf {x}\) is the mean of \(y_{l}(\mathbf {X})\) with \({\mathbb {E}_{\mathbf {d}}}\) denoting the expectation operator with respect to the probability measure \(P_{\mathbf {d}},\mathbf {d}\in \mathbb {R}^{M}\), \(\mathrm {var}_{\mathbf {d}}[y_{l}(\mathbf {X})]:={ \mathbb {E}_{\mathbf {d}}}[\{y_{l}(\mathbf {X})-{\mathbb {E}_{\mathbf {d}}}[y_{l}(\mathbf {X})]\}^{2}]\) is the variance of \(y_{l}(\mathbf {X})\), and \(g_{l}\), \(l=0,1,\ldots ,K\), are arbitrary functions of \({ \mathbb {E}_{\mathbf {d}}}[y_{l}(\mathbf {X})]\) and \(\mathrm {var}_{\mathbf {d}}[y_{l}(\mathbf {X})]\). However, in most applications (Huang and Du 2007; Wang and Kim 2006; Lee et al. 2008, 2009), the functions \(g_{l}\) are prescribed as linear transformations of the mean and standard deviation of \(y_{l}\), leading one to
$$ \begin{array}{rll} \min_{\mathbf{d}\in\mathcal{D}\subseteq\mathbb{R}^{M}}\;\; & c_{0}(\mathbf{d}):=w_{1}\frac{{ \mathbb{E}_{\mathbf{d}}}\left[y_{0}(\mathbf{X})\right]}{\mu_{0}^{\ast}}\\ & \qquad\qquad+w_{2}\frac{\sqrt{\mathrm{var}_{\mathbf{d}}\left[y_{0}(\mathbf{X})\right]}}{\sigma_{0}^{\ast}},\\ \mathrm{subject\: to\;\;\;} & c_{l}(\mathbf{d}):=\alpha_{l}\sqrt{\mathrm{\mathrm{var}_{\mathbf{d}}}\left[y_{l}(\mathbf{X})\right]}-{ \mathbb{E}_{\mathbf{d}}}\left[y_{l}(\mathbf{X})\right]\\ & \le 0,\; l=1,\ldots,K,\\ & d_{k,L}\le d_{k}\le d_{k,U},\: k=1,\ldots,M, \end{array} $$
(2)
where \(w_{1}\in \mathbb {R}_{0}^{+}\) and \(w_{2}\in \mathbb {R}_{0}^{+}\) are two non-negative, real-valued weights, satisfying \(w_{1}+w_{2}=1\), \(\mu _{0}^{\ast }\in \mathbb {R}\setminus \{0\}\) and \(\sigma _{0}^{\ast }\in \mathbb {R}_{0}^{+}\setminus \{0\}\) are two non-zero, real-valued scaling factors; \(\alpha _{l}\in \mathbb {R}_{0}^{+}\), \(l=0,1,\ldots ,K\), are non-negative, real-valued constants associated with the probabilities of constraint satisfaction; and \(d_{k,L}\) and \(d_{k,U}\) are the lower and upper bounds, respectively, of \(d_{k}\). Other formulations entailing nonlinear functions of the first two or higher-order moments can be envisioned, but they are easily tackled by the proposed methods. Nonetheless, the focus of this work is solving the RDO problem described by (2) for arbitrary functions \(y_{l}\), \(l=0,1,2,\ldots ,K\), and arbitrary probability distributions of \(\mathbf {X}\).

3 Statistical moment analysis

Let \(y(\mathbf {X}):=y(X_{1},\ldots ,X_{N}\)) represent any one of the random functions \(y_{l}\), \(l=0,1,\ldots ,K\), introduced in Section 2, and let \(\mathcal {L}_{2}(\Omega ,\mathcal {F},P_{\mathbf {d}})\) represent a Hilbert space of square-integrable functions, including y, with respect to the probability measure \(f_{\mathbf {X}}(\mathbf {x};\mathbf {d})d\mathbf {x}\) supported on \(\mathbb {R}^{N}\). Assuming independent coordinates of \(\mathbf {X}\), its joint probability density is expressed by a product, \(f_{\mathbf {\mathbf {X}}}(\mathbf {x};\mathbf {d})=\prod _{i=1}^{i=N}f_{X_{i}}(x_{i};\mathbf {d})\), of marginal probability density functions \(f_{X_{i}}:\mathbb {R}\to \mathbb {R}_{0}^{+}\) of \(X_{i}\), \(i=1,\ldots ,N\), defined on its probability triple \((\Omega _{i},\mathcal {F}_{i},P_{i,\mathbf {d}})\) with a bounded or an unbounded support on \(\mathbb {R}\). Then, for a given subset \(u\subseteq \{1,\ldots ,N\}\), \(f_{\mathbf {X}_{-u}}(\mathbf {x}_{-u};\mathbf {d}):=\prod _{i=1,i\notin u}^{N}f_{i}(x_{i};\mathbf {d})\) defines the marginal density function of \(\mathbf {X}_{-u}:=\mathbf {X}_{\{1,\ldots ,N\}\backslash u}\).

3.1 ANOVA dimensional decomposition

The ANOVA dimensional decomposition, expressed by the recursive form (Efron and Stein 1981; Sobol 2003; Rahman 2012)
$$ y(\mathbf{X}) ={\sum\limits_{u\subseteq\{1,\ldots,N\}}y_{u}(\mathbf{X}_{u};\mathbf{d})}, $$
(3)
$$ y_{\emptyset}(\mathbf{d}) =\int_{\mathbb{R}^{N}}y(\mathbf{x})f_{\mathbf{X}}(\mathbf{x};\mathbf{d})d\mathbf{x},$$
(4)
$$ \begin{array}{rll} y_{u}(\mathbf{X}_{u};\mathbf{d}) & =& { \int_{\mathbb{R}^{N-|u|}}y(\mathbf{X}_{u},\mathbf{x}_{-u})}f_{\mathbf{X}_{-u}}(\mathbf{x}_{-u};\mathbf{d})d\mathbf{\mathbf{x}}_{-u}\\ && -{\sum\limits_{v\subset u}}y_{v}(\mathbf{X}_{v};\mathbf{d}), \end{array} $$
(5)
is a finite, hierarchical expansion of y in terms of its input variables with increasing dimensions, where \(u\subseteq \{1,\ldots ,N\}\) is a subset with the complementary set \(-u=\{1,\ldots ,N\}\backslash u\) and cardinality \(0\le |u|\le N\), and \(y_{u}\) is a \(|u|\)-variate component function describing a constant or the interactive effect of \(\mathbf {X}_{u}=(X_{i_{1}},\ldots ,X_{i_{|u|}})\), \(1\leq i_{1}<\ldots <i_{|u|}\leq N\), a subvector of \(\mathbf {X}\), on y when \(|u|=0\) or \(|u|>0\). The summation in (3) comprises \(2^{N}\) terms, with each term depending on a group of variables indexed by a particular subset of \(\{1,\ldots ,N\}\), including the empty set \(\emptyset \). In (5), \((\mathbf {X}_{u},\mathbf {x}_{-u})\) denotes an N-dimensional vector whose ith component is \(X_{i}\) if \(i\in u\) and \(x_{i}\) if \(i\notin u\). When \(u=\emptyset \), the sum in (5) vanishes, resulting in the expression of the constant function \(y_{\emptyset }\) in (4). When \(u=\{1,\ldots ,N\}\), the integration in (5) is on the empty set, reproducing (3) and hence finding the last function \(y_{\{1,\ldots ,N\}}\). Indeed, all component functions of y can be obtained by interpreting literally (5).

Remark 1

The ANOVA component functions \(y_{u}\), \(\emptyset \ne u\subseteq \{1,\ldots ,N\}\), are uniquely determined from the annihilating conditions (Sobol 2003; Rahman 2012),
$$ \int_{\mathbb{R}}y_{u}(\mathbf{x}_{u};\mathbf{d})f_{X_{i}}(x_{i};\mathbf{d})dx_{i}=0\;\mathrm{for}\; i\in u, $$
(6)
resulting in two remarkable properties: (1) the component functions, \(y_{u}\), \(\emptyset \ne u\subseteq \{1,\ldots ,N\}\), have zero means; and (2) any two distinct component functions \(y_{u}\) and \(y_{v}\), where \(u\subseteq \{1,\ldots ,N\}\), \(v\subseteq \{1,\ldots ,N\}\), and \(u\neq v\), are orthogonal. Further details are available elsewhere (Rahman 2012).

Remark 2

The coefficient \(y_{\emptyset }=\mathbb {E}_{\mathbf {d}}[y(\mathbf {X})]\) in (4) is a function of the design vector \(\mathbf {d}\), which describes the probability distribution of the random vector \(\mathbf {X}\). Therefore, the adjective “constant” used to describe \(y_{\emptyset }\) should be interpreted with respect to \(\mathbf {X}\), not \(\mathbf {d}\). A similar condition applies for the component functions \(y_{u}\), \(\emptyset \ne u\subseteq \{1,\ldots ,N\}\), which also depend on \(\mathbf {d}\).

3.2 Polynomial dimensional decomposition

3.2.1 Orthonormal polynomials

Let \(\{\psi _{ij}(x_{i};\mathbf {d});\; j=0,1,\ldots \}\) be a set of univariate, orthonormal polynomial basis functions in the Hilbert space \(\mathcal {L}_{2}(\Omega _{i},\mathcal {F}_{i},P_{i,\mathbf {d}})\) that is consistent with the probability measure \(P_{i,\mathbf {d}}\) or \(f_{X_{i}}(x_{i};\mathbf {d})dx_{i}\) of \(X_{i}\) for a given design \(\mathbf {d}\). For \(\emptyset \ne u=\{i_{1},\ldots ,i_{|u|}\}\subseteq \{1,\ldots ,N\}\), where 1 \(\le \)\(|u|\)\(\le \)N and 1 \(\le \)\(i_{1}\)\(<\)\(\ldots \)\(<\)\(i_{|u|}\)\(\le \)N, let \((\times _{p=1}^{p=|u|}\Omega _{i_{p}}\), \(\times _{p=1}^{p=|u|}\mathcal {F}_{i_{p}}\), \(\times _{p=1}^{p=|u|}P_{i_{p}, \mathbf {d}})\) be the product probability triple of \(\mathbf {X}_{u}=(X_{i_{1}}\), \(\ldots \), \(X_{i_{|u|}})\). Denote the associated space of the \(|u|\)-variate component functions of y by \(\mathcal {L}_{2}(\times _{p=1}^{p=|u|}\Omega _{i_{p}}\), \(\times _{p=1}^{p=|u|}\mathcal {F}_{i_{p}}\), \(\times _{p=1}^{p=|u|}P_{i_{p}, \mathbf {d}})\)\(:=\)\(\{y_{u}:\int _{\mathbb {R}^{|u|}}y_{u}^{2}\)\((\mathbf {x}_{u};\mathbf {d})f_{\mathbf {X}_{u}}(\mathbf {x}_{u};\mathbf {d})d\mathbf {x}_{u}<\infty \}\), which is a Hilbert space. Since the joint density of \(\mathbf {X}_{u}\) is separable (independence of \(X_{i},i\in u\)), that is, \(f_{\mathbf {X}_{u}}(\mathbf {x}_{u};\mathbf {d})={\textstyle \prod _{p=1}^{|u|}}f_{X_{i_{p}}}(x_{i_{p}};\mathbf {d})dx_{i_{p}}\), the product \(\psi _{u\mathbf {j}_{|u|}}(\mathbf {X}_{u};\mathbf {d}):=\prod _{p=1}^{|u|}\psi _{i_{p}j_{p}}(X_{i_{p}};\mathbf {d})\), where \(\mathbf {j}_{|u|}=\)\((j_{1},\ldots ,j_{|u|})\in \mathbb {N}_{0}^{|u|}\), a \(|u|\)-dimensional multi-index with \(\infty \)-norm \(||\mathbf {j}_{|u|}||_{\infty }=\max (j_{1},\ldots ,j_{|u|}\)), constitutes a multivariate orthonormal polynomial basis in \(\mathcal {L}_{2}(\times _{p=1}^{p=|u|}\Omega _{i_{p}}\), \(\times _{p=1}^{p=|u|}\mathcal {F}_{i_{p}}\), \(\times _{p=1}^{p=|u|}P_{i_{p},\mathbf {d}})\). Two important properties of these product polynomials from tensor products of Hilbert spaces are as follows.

Proposition 1

The product polynomials \(\psi _{u\mathbf {j}_{|u|}}(\mathbf {X}_{u};\mathbf {d})\), \(\emptyset \ne u\subseteq \{1,\ldots ,N\}\), \(j_{1},\ldots ,j_{|u|}\ne 0\), \(\mathbf {d}\in \mathcal {D}\), have zero means, i.e.,
$$ \mathbb{E}_{\mathbf{d}}\left[\psi_{u\mathbf{j}_{|u|}}(\mathbf{X}_{u};\mathbf{d})\right]=0. $$
(7)

Proposition 2

Any two distinct product polynomials \(\psi _{u\mathbf {j}_{|u|}}(\mathbf {X}_{u};\mathbf {d})\) and \(\psi _{v\mathbf {k}_{|v|}}(\mathbf {X}_{v};\mathbf {d})\) for \(\mathbf {d}\in \mathcal {D}\), where \(\emptyset \ne u\subseteq \{1,\ldots ,N\}\), \(\emptyset \ne v\subseteq \{1,\ldots ,N\}\), \(j_{1},\ldots ,j_{|u|}\ne 0\), \(k_{1},\ldots ,k_{|v|}\ne 0\), are uncorrelated and each has unit variance, i.e.,
$$ \mathbb{E}_{\mathbf{d}}\left[\psi_{u\mathbf{j}_{|u|}}(\mathbf{X}_{u};\mathbf{d})\psi_{v\mathbf{k}_{|v|}}(\mathbf{X}_{u};\mathbf{d})\right]=\left\{ \begin{array}{ll} 1 & \mathrm{if}\; u=v; \mathbf{j}_{|u|}=\mathbf{k}_{|v|},\\ 0 & \mathrm{otherwise.} \end{array}\right. $$
(8)

Proof

The results of Propositions 1 and 2 follow by recognizing independent coordinates of \(\mathbf {X}\) and using the second-moment properties of univariate orthonormal polynomials: (1) \(\mathbb {E}_{\mathbf {d}}[\psi _{ij}(X_{i};\mathbf {d})]=1\) when \(j=0\) and zero when \(j\ge 1\); and (2) \(\mathbb {E}_{\mathbf {d}}[\psi _{ij_{1}}(X_{i};\mathbf {d})\psi _{ij_{2}}(X_{i};\mathbf {d})]=1\) when \(j_{1}=j_{2}\) and zero when \(j_{1}\neq j_{2}\) for an arbitrary random variable \(X_{i}\). □

Remark 3

Given a probability measure \(P_{i,\mathbf {d}}\) of any random variable \(X_{i}\), the well-known three-term recurrence relation is commonly used to construct the associated orthogonal polynomials (Rahman 2009a; Gautschi 2004). For \(m\in \mathbb {N}\), the first m recursion coefficient pairs are uniquely determined by the first \(2m\) moments of \(X_{i}\) that must exist. When these moments are exactly calculated, they lead to exact recursion coefficients, some of which belong to classical orthogonal polynomials. For an arbitrary probability measure, approximate methods, such as the Stieltjes procedure, can be employed to obtain the recursion coefficients (Rahman 2009a; Gautschi 2004).

3.2.2 Stochastic expansion

The orthogonal polynomial expansion of a non-constant \(|u|\)-variate component function in (5) becomes (Rahman 2008; 2009a)
$$\begin{array}{*{20}l} y_{u}(\mathbf{X}_{u};\mathbf{d}) & =\sum\limits_{{\textstyle {\mathbf{j}_{|u|}\in\mathbb{N}_{0}^{|u|}\atop j_{1},\ldots,j_{|u|}\neq0}}}C_{u\mathbf{j}_{|u|}}(\mathbf{d})\psi_{u\mathbf{j}_{|u|}}(\mathbf{X}_{u};\mathbf{d}) \end{array} $$
(9)
for any \(\emptyset \ne u\subseteq \{1,\ldots ,N\}\) with
$$ C_{u\mathbf{j}_{|u|}}(\mathbf{d}):=\int_{\mathbb{R}^{N}}y(\mathbf{x})\psi_{u\mathbf{j}_{|u|}}(\mathbf{\mathbf{x}}_{u};\mathbf{d})f_{\mathbf{X}}(\mathbf{x};\mathbf{d})d\mathbf{x} $$
(10)
representing the corresponding expansion coefficient. Similar to \(y_{\emptyset }\), the coefficient \(C_{u\mathbf {j}_{|u|}}\) also depends on the design vector \(\mathbf {d}\). When \(u=\{i\}\), \(i=1,\ldots ,N\), the univariate component functions and expansion coefficients are \(y_{\{i\}}(X_{i};\)\(\mathbf {d})=\sum _{j=1}^{\infty }C_{ij}(\mathbf {d})\psi _{ij}(X_{i};\mathbf {d})\) and \(C_{ij}(\mathbf {d}):=C_{\{i\}(j)}(\mathbf {d})\), respectively. When \(u=\{i_{1},i_{2}\}\), \(i_{1}=1,\ldots ,N-1\), \(i_{2}=i_{1}+1,\ldots ,N\), the bivariate component functions and expansion coefficients are \(y_{\{i_{1},i_{2}\}}(X_{i_{1}},X_{i_{2}};\mathbf {d})=\sum _{j_{1}=1}^{\infty }\sum _{j_{2}=1}^{\infty }C_{i_{1}i_{2}j_{1}j_{2}}(\mathbf {d})\psi _{i_{1}j_{1}}(X_{i_{1}};\mathbf {d})\psi _{i_{2}j_{2}}(X_{i_{2}};\mathbf {d})\) and \(C_{i_{1}i_{2}j_{1}j_{2}}(\mathbf {d}):=C_{\{i_{1},i_{2}\}(j_{1},j_{2})}(\mathbf {d})\), respectively, and so on. Using Propositions 1 and 2, all component functions \(y_{u}\), \(\emptyset \ne u\subseteq \{1,\ldots ,N\}\), are found to satisfy the annihilating conditions of the ANOVA dimensional decomposition. The end result of combining (3)–(5) and (9) is the PDD (Rahman 2008, 2009a),
$$ y(\mathbf{X})=y_{\emptyset}(\mathbf{d})+{\sum\limits_{\emptyset\ne u\subseteq\{1,\ldots,N\}}}\sum\limits_{{\textstyle {\mathbf{j}_{|u|}\in\mathbb{N}_{0}^{|u|}\atop j_{1},\ldots,j_{|u|}\neq0}}} C_{u\mathbf{j}_{|u|}}(\mathbf{d})\psi_{u\mathbf{j}_{|u|}}(\mathbf{X}_{u};\mathbf{d}), $$
(11)
providing an exact, hierarchical expansion of y in terms of an infinite number of coefficients or orthonormal polynomials. In practice, the number of coefficients or polynomials must be finite, say, by retaining at most mth-order polynomials in each variable. Furthermore, in many applications, the function y can be approximated by a sum of at most S-variate component functions, where \(S\in \mathbb {N};1\le S\le N\), resulting in the S-variate, mth-order PDD approximation
$$\begin{array}{rll} \tilde{y}_{S,m}(\mathbf{X}) &=& y_{\emptyset}(\mathbf{d})+{\sum\limits_{{\textstyle {\emptyset\ne u\subseteq\{1,\ldots,N\}\atop 1\le|u|\le S}}}}\sum\limits_{{\textstyle {\mathbf{j}_{|u|}\in\mathbb{N}_{0}^{|u|},||\mathbf{j}_{|u|}||_{\infty}\le m\atop j_{1},\ldots,j_{|u|}\neq0}}} \\ && \,C_{u\mathbf{j}_{|u|}}(\mathbf{d})\psi_{u\mathbf{j}_{|u|}}(\mathbf{X}_{u};\mathbf{d}), \end{array} $$
(12)
containing \(\sum\limits_{k=0}^{S}\binom {N}{S-k}m^{S-k}=\sum\limits_{k=0}^{S}\binom {N}{k}m^{k}\) number of PDD coefficients and corresponding orthonormal polynomials. Due to its additive structure, the approximation in (12) includes degrees of interaction among at most S input variables \(X_{i_{1}},\ldots ,X_{i_{S}}\), \(1\le i_{1}\le \ldots \le i_{S}\le N\). For instance, by selecting \(S=1\) and 2, the functions
$$ \tilde{y}_{1,m}(\mathbf{\mathbf{X}})=y_{\emptyset}+{\sum\limits_{i=1}^{N}}{ \sum\limits_{j=1}^{m}}C_{ij}(\mathbf{d})\psi_{ij}(X_{i};\mathbf{d})\; $$
(13)
and
$$\begin{array}{rll} \tilde{y}_{2,m}(\mathbf{X})&=& y_{\emptyset}(\mathbf{d})+{\sum\limits_{i=1}^{N}}{ \sum\limits_{j=1}^{m}}C_{ij}(\mathbf{d})\psi_{ij}(X_{i};\mathbf{d})\\ &&+\,{\sum\limits_{i_{1}=1}^{N-1}}\;{\sum\limits_{i_{2}=i_{1}+1}^{N}}{\sum\limits_{j_{1}=1}^{m}}{ \sum\limits_{j_{2}=1}^{m}}C_{i_{1}i_{2}j_{1}j_{2}}(\mathbf{d}) \\ &&\times \,\psi_{i_{1}j_{1}}(X_{i_{1}};\mathbf{d})\psi_{i_{2}j_{2}}(X_{i_{2}};\mathbf{d}), \end{array} $$
(14)
respectively, provide univariate and bivariate mth-order PDD approximations, contain contributions from all input variables, and should not be viewed as first- and second-order approximations, nor as limiting the nonlinearity of y. Depending on how the component functions are constructed, arbitrarily high-order univariate and bivariate terms of y could be lurking inside \(\tilde {y}_{1,m}\) and \(\tilde {y}_{2,m}\). When \(S\to N\) and \(m\to \infty \), \(\tilde {y}_{S,m}\) converges to y in the mean-square sense, permitting (12) to generate a hierarchical and convergent sequence of approximations of y. Readers interested in further details of PDD are referred to the authors’ past works (Rahman 2008, 2009a).

3.3 Statistical moments

Applying the expectation operator on \(\tilde {y}_{S,m}(\mathbf {X})\) and noting Proposition 1, the mean (Rahman 2010)
$$ \mathbb{E}_{\mathbf{d}}\left[\tilde{y}_{S,m}(\mathbf{X})\right]=y_{\emptyset}(\mathbf{d}) $$
(15)
of the S-variate, mth-order PDD approximation matches the exact mean \(\mathbb {E}_{\mathbf {d}}\left [y(\mathbf {X})\right ]=y_{\emptyset }(\mathbf {d})\), regardless of S or m. Applying the expectation operator again, this time on \([\tilde {y}_{S,m}(\mathbf {X})-y_{\emptyset }(\mathbf {d})]^{2}\), and recognizing Propositions 1 and 2, results in the approximate variance (Rahman 2010)
$$\begin{array}{rcl} \mathrm{var}_{\mathbf{d}}\left[\tilde{y}_{S,m}(\mathbf{X})\right]&:=&\mathbb{E}_{\mathbf{d}}\left[\left(\tilde{y}_{S,m}(\mathbf{X})-\mathbb{E}\left[\tilde{y}_{S,m}(\mathbf{X})\right]\right)^{2}\right] \\ &=&{\sum\limits_{{\textstyle {\emptyset\ne u\subseteq\{1,\ldots,N\}\atop 1\le|u|\le S}}}}\sum\limits_{{\textstyle {\mathbf{j}_{|u|}\in\mathbb{N}_{0}^{|u|},||\mathbf{j}_{|u|}||_{\infty}\le m\atop j_{1},\ldots,j_{|u|}\neq0}}} C_{u\mathbf{j}_{|u|}}^{2}(\mathbf{d}), \\ \end{array} $$
(16)
calculated as the sum of squares of the expansion coefficients from the S-variate, mth-order PDD approximation of \(y(\mathbf {X})\). Clearly, the approximate variance in (16) approaches the exact variance
$$\begin{array}{rcl} \mathrm{var}_{\mathbf{d}}[y(\mathbf{X})]&:=&\mathbb{E}_{\mathbf{d}}\left[\left(y(\mathbf{X})-\mathbb{E}\left[y(\mathbf{X})\right]\right)^{2}\right] \\ &=&{\sum\limits_{\emptyset\ne u\subseteq\{1,\ldots,N\}}}\sum\limits_{{\textstyle {\mathbf{j}_{|u|}\in\mathbb{N}_{0}^{|u|}\atop j_{1},\ldots,j_{|u|}\neq0}}} C_{u\mathbf{j}_{|u|}}^{2}(\mathbf{d}) \end{array} $$
(17)
of y when \(S\to N\) and \(m\to \infty \). The mean-square convergence of \(\tilde {y}_{S,m}\) is guaranteed as y and its component functions are all members of the associated Hilbert spaces.
For the two special cases, \(S=1\) and \(S=2\), the univariate and bivariate PDD approximations yield the same exact mean value, \(\tilde {\mu }_{1,m}(\mathbf {d})=\tilde {\mu }_{2,m}(\mathbf {d})=y_{\emptyset }(\mathbf {d})\), as noted in (15). However, the respective variance approximations,
$$ \mathrm{var}_{\mathbf{d}}[\tilde{y}_{1,m}(\mathbf{X})]={\sum\limits_{i=1}^{N}}{ \sum\limits_{j=1}^{m}}C_{ij}^{2}(\mathbf{d})\; $$
(18)
and
$$\begin{array}{rcl} \mathrm{var}_{\mathbf{d}}[\tilde{y}_{2,m}(\mathbf{X})]&=& {\sum\limits_{i=1}^{N}}{ \sum\limits_{j=1}^{m}}C_{ij}^{2}(\mathbf{d}) \\ &&+\,{\sum\limits_{i_{1}=1}^{N-1}}\,{\sum\limits_{i_{2}=i_{1}+1}^{N}}\,{\sum\limits_{j_{2}=1}^{m}}\,{\sum\limits_{j_{1}=1}^{m}}C_{i_{1}i_{2}j_{1}j_{2}}^{2}(\mathbf{d}), \end{array} $$
(19)
differ, depend on m, and progressively improve as S becomes larger. Recent works on error analysis indicate that the second-moment properties obtained from the ANOVA dimensional decomposition, which leads to PDD approximations, are superior to those derived from dimension-reduction methods that are grounded in the referential dimensional decomposition (Rahman 2011, 2012). Therefore, employing PDD for solving RDO problems contributes to development of a new, significant design paradigm.

3.4 Expansion coefficients

The determination of the expansion coefficients \(y_{\emptyset }\) and \(C_{u\mathbf {j}_{|u|}}\) in (4) and (10), respectively, of the stochastic responses involves various N-dimensional integrals over \(\mathbb {R}^{N}\). For large N, a full numerical integration employing an N-dimensional tensor product of a univariate quadrature formula is computationally prohibitive. Instead, a dimension-reduction integration scheme can be applied to estimate the coefficients efficiently.

The dimension-reduction integration, originally developed by Xu and Rahman (2004), entails approximating a high-dimensional integral of interest by finite-sum lower-dimensional integrations. For calculating the expansion coefficients \(y_{\emptyset }\) and \(C_{u\mathbf {j}_{|u|}}\), this is accomplished by replacing the N-variate function y in (4) and (10) with an R-variate truncation, where \(R<N\), of its referential dimensional decomposition at a chosen reference point (Rahman 2011, 2012). The result is a reduced integration scheme, requiring evaluations of at most R-dimensional integrals. The scheme facilitates calculation of the coefficients approaching their exact values as \(R\to N\), and is significantly more efficient than performing one N-dimensional integration, particularly when \(R\ll N\). Hence, the computational effort is significantly lowered using the dimension-reduction integration. When \(R=1\) or 2, the scheme involves one-, or, at most, two-dimensional integrations, respectively. Nonetheless, numerical integration is still required for a general integrand. The Gauss-type quadrature rule was used to perform integrations. The integration points and associated weights, which depend on the probability distribution of \(X_{i}\), are readily available when the basis functions are polynomials (Gautschi 2004; Rahman 2009a). Further details are available elsewhere (Xu and Rahman 2004).

The S-variate, mth-order PDD approximation requires evaluations of \(Q_{S,m}=\sum\limits_{k=0}^{k=S}\binom {N}{S-k}m^{S-k}=\sum\limits_{k=0}^{k=S}\binom {N}{k}m^{k}\) expansion coefficients, including \(y_{\emptyset }\). If these coefficients are estimated by dimension-reduction integration with \(R=S<N\) and, therefore, involve at most S-dimensional tensor product of an n-point univariate quadrature rule depending on m, then the total cost for the S-variate, mth-order approximation entails a maximum of \(\sum_{k=0}^{k=S}\binom {N}{S-k}n^{S-k}(m)=\sum _{k=0}^{k=S}\binom {N}{k}n^{k}(m)\) function evaluations. If the integration points include a common point in each coordinate – a special case of symmetric input probability density functions and odd values of n – the number of function evaluations reduces to \(\sum _{k=0}^{k=S}\binom {N}{S-k}(n(m)-1)^{S-k}=\sum _{k=0}^{k=S}\binom {N}{k}(n(m)-1)^{k}\). Nonetheless, the computational complexity of the S-variate PDD approximation is Sth-order polynomial with respect to the number of random variables or integration points. Therefore, PDD alleviates the curse of dimensionality to some extent.

4 Proposed methods for design sensitivity analysis1

When solving design problems using gradient-based optimization algorithms, at least first-order derivatives of both the objective and constraint functions with respect to each design variable are required. For an RDO problem defined by (2), calculating such derivatives is trivial once the derivatives of the first two moments of \(y_{l}(\mathbf {X})\), \(l=0,1,\ldots ,K\), are known. In this subsection, a new, analytical method, developed by blending PDD with score functions, for design sensitivity analysis is presented.

4.1 Score functions

Suppose that the first-order derivatives of the first two moments, \(\mathbb {E}_{\mathbf {d}}[y^{r}(\mathbf {X})]\), \(r=1,2\), of a generic stochastic response \(y(\mathbf {X})\) with respect to a design variable \(d_{k}\) are sought. Taking the partial derivative of these moments with respect to \(d_{k}\) and then applying the Lebesgue dominated convergence theorem (Browder 1996), which permits the differential and integral operators to be interchanged, yields the sensitivities
$$\begin{array}{rcl} \frac{\partial\mathbb{E}_{\mathbf{d}}\left[y^{r}(\mathbf{X})\right]}{\partial d_{k}}&=&\frac{\partial}{\partial d_{k}}{\int_{\mathbb{R}^{N}}{y^{r}(\mathbf{x})f_{\mathbf{X}}(\mathbf{x};\mathbf{d})d\mathbf{x}}} \\ &=&{\int_{\mathbb{R}^{N}}{y^{r}(\mathbf{x}){\frac{\partial\ln f_{\mathbf{X}}(\mathbf{x};\mathbf{d})}{\partial d_{k}}}f_{\mathbf{X}}(\mathbf{x};\mathbf{d})d\mathbf{x}}}, \\ && r=1,2;\; k=1,\ldots,M, \end{array} $$
(20)
provided that \(f_{\mathbf {X}}(\mathbf {x};\mathbf {d})>0\). In (20), \(\partial \ln f_{\mathbf {X}}(\mathbf {X};\mathbf {d})\left /\partial d_{k}\right .\) is known as the first-order score function for the design variable \(d_{k}\) (Rubinstein and Shapiro 1993; Rahman 2009b). In general, the sensitivities are not available analytically since the moments are not either. Nonetheless, the moments and their sensitivities have both been formulated as expectations of stochastic quantities with respect to the same probability measure, facilitating their concurrent evaluations in a single stochastic simulation or analysis.

Remark 4

The evaluation of score functions, \(\partial \ln f_{\mathbf {X}}(\mathbf {X}\); \(\mathbf {d})\left /\partial d_{k}\right .\); \(k=1,\ldots ,M\), requires differentiating only the probability density function of \(\mathbf {X}\). Therefore, the resulting score functions can be determined easily and, in many cases, analytically—for instance, when \(\mathbf {X}\) follows classical probability distributions (Rahman 2009b). If the density function of \(\mathbf {X}\) is arbitrarily prescribed, the score functions can be calculated numerically, yet inexpensively, since no evaluation of the response function is involved.

When \(\mathbf {X}\) comprises independent variables, as assumed here, the log density, \(\ln f_{\mathbf {X}}(\mathbf {X};\mathbf {d})=\sum _{i=1}^{i=N}\ln f_{X_{i}}(x_{i};\mathbf {d})\), is a sum of univariate functions. If \(d_{k}\) is a distribution parameter of the random variable \(X_{i_{k}}\), then the first-order score function for the kth design variable simplifies to \(\partial \ln f_{\mathbf {X}}(\mathbf {x};\mathbf {d})\left /\partial d_{k}\right .=\partial \ln f_{X_{i_{k}}}(X_{i_{k}};\mathbf {d})\left /\partial d_{k}\right .\), which is a univariate function of \(X_{i_{k}}\). Defining \(s_{k}(X_{i_{k}};\mathbf {d}):=\partial \ln f_{X_{i_{k}}}(X_{i_{k}};\mathbf {d})\left /\partial d_{k}\right .\) as the kth first-order score function, the sensitivity is obtained from
$$\begin{array}{rcl} \frac{\partial\mathbb{E}_{\mathbf{d}}\left[y^{r}(\mathbf{X})\right]}{\partial d_{k}}&=&{ \int_{\mathbb{R}^{N}}{ y^{r}(\mathbf{x})s_{k}(x_{i_{k}};\mathbf{d})f_{\mathbf{X}}(\mathbf{x};\mathbf{d})d\mathbf{x}}}\\ &=:&\mathbb{E}_{\mathbf{d}}\left[y^{r}(\mathbf{X})s_{k}(X_{i_{k}};\mathbf{d})\right], \end{array} $$
(21)
an expectation of a product comprising response and score functions with respect to the probability measure \(P_{\mathbf {d}}\), \(\mathbf {d}\in \mathcal {D}\).

4.2 Exact sensitivities

For independent coordinates of \(\mathbf {X}\), consider the Fourier-polynomial expansion of the kth score function
$$ s_{k}(X_{i_{k}};\mathbf{d})=s_{k,\emptyset}(\mathbf{d})+{ \sum\limits_{j=1}^{\infty}}D_{i_{k},j}(\mathbf{d})\psi_{i_{k}j}(X_{i_{k}};\mathbf{d}), $$
(22)
consisting of its own expansion coefficients
$$ s_{k,\emptyset}(\mathbf{d}):=\int_{\mathbb{R}}s_{k}(x_{i_{k}};\mathbf{d})f_{X_{i_{k}}}(x_{i_{k}};\mathbf{d})dx_{i_{k}}\; $$
(23)
and
$$ D_{i_{k},j}(\mathbf{d}):=\int_{\mathbb{R}}s_{k}(x_{i_{k}};\mathbf{d})\psi_{i_{k}j}(x_{i_{k}};\mathbf{d})f_{X_{i_{k}}}(x_{i_{k}};\mathbf{d})dx_{i_{k}}. $$
(24)
Employing (11) and (22), the product appearing in the last expression of (21) expands to
$$\begin{array}{rcl} && y^{r}(\mathbf{X})s_{k}(X_{i_{k}};\mathbf{d}) \\ &&=\left( y_{\emptyset}(\mathbf{d})+{\sum\limits_{\emptyset\ne u\subseteq\{1,\ldots,N\}}}\sum\limits_{{\textstyle {\mathbf{j}_{|u|}\in\mathbb{N}_{0}^{|u|}\atop j_{1},\ldots,j_{|u|}\neq0}}} C_{u\mathbf{j}_{|u|}}(\mathbf{d})\psi_{u\mathbf{j}_{|u|}}(\mathbf{X}_{u};\mathbf{d})\right)^{r} \\ && \quad\times \left( s_{k,\emptyset}(\mathbf{d})+{ \sum_{j=1}^{\infty}}D_{i_{k},j}(\mathbf{d})\psi_{i_{k}j}(X_{i_{k}};\mathbf{d})\right), \end{array} $$
(25)
encountering the same orthonormal polynomials bases that are consistent with the probability measure \(f_{\mathbf {X}}(\mathbf {x};\mathbf {d})d\mathbf {x}\). The expectation of (25) for \(r=1\) and 2, aided by Propositions 1 and 2, produces
$$ \frac{\partial\mathbb{E}_{\mathbf{d}}\left[y(\mathbf{X})\right]}{\partial d_{k}}=s_{k,\emptyset}(\mathbf{d})y_{\emptyset}(\mathbf{d})+{ \sum\limits_{j=1}^{\infty}}C_{i_{k}j}(\mathbf{d})D_{i_{k},j}(\mathbf{d}), $$
(26)
$$ \begin{array}{rcl} \frac{\partial\mathbb{E}_{\mathbf{d}}\left[y^{2}(\mathbf{X})\right]}{\partial d_{k}} &=& 2y_{\emptyset}(\mathbf{d}){ \sum\limits_{j=1}^{\infty}}C_{i_{k}j}(\mathbf{d})D_{i_{k},j}(\mathbf{d})+s_{k,\emptyset}(\mathbf{d})y_{\emptyset}^{2}(\mathbf{d})\\ && + s_{k,\emptyset}(\mathbf{d})\mathrm{var}_{\mathbf{d}}[y(\mathbf{X})]+T_{k}, \end{array} $$
(27)
representing closed-form expressions of the sensitivities in terms of the PDD or Fourier-polynomial expansion coefficients of the response or score function. The last term on the right side of (27) is
$$\begin{array}{rcl} T_{k}&=&\sum\limits_{i_{1}=1}^{N}{\sum\limits_{i_{2}=1}^{N}}{\sum\limits_{j_{1}=1}^{\infty}}{ \sum\limits_{j_{2}=1}^{\infty}}{ \sum\limits_{j_{3}=1}^{\infty}}C_{i_{1}j_{1}}(\mathbf{d})C_{i_{2}j_{2}}(\mathbf{d})D_{i_{k},j_{3}}(\mathbf{d}) \\ &&\times\;\mathbb{E}_{\textrm{\textbf{d}}}\left[\psi_{i_{1}j_{1}}(X_{i_{1}};\mathbf{d})\psi_{i_{2}j_{2}}(X_{i_{2}};\mathbf{d})\psi_{i_{k}j_{3}}(X_{i_{k}};\mathbf{d})\right], \end{array} $$
(28)
which requires expectations of various products of three random orthonormal polynomials. However, if \(X_{i}\) follows Gaussian distribution, then the expectations are easily determined from the properties of univariate Hermite polynomials, yielding (Busbridge 1948)
$$\begin{array}{rcl} &&{} \mathbb{E}_{\mathbf{d}}\left[\psi_{i_{1}j_{1}}(X_{i_{1}};\mathbf{d})\psi_{i_{2}j_{2}}(X_{i_{2}};\mathbf{d})\psi_{i_{3}j_{3}}(X_{i_{2}};\mathbf{d})\right]\\ &&=\left\{ \begin{array}{ll} \frac{\sqrt{j_{1}!j_{2}!j_{3}!}}{(q-j_{1})!(q-j_{2})!(q-j_{3})!} & \mathrm{if}\;\;\begin{array}{l} i_{1}=i_{2}=i_{3},\ \ensuremath{q\in\mathbb{N},}\\ j_{1}+j_{2}+j_{3}=2q,\\ j_{1},j_{2},j_{3}\le q, \end{array}\\[1.5em] 0 & \mathrm{otherwise}. \end{array}\right.\\ \end{array} $$
(29)
Note that these sensitivity equations are exact because PDD and Fourier-polynomial expansion provide exact representation of a square-integrable function.

4.3 Approximate sensitivities

When \(y(\mathbf {X})\) and \(s_{k}(X_{i_{k}};\mathbf {d})\) are replaced by their S-variate, mth-order PDD and \(m^{\prime}\)th-order Fourier-polynomial approximations, respectively, the resultant sensitivity equations, expressed by
$$ \frac{\partial\mathbb{E}_{\mathbf{d}}\left[\tilde{y}_{S,m}(\mathbf{X})\right]}{\partial d_{k}}=s_{k,\emptyset}(\mathbf{d})y_{\emptyset}(\mathbf{d})+\sum\limits_{j=1}^{m_{\min}}C_{i_{k}j}(\mathbf{d})D_{i_{k},j}(\mathbf{d})\; $$
(30)
and
$$\begin{array}{rcl} &&{} \frac{\partial\mathbb{E}_{\mathbf{d}}\left[\tilde{y}_{S,m}^{2}(\mathbf{X})\right]}{\partial d_{k}} \\ &&= 2y_{\emptyset}(\mathbf{d})\sum\limits_{j=1}^{m_{\min}}C_{i_{k}j}(\mathbf{d})D_{i_{k},j}(\mathbf{d})+s_{k,\emptyset}(\mathbf{d})y_{\emptyset}^{2}(\mathbf{d})\\ &&\quad +\,s_{k,\emptyset}(\mathbf{d})\mathrm{var}_{\mathbf{d}}\left[\tilde{y}_{S,m}(\mathbf{X})\right]+\tilde{T}_{k,m,m^{\prime}}, \end{array} $$
(31)
where \(m_{\text {min}}:=\min (m,m^{\prime})\) and
$$\begin{array}{rcl} \tilde{T}_{k,m,m^{\prime}}&\,=\,&\sum\limits_{i_{1}=1}^{N}{\sum\limits_{i_{2}=1}^{N}}{\sum\limits_{j_{1}=1}^{m}}{ \sum\limits_{j_{2}=1}^{m}}{ \sum\limits_{j_{3}=1}^{m^{\prime}}}C_{i_{1}j_{1}}(\mathbf{d})C_{i_{2}j_{2}}(\mathbf{d})D_{i_{k},j_{3}}(\mathbf{d}) \mathbb{E_{\textrm{\textbf{d}}}}\\ &&\mathbb{E}_{\mathbf{d}}\left[\psi_{i_{1}j_{1}}(X_{i_{1}};\mathbf{d})\psi_{i_{2}j_{2}}(X_{i_{2}};\mathbf{d})\psi_{i_{i_{k}}j_{3}}(X_{i_{k}};\mathbf{d})\right],\\ \end{array} $$
(32)
become approximate, relying on the truncation parameters S, m, and \(m^{\prime}\) in general. It is elementary to show that when \(S=N\) and \(m=m^{\prime}=\infty \), \(\mathrm {var}_{\mathbf {d}}[\tilde {y}_{S,m}(\mathbf {X})]=\mathrm {var}_{\mathbf {d}}[y(\mathbf {X})]\) and \(\tilde {T}_{k,m,m^{\prime}}=T_{k}\). Therefore, the approximate sensitivities of the moments also converge to exactness when \(S\to N\) and \(m_{\min }\to \infty \).
Of the two sensitivities, \(\partial \mathbb {E}_{\mathbf {d}}[\tilde {y}_{S,m}(\mathbf {X})]/\partial d_{k}\) does not depend on S, meaning that both the univariate (\(S=1\)) and bivariate (\(S=2\)) approximations, given the same \(m_{\min }<\infty \), form the same result, as displayed in (30). However, the sensitivity equations of \(\partial \mathbb {E}_{\mathbf {d}}[\tilde {y}_{S,m}^{2}(\mathbf {X})]/\partial d_{k}\) for the univariate and bivariate approximations vary with respect to S, m, and \(m^{\prime}\). For instance, the univariate approximation results in
$$\begin{array}{rcl} &&{} \frac{\partial\mathbb{E}_{\mathbf{d}}\left[\tilde{y}_{1,m}^{2}(\mathbf{X})\right]}{\partial d_{k}} \\ &&= 2y_{\emptyset}(\mathbf{d})\sum\limits_{j=1}^{m_{\min}}C_{i_{k}j}(\mathbf{d})D_{i_{k},j}(\mathbf{d})+s_{k,\emptyset}(\mathbf{d})y_{\emptyset}^{2}(\mathbf{d})\\ &&\quad+\,s_{k,\emptyset}(\mathbf{d})\mathrm{var}_{\mathbf{d}}\left[\tilde{y}_{1,m}(\mathbf{X})\right]+\tilde{T}_{k,m,m^{\prime}}, \end{array} $$
(33)
whereas the bivariate approximation yields
$$\begin{array}{rcl} &&{} \frac{\partial\mathbb{E}_{\mathbf{d}}\left[\tilde{y}_{2,m}^{2}(\mathbf{X})\right]}{\partial d_{k}} \\ &&= 2y_{\emptyset}(\mathbf{d}){\sum\limits_{j=1}^{m_{\min}}}C_{i_{k}j}(\mathbf{d})D_{i_{k},j}(\mathbf{d})+s_{k,\emptyset}(\mathbf{d})y_{\emptyset}^{2}(\mathbf{d}) \\ &&\quad+\,s_{k,\emptyset}(\mathbf{d})\mathrm{var}_{\mathbf{d}}[\tilde{y}_{2,m}(\mathbf{X})]+\tilde{T}_{k,m,m^{\prime}}. \end{array} $$
(34)
Analogous to the moments, the univariate and bivariate approximations of the sensitivities of the moments involve only univariate and at most bivariate expansion coefficients of y, respectively. Since the expansion coefficients of the score function do not involve the response function, no additional cost is incurred from response analysis. In other words, the effort required to obtain the statistical moments of a response also furnish the sensitivities of moments, a highly desirable trait for efficiently solving RDO problems.

Remark 5

Since the score functions are univariate functions, their expansion coefficients require only univariate integration for their evaluations. When \(X_{i}\) follows classical distributions—for instance, the Gaussian distribution—then the coefficients can be calculated exactly or analytically. Otherwise, numerical quadrature is required. Nonetheless, there is no need to employ dimension-reduction integration for calculating the expansion coefficients of the score functions.

5 Proposed methods for design optimization

The PDD approximations described in the preceding section provide a means to approximate the objective and constraint functions, including their design sensitivities, from a single stochastic analysis. Therefore, any gradient-based algorithm employing PDD approximations should render a convergent solution of the RDO problem in (2). However, there exist multiple ways to dovetail stochastic analysis with an optimization algorithm. Four such design optimization methods, all anchored in PDD, are presented in this section.

5.1 Direct PDD

The direct PDD method involves straightforward integration of the PDD-based stochastic analysis with design optimization. Given a design vector at the current iteration and the corresponding values of the objective and constraint functions and their sensitivities, the design vector at the next iteration is generated from a suitable gradient-based optimization algorithm. However, new statistical moment and sensitivity analyses, entailing re-calculations of the PDD expansion coefficients, are needed at every design iteration. Therefore, the direct PDD method is expensive, depending on the cost of evaluating the objective and constraint functions and the requisite number of design iterations.

5.2 Single-step PDD

The single-step PDD method is motivated on solving the entire RDO problem from a single stochastic analysis by sidestepping the need to recalculate the PDD expansion coefficients at every design iteration. It subsumes two important assumptions: (1) an S-variate, mth-order PDD approximation \(\tilde {y}_{S,m}\) of y at the initial design is acceptable for all possible designs; and (2) the expansion coefficients for one design, derived from those generated for another design, are accurate.

Consider a change of the probability measure of \(\mathbf {X}\) from \(f_{\mathbf {X}}(\mathbf {x};\mathbf {d})d\mathbf {x}\) to \(f_{\mathbf {X}}(\mathbf {x};\mathbf {d}^{\prime})d\mathbf {x}\), where \(\mathbf {d}\) and \(\mathbf {d}^{\prime}\) are two arbitrary design vectors corresponding to old and new designs, respectively. Let \(\{\psi _{ij}(X_{i};\mathbf {d}^{\prime});\; j=0,1,\ldots \}\) be a set of new orthonormal polynomial basis functions consistent with the marginal probability measure \(f_{X_{i}}(x_{i};\mathbf {d}^{\prime})dx_{i}\) of \(X_{i}\), producing new product polynomials \(\psi _{u\mathbf {j}_{|u|}}(\mathbf {X}_{u};\mathbf {d}^{\prime})=\prod _{p=1}^{|u|}\psi _{i_{p}j_{p}}(X_{i_{p}};\mathbf {d}^{\prime})\), \(\emptyset \ne u\subseteq \{1,\ldots ,N\}\). Assume that the expansion coefficients, \(y_{\emptyset }(\mathbf {d})\) and \(C_{u\mathbf {j}_{|u|}}(\mathbf {d})\), for the old design have been calculated already. Then, the expansion coefficients for the new design are determined from
https://static-content.springer.com/image/art%3A10.1007%2Fs00158-013-0883-z/MediaObjects/158_2013_883_Figa_HTML.gif
(35)
and
https://static-content.springer.com/image/art%3A10.1007%2Fs00158-013-0883-z/MediaObjects/158_2013_883_Figb_HTML.gif
(36)
for all \(\emptyset \ne u\subseteq \{1,\ldots ,N\}\) by recycling the old expansion coefficients and using orthonormal polynomials associated with both designs. The relationship between the old and new coefficients, described by (35) and (36), is exact and is obtained by replacing y with the right side of (11) in (4) and (10). However, in practice, when the S-variate, mth-order PDD approximation (12) is used to replace y in (4) and (10), then the new expansion coefficients,
https://static-content.springer.com/image/art%3A10.1007%2Fs00158-013-0883-z/MediaObjects/158_2013_883_Figc_HTML.gif
(37)
and
https://static-content.springer.com/image/art%3A10.1007%2Fs00158-013-0883-z/MediaObjects/158_2013_883_Figd_HTML.gif
(38)
which are applicable for \(\emptyset \ne u\subseteq \{1,\ldots ,N\}\), \(1\le |u|\le S\), become approximate, although convergent. In the latter case, the integrals in (37) and (38) consist of finite-order polynomial functions of at most S variables and can be evaluated inexpensively without having to compute the original function y for the new design. Therefore, new stochastic analyses, all employing S-variate, mth-order PDD approximation of y, are conducted with little additional cost during all design iterations, drastically curbing the computational effort in solving the RDO problem.

5.3 Sequential PDD

When the truncations parameters, S and/or m, of a PDD approximation are too low, the assumptions of the single-step PDD method are likely to be violated, resulting in a premature or an inaccurate optimal solution. To overcome this problem, a sequential PDD method, combining the ideas of the single-step PDD and direct PDD methods, was developed. It forms a sequential design process, where each sequence begins with a single-step PDD using the expansion coefficients calculated at an optimal design solution generated from the previous sequence. Although more expensive than the single-step PDD method, the sequential PDD method is expected to be more economical than the direct PDD method.

The sequential PDD method is outlined by the following steps. The flow chart of this method is shown in Fig. 1.
  1. Step 1:

    Select an initial design vector \(\mathbf {d}_{0}\). Define a tolerance \(\epsilon >0\). Set the iteration \(q=1\), qth initial design vector \(\mathbf {d}_{0}^{(q)}=\mathbf {d}_{0}\), and approximate optimal solution \(\mathbf {d}_{\ast }^{(0)}=\mathbf {d}_{0}\) at \(q=0\).

     
  2. Step 2:

    Select (\(q=1\)) or use (\(q>1\)) the PDD and Fourier truncation parameters S, m, and \(m^{\prime}\). At \(\mathbf {d}=\mathbf {d}_{0}^{(q)}\), generate the PDD expansion coefficients, \(y_{\emptyset }(\mathbf {d})\) and \(C_{u\mathbf {j}_{|u|}}(\mathbf {d})\), where \(\emptyset \ne u\subseteq \{1,\ldots ,N\}\), \(1\le |u|\le S\), \(\mathbf {j}_{|u|}\in \mathbb {N}_{0}^{|u|}\), \(||\mathbf {j}_{|u|}||_{\infty }\le m\), \(j_{1},\ldots ,j_{|u|}\neq 0\), using dimension-reduction integration with \(R=S\), \(n=m+1\), leading to S-variate, mth-order PDD approximations of \(y_{l}(\mathbf {X})\), \(l=0,1,\ldots ,K\), in (2). Calculate the expansion coefficients of the score functions, \(s_{k,\emptyset }(\mathbf {d})\) and \(D_{i_{k},j}(\mathbf {d})\), where \(k=1,\ldots ,M\) and \(j=1,\ldots ,m^{\prime}\), analytically, if possible, or numerically, resulting in \(m^{\prime}\)th-order Fourier-polynomial approximations of \(s_{k}(X_{i_{k}};\mathbf {d})\), \(k=1,\ldots ,M\).

     
  3. Step 3:

    Solve the design problem in (2) employing PDD approximations of \(y_{l}\), \(l=0,1,\ldots ,K\) and a standard gradient-based optimization algorithm. In so doing, recycle the PDD expansion coefficients obtained from Step 2 in (37) and (38), producing approximations of the objective and constraint functions that stem from single calculation of these coefficients. To evaluate the gradients, recalculate the Fourier expansion coefficients of score functions as needed. Denote the approximate optimal solution by \(\mathbf {d}_{\ast }^{(q)}\). Set \(\mathbf {d}_{0}^{(q+1)}=\mathbf {d}_{\ast }^{(q)}\).

     
  4. Step 4:

    If \(||\mathbf {d}_{\ast }^{(q)}-\mathbf {d}_{\ast }^{(q-1)}||_{2}<\epsilon \), then stop and denote the final approximate optimal solution as \(\tilde {\mathbf {d}^{\ast }}=\mathbf {d}_{\ast }^{(q)}\). Otherwise, update \(q=q+1\) and go to Step 2.

     
https://static-content.springer.com/image/art%3A10.1007%2Fs00158-013-0883-z/MediaObjects/158_2013_883_Fig1_HTML.gif
Fig. 1

A flow chart of the sequential PDD method

5.4 Multi-point single-step PDD

The optimization methods described in the preceding subsections are founded on PDD approximations of stochastic responses, supplying surrogates of objective and constraint functions for the entire design space. Therefore, these methods are global and may not be cost-effective when the truncation parameters of PDD are required to be exceedingly large to capture high-order responses or high-variate interactions of input variables. Furthermore, a global method using a truncated PDD, obtained by retaining only low-order or low-variate terms, may not even find a true optimal solution. An attractive alternative method, developed in this work and referred to as the multi-point single-step PDD method, involves local implementations of the single-step PDD approximation that are built on a local subregion of the design space. According to this method, the original RDO problem is exchanged with a succession of simpler RDO sub-problems, where the objective and constraint functions in each sub-problem represent their multi-point approximations (Toropov et al. 1993). The design solution of an individual sub-problem, obtained by the single-step PDD method, becomes the initial design for the next sub-problem. Then, the move limits are updated, and the optimization is repeated iteratively until the optimal solution is attained. Due to its local approach, the multi-point single-step PDD method should solve practical engineering problems using low-order and/or low-variate PDD approximations.

Let \(\mathcal {D}=\times _{k=1}^{k=M}[d_{k,L},d_{k,U}]\subseteq \mathbb {R}^{M}\) be a rectangular domain, representing the design space of the RDO problem defined by (2). For a scalar variable \(0<\beta _{k}^{(q)}\le 1\) and an initial design vector \(\mathbf {d}_{0}^{(q)}=(d_{1,0}^{(q)},\ldots ,d_{M,0}^{(q)})\), the subset \(\mathcal {D}^{(q)}=\times _{k=1}^{k=M}[d_{k,0}^{(q)}-\beta _{k}^{(q)}(d_{k,U}-d_{k,L})/2,d_{k,0}^{(q)}+\beta _{k}^{(q)}(d_{k,U}-d_{k,L})/2]\subseteq \mathcal {D}\subseteq \mathbb {R}^{M}\) defines the qth subregion for \(q=1,2,\ldots \). According to the multi-point single-step PDD method, the RDO problem in (2) is reformulated to
$$\begin{array}{rcl} \min_{\mathbf{d}\in\mathcal{D}^{(q)}\subseteq\mathcal{D}}&\quad& \tilde{c}_{0,S,m}^{(q)}(\mathbf{d}):=w_{1}\frac{{ \mathbb{E}_{\mathbf{d}}}\left[\tilde{y}_{0,S,m}^{(q)}(\mathbf{X})\right]}{\mu_{0}^{\ast}}\\ && \quad\qquad\qquad+\,w_{2}\frac{\sqrt{\mathrm{var}_{\mathbf{d}}\left[\tilde{y}_{0,S,m}^{(q)}(\mathbf{X})\right]}}{\sigma_{0}^{\ast}},\\ \mathrm{subject\: to} &\quad& \tilde{c}_{l,S,m}^{(q)}(\mathbf{d}):=\alpha_{l}\sqrt{\mathrm{\mathrm{var}_{\mathbf{d}}}\left[\tilde{y}_{l,S,m}^{(q)}(\mathbf{X})\right]}\\ &&\qquad\quad\qquad -\,{\mathbb{E}_{\mathbf{d}}}\left[\tilde{y}_{l,S,m}^{(q)}(\mathbf{X})\right]\le0,\\ &&l=1,\ldots,K,\\ &&d_{k,0}^{(q)}-\beta_{k}^{(q)}(d_{k,U}-d_{k,L})/2\le d_{k}\le d_{k,0}^{(q)}\\ &&+\,\beta_{k}^{(q)}(d_{k,U}-d_{k,L})/2,\: k=1,\ldots,M, \\ \end{array} $$
(39)
where \(\tilde {y}_{l,S,m}^{(q)}(\mathbf {X})\) and \(\tilde {c}_{l,S,m}^{(q)}(\mathbf {d})\), \(l=0,1,2,\ldots ,K\), are local, S-variate, mth-order PDD approximations of \(y_{l}(\mathbf {X})\) and \(c_{l}(\mathbf {d})\), respectively, at iteration q, and \(d_{k,0}^{(q)}-\beta _{k}^{(q)}(d_{k,U}-d_{k,L})/2\) and \(d_{k,0}^{(q)}+\beta _{k}^{(q)}(d_{k,U}-d_{k,L})/2\), also known as the move limits, are the lower and upper bounds, respectively, of the subregion \(\mathcal {D}^{(q)}\). The multi-point single-step PDD method solves the optimization problem in (39) for \(q=1,2,\ldots \) by successively employing the single-step PDD approximation at each subregion or iteration until convergence is attained. When \(S\to N\) and \(m\to \infty \), the second-moment properties of PDD approximations converge to their exact values, yielding coincident solutions of the optimization problems described by (2) and (39). However, if the subregions are sufficiently small, then for finite and possibly low values of S and m, (39) is expected to generate an accurate solution of (2), the principal motivation of this method.
The multi-point single-step PDD method is outlined by the following steps. The flow chart of this method is shown in Fig. 2.
  1. Step 1:

    Select an initial design vector \(\mathbf {d}_{0}\). Define tolerances \(\epsilon _{1}>0\), \(\epsilon _{2}>0\), and \(\epsilon _{3}>0\). Set the iteration \(q=1\), \(\mathbf {d}_{0}^{(q)}=(d_{1,0}^{(q)},\ldots ,d_{M,0}^{(q)})=\mathbf {d}_{0}\). Define the subregion size parameters \(0<\beta _{k}^{(q)}\le 1\), \(k=1,\ldots ,M\), describing \(\mathcal {D}^{(q)}=\times _{k=1}^{k=M}[d_{k,0}^{(q)}-\beta _{k}^{(q)}(d_{k,U}-d_{k,L})/2,d_{k,0}^{(q)}+\beta _{k}^{(q)}(d_{k,U}-d_{k,L})/2]\). Denote the subregion’s increasing history by a set \(H^{(0)}\) and set it to empty. Set two designs \(\mathbf {d}_{f}=\mathbf {d}_{0}\) and \(\mathbf {d}_{f,last}\ne \mathbf {d}_{0}\) such that \(||\mathbf {d}_{f}-\mathbf {d}_{f,last}||_{2}>\epsilon _{1}\). Set \(\mathbf {d}_{\ast }^{(0)}=\mathbf {d}_{0}\) , \(q_{f,last}=1\) and \(q_{f}=1\). Usually, a feasible design should be selected to be the initial design \(\mathbf {d}_{0}\). However, when an infeasible initial design is chosen, a new feasible design can be obtained during the iteration if the initial subregion size parameters are large enough.

     
  2. Step 2:

    Select (\(q=1\)) or use (\(q>1\)) the PDD truncation parameters S and m. At \(\mathbf {d}=\mathbf {d}_{0}^{(q)}\), generate the PDD expansion coefficients, \(y_{\emptyset }(\mathbf {d})\) and \(C_{u\mathbf {j}_{|u|}}(\mathbf {d})\), where \(\emptyset \ne u\subseteq \{1,\ldots ,N\}\), \(1\le |u|\le S\), \(\mathbf {j}_{|u|}\in \mathbb {N}_{0}^{|u|}\), \(||\mathbf {j}_{|u|}||_{\infty }\le m\), \(j_{1},\ldots ,j_{|u|}\neq 0\), using dimension-reduction integration with \(R=S\), \(n=m+1\), leading to S-variate, mth-order PDD approximations \(\tilde {y}_{l,S,m}^{(q)}(\mathbf {X})\) of \(y_{l}(\mathbf {X})\) and \(\tilde {c}_{l,S,m}^{(q)}(\mathbf {d})\) of \(c_{l}(\mathbf {d})\), \(l=0,1,\ldots ,K\), in (2). Calculate the expansion coefficients of score functions, \(s_{k,\emptyset }(\mathbf {d})\) and \(D_{i_{k},j}(\mathbf {d})\), where \(k=1,\ldots ,M\) and \(j=1,\ldots ,m^{\prime}\), analytically, if possible, or numerically, resulting in \(m^{\prime}\)th-order Fourier-polynomial approximations of \(s_{k}(X_{i_{k}};\mathbf {d})\), \(k=1,\ldots ,M\).

     
  3. Step 3:

    If \(q=1\) and \(\tilde {c}_{l}^{(q)}(\mathbf {d}_{0}^{(q)})<0\) for \(l=1,\ldots ,K\), then go to Step 4. If \(q>1\) and \(\tilde {c}_{l}^{(q)}(\mathbf {d}_{0}^{(q)})<0\) for \(l=1,\ldots ,K\), then set \(\mathbf {d}_{f,\mathrm {last}}=\mathbf {\mathbf {d}}_{f}\), \(\mathbf {\mathbf {d}}_{f}=\mathbf {d}_{0}^{(q)}\) , \(q_{f,\mathrm {last}}=q_{f}\), \(q_{f}=q\) and go to Step 4. Otherwise, go to Step 5.

     
  4. Step 4:

    If \(||\mathbf {\mathbf {d}}_{f}-\mathbf {d}_{f,last}||_{2}<\epsilon _{1}\) or \(\left |\left [\tilde {c}_{0}^{(q)}(\mathbf {d}_{f})-\tilde {c}_{0}^{(q_{f,\mathrm {last}})}(\mathbf {d}_{f,\mathrm {last}})\right ]/\tilde {c}_{0}^{(q)}(\mathbf {d}_{f})\right |<\epsilon _{3}\), then stop and denote the final optimal solution as \(\tilde {\mathbf {d}}^{\ast }=\mathbf {d}_{f}\). Otherwise, go to Step 6.

     
  5. Step 5:

    Compare the infeasible design \(\mathbf {d}_{0}^{(q)}\) with the feasible design \(\mathbf {d}_{f}\) and interpolate between \(\mathbf {d}_{0}^{(q)}\) and \(\mathbf {d}_{f}\) to obtain a new feasible design and set it as \(\mathbf {d}_{0}^{(q+1)}\). For dimensions with large differences between \(\mathbf {d}_{0}^{(q)}\) and\(\mathbf {d}_{f}\), interpolate aggressively. Reduce the size of the subregion \(\mathcal {D}^{(q)}\) to obtain new subregion \(\mathcal {D}^{(q+1)}\). For dimensions with large differences between \(\mathbf {d}_{0}^{(q)}\) and \(\mathbf {d}_{f}\), reduce aggressively. Also, for dimensions with large differences between the sensitivities of \(\tilde {c}_{l,Sm}^{(q)}(\mathbf {d}_{0}^{(q)})\) and \(\tilde {c}_{l,Sm}^{(q-1)}(\mathbf {d}_{0}^{(q)})\), reduce aggressively. Update \(q=q+1\) and go to Step 2.

     
  6. Step 6:

    If the subregion size is small, that is, \(\beta _{k}^{(q)}(d_{k,U}-d_{k,L})<\epsilon _{2}\), and \(\mathbf {d}_{\ast }^{(q-1)}\) is located on the boundary of the subregion, then go to Step 7. Otherwise, go to Step 9.

     
  7. Step 7:

    If the subregion centered at \(\mathbf {d}_{0}^{(q)}\) has been enlarged before, that is, \(\mathbf {d}_{0}^{(q)}\in H^{(q-1)}\), then set \(H^{(q)}=H^{(q-1)}\) and go to Step 9. Otherwise, set \(H^{(q)}=H^{(q-1)}\bigcup \{\mathbf {d}_{0}^{(q)}\}\) and go to Step 8.

     
  8. Step 8:

    For coordinates of \(\mathbf {d}_{0}^{(q)}\) located on the boundary of the subregion and \(\beta _{k}^{(q)}(d_{k,U}-d_{k,L})<\epsilon _{2}\), increase the sizes of corresponding components of \(\mathcal {D}^{(q)}\); for other coordinates, keep them as they are. Set the new subregion as \(\mathcal {D}^{(q+1)}\).

     
  9. Step 9:

    Solve the design problem in (39) employing the single-step PDD method. In so doing, recycle the PDD expansion coefficients obtained from Step 2 in (37) and (38), producing approximations of the objective and constraint functions that stem from single calculation of these coefficients. To evaluate the gradients, recalculate the Fourier expansion coefficients of score functions as needed. Denote the optimal solution by \(\mathbf {d}_{\ast }^{(q)}\) and set \(\mathbf {d}_{0}^{(q+1)}=\mathbf {d}_{\ast }^{(q)}\). Update \(q=q+1\) and go to Step 2.

     
https://static-content.springer.com/image/art%3A10.1007%2Fs00158-013-0883-z/MediaObjects/158_2013_883_Fig2_HTML.gif
Fig. 2

A flow chart of the multi-point single-step PDD method

Table 1 summarizes several features of all four design methods developed in this work. It describes the design space of a method, how many times a method requires the PDD approximation, and whether the original problem or a sequence of subproblems are solved.
Table 1

Summary of features of the four proposed methods

Feature

Direct PDD

Single-step PDD

Sequential PDD

Multi-point single-step PDD

Design space

Global

Global

Global

Local

Frequency of PDD approximations

Every iteration

Only first iteration

A few iterations

For every subproblem

Problem solved in every iteration

Original problem

Original problem

Original problem

Subproblems

6 Numerical examples

Four examples are presented to illustrate the PDD methods developed in solving various RDO problems. The objective and constraint functions are either elementary mathematical functions or relate to engineering problems, ranging from simple structural to complex FEA-aided mechanical designs. Both size and shape design problems are included. In Examples 1–4, orthonormal polynomials, consistent with the probability distributions of input random variables, were used as bases. For the Gaussian distribution, the Hermite polynomials were used. For random variables following non-Gaussian probability distributions, such as the Lognormal, Beta, and Gumbel distributions in Example 2, the orthonormal polynomials were obtained either analytically when possible or numerically, exploiting the Stieltjes procedure (Rahman 2009a; Gautschi 2004). However, in Examples 3 and 4, the original random variables were transformed into standard Gaussian random variables, facilitating the use of classical Hermite polynomials as orthonormal polynomials. The PDD truncation parameters S and m vary, depending on the function or the example, but in all cases the PDD expansion coefficients were calculated using dimension-reduction integration with \(R=S\) and the number of integration points \(n=m+1\). The Gauss-quadrature rules are consistent with the polynomial basis functions employed. Since the design variables are the means of Gaussian random variables, the order \(m^{\prime}\) used for Fourier expansion coefficients of score functions in Examples 1, 3, and 4 is one. However, in Example 2, where the design variables describe both means and standard deviations of random variables, \(m^{\prime}\) is two. The tolerances and initial subregion size parameters are as follows: (1) \(\epsilon =0.001\); \(\epsilon _{1}=0.1\), \(\epsilon _{2}=2\); \(\epsilon _{3}=0\) (Example 3), \(\epsilon _{3}=0.005\) (Example 4); and (2) \(\beta _{1}^{(1)}=\ldots =\beta _{M}^{(1)}=0.5\). The optimization algorithm selected is sequential quadratic programming (DOT 2001) in all examples.

6.1 Example 1: optimization of a mathematical function

Consider a mathematical example, studied by Lee et al. (2009), involving two independent Gaussian random variables \(X_{1}\) and \(X_{2}\) and two design variables \(d_{1}={\mathbb {E}_{\mathbf {d}}}[X_{1}]\) and \(d_{2}={\mathbb {E}_{\mathbf {d}}}[X_{2}]\), which requires to
$$\begin{array}{rcl} \min_{\mathbf{d}\in\mathcal{D}} &\;& c_{0}(\mathbf{d})=\frac{\sqrt{\mathrm{var}_{\mathbf{d}}\left[y_{0}(\mathbf{X})\right]}}{15},\\ \mathrm{subject\: to} &\;& c_{1}(\mathbf{d})=3\sqrt{\mathrm{\mathrm{var}_{\mathbf{d}}}\left[y_{1}(\mathbf{X})\right]}-{ \mathbb{E}_{\mathbf{d}}}\left[y_{1}(\mathbf{X})\right]\le0,\\ && 1\le d_{1}\le10,\:1\le d_{2}\le10, \end{array} $$
(40)
where
$$ y_{0}(\mathbf{X})=(X_{1}-4)^{3}+(X_{1}-3)^{4}+(X_{2}-5)^{2}+10\; $$
(41)
and
$$ y_{1}(\mathbf{X})=X_{1}+X_{2}-6.45 $$
(42)
are two random functions. The random variables \(X_{1}\) and \(X_{2}\) have respective means \(d_{1}\) and \(d_{2}\), but the same standard deviation, which is equal to 0.4. The design vector \(\mathbf {d}=(d_{1},d_{2})\in \mathcal {D}\), where \(\mathcal {D}=(1,10)\times (1,10)\subset \mathbb {R}^{2}\).

Two proposed RDO methods, the direct PDD and single-step PDD methods, were applied to solve this problem. Since \(y_{0}\) and \(y_{1}\) are both univariate functions, only univariate (\(S=1\)) PDD approximations are required. The chosen PDD expansion orders are \(m=4\) for \(y_{0}\) and \(m=1\) for \(y_{1}\). The initial design vector \(\mathbf {d}_{0}=(5,5)\) and, correspondingly, \(\sqrt {\mathrm {var}_{\mathbf {d}_{0}}\left [y_{0}(\mathbf {X})\right ]}=18.2987\). The approximate optimal solution is denoted by \(\tilde {\mathbf {d}}^{\ast }=(\tilde {d}_{1}^{\ast },\tilde {d}_{2}^{\ast })\).

Table 2 summarizes the approximate optimal solutions, including the numbers of design iterations and function evaluations, by the two PDD methods. For comparison, the results of a tensor product quadrature (TPQ) method and Taylor series approximation, proposed by and obtained from Lee et al. (2009), are also included. From Table 2, all four methods engender close optimal solutions in four to five iterations. Hence, each method can be used to solve this optimization problem. Both PDD versions yield identical solutions due to the same truncation parameters selected. However, the numbers of function evaluations required to reach optimal solutions reduce dramatically when the single-step PDD is employed. This is because a univariate PDD approximation is adequate for the entire design space, facilitating exact calculations of the expansion coefficients by (37) and (38) for any design. In which case, the expansion coefficients need to be calculated only once during all design iterations. At respective optima, the exact values of objective functions for the PDD methods are smaller than those for the TPQ and first-order Taylor series methods. In addition, the numbers of function evaluations by the direct PDD or single-step PDD method are moderately or significantly lower than those by the TPQ method. Therefore, the PDD methods not only furnish a slightly better optimal solution, but also a more computationally efficient one than the TPQ method, at least in this example. Although the total numbers of function evaluations by the direct PDD and Taylor series methods are similar, the single-step PDD method is more efficient than the Taylor series method by almost a factor of six.
Table 2

Optimization results for the mathematical example

Results

Method

Direct PDD

Single-Step PDD

TPQa

Taylor seriesa

\(\tilde {d}_{1}^{\ast }\)

3.3508

3.3508

3.4449

3.4983

\(\tilde {d}_{1}^{\ast }\)

4.9856

4.9856

5.000

4.9992

\(c_{0}(\tilde {\mathbf {d}}^{\ast })\)b

0.0756

0.0756

0.0861c

0.0902c

\(c_{1}(\tilde {\mathbf {d}}^{\ast })\)b

−0.1873

−0.1599

−0.2978c

−0.3504c

\(\sqrt {\mathrm {var}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]}\)b

1.1340

1.1340

1.2915c

1.3535c

No. of iterations

5

5

4

4

No. of \(y_{0}\) evaluations

66

11

81

45

No. of \(y_{1}\) evaluations

30

5

81

45

a The results of TPQ (DSA) and Taylor series were obtained from Lee et al. (2009)

b The objective function, constraint functions, and \(\sqrt {\mathrm {var}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]}\) were evaluated exactly

c The objective and constraint functions of optimal designs by TPQ (DSA) and Taylor series were evaluated exactly

6.2 Example 2: size design of a two-bar truss

The second example, studied by Ramakrishnan and Rao (1996) and Lee et al. (2009), entails RDO of a two-bar truss structure, as shown in Fig. 3. There are five independent random variables, comprising the cross-sectional area \(X_{1}\), the half-horizontal span \(X_{2}\), mass density \(X_{3}\), load magnitude \(X_{4}\), and material yield (tensile) strength \(X_{5}\). Their probability distributions are listed in Table 3. The design variables are as follows: \(d_{1}={\mathbb {E}_{\mathbf {d}}}[X_{1}]\) and \(d_{2}={ \mathbb {E}_{\mathbf {d}}}[X_{2}]\). The objective is to minimize the second-moment properties of the mass of the structure subject to constraints, limiting axial stresses of both members at or below the yield strength of the material with 99.875 % probability if \(y_{l}\), \(l=1,2\), are Gaussian. The RDO problem is formulated to
$$\begin{array}{rcl} \min_{\mathbf{d}\in\mathcal{D}} &\;\;& c_{0}(\mathbf{d})=0.5\frac{{ \mathbb{E}_{\mathbf{d}}}\left[y_{0}(\mathbf{X})\right]}{10}+0.5\frac{\sqrt{\mathrm{var}_{\mathbf{d}}\left[y_{0}(\mathbf{X})\right]}}{2},\\ \mathrm{subject\: to} &\;\;& c_{1}(\mathbf{d})=3\sqrt{\mathrm{\mathrm{var}_{\mathbf{d}}}\left[y_{1}(\mathbf{X})\right]}-{ \mathbb{E}_{\mathbf{d}}}\left[y_{1}(\mathbf{X})\right]\le0,\\ && c_{2}(\mathbf{d})=3\sqrt{\mathrm{\mathrm{var}_{\mathbf{d}}}\left[y_{2}(\mathbf{X})\right]}-{ \mathbb{E}_{\mathbf{d}}}\left[y_{2}(\mathbf{X})\right]\le0\\ && 0.2\;\mathrm{cm^{2}}\le d_{1}\le20\;\mathrm{cm^{2}},0.1\;\mathrm{m}\le d_{2}\le1.6\;\mathrm{m}, \end{array} $$
(43)
where
$$ y_{0}(\mathbf{X})=X_{3}X_{1}\sqrt{1+X_{2}^{2}}, $$
(44)
$$ y_{1}(\mathbf{X})=1-\frac{5X_{4}\sqrt{1+X_{2}^{2}}}{\sqrt{65}X_{5}}\left(\frac{8}{X_{1}}+\frac{1}{X_{1}X_{2}}\right),\; $$
(45)
and
$$ y_{2}(\mathbf{X})=1-\frac{5X_{4}\sqrt{1+X_{2}^{2}}}{\sqrt{65}X_{5}}\left(\frac{8}{X_{1}}-\frac{1}{X_{1}X_{2}}\right) $$
(46)
are three random response functions. The design vector \(\mathbf {d}=(d_{1},d_{2})\in \mathcal {D}\), where \(\mathcal {D}=(0.2\;\mathrm {cm^{2}},20\;\mathrm {cm^{2}})\times (0.1\;\mathrm {m},1.6\;\mathrm {m})\subset \mathbb {R}^{2}\). The initial design vector is \(\mathbf {d}_{0}=(10\;\mathrm {cm}^{2},1\;\mathrm {m})\). The corresponding mean and standard deviation of \(y_{0}(\mathbf {d}_{0})\) at the initial design, calculated by crude MCS simulation with \(10^{8}\) samples, are 14.1422 kg and 2.8468 kg, respectively. The approximate optimal solution is denoted by \(\tilde {\mathbf {d}}^{\ast }=(\tilde {d}_{1}^{\ast },\tilde {d}_{2}^{\ast })\).
https://static-content.springer.com/image/art%3A10.1007%2Fs00158-013-0883-z/MediaObjects/158_2013_883_Fig3_HTML.gif
Fig. 3

A two-bar truss structure

Table 3

Statistical properties of random input for the two-bar truss problem

Random variable

Mean

Standard deviation

Probability distribution

Cross-sectional area (\(X_{1}\)), \(\mathrm {cm}{}^{2}\)

\(d_{1}\)

\(0.02d_{1}\)

Gaussian

Half-horizontal span (\(X_{2}\)), m

\(d_{2}\)

\(0.02d_{2}\)

Gaussian

Mass density (\(X_{3}\)), \(\mathrm {kg}/\mathrm {m}^{3}\)

10,000

2,000

Beta

Load magnitude (\(X_{4}\)), kN

800

200

Gumbel

Yield strength (\(X_{5}\)), MPa

1,050

250

Lognormal

Table 4 presents detailed optimization results generated by the direct and sequential PDD methods, each entailing univariate, bivariate, and trivariate PDD approximations with \(m=2\), \(n=3\). The optimal solutions by all PDD methods or approximations are very close to each other, all indicating that the first constraint is active. Although there are slight constraint violations (\(c_{1}>0\)), they are negligibly small. The results of bivariate and trivariate PDD approximations confirm that the univariate solution by either direct or sequential PDD method is valid and adequate. However, the numbers of function evaluations step up for higher-variate PDD approximations, as expected. When the sequential PDD method is employed, the respective numbers of function evaluations diminish by a factor of approximately two, regardless of the PDD approximation. While this reduction is not as dramatic as the one found in the single-step PDD method (Example 1), the sequential PDD method should still greatly improve the current state-of-the-art of robust design.
Table 4

Optimization results for the two-bar truss problem (\(m=2\), \(n=3\))

Results

Direct PDD (Univariate)

Direct PDD (Bivariate)

Direct PDD (Trivariate)

Sequential PDD (Univariate)

Sequential PDD (Bivariate)

Sequential PDD (Trivariate)

TPQa

Taylor seriesa

\(\tilde {d}_{1}^{\ast }\), \(\mathrm {cm}^{2}\)

11.4749

11.5561

11.5561

11.4811

11.5710

11.5714

11.5669

10.9573

\(\tilde {d}_{2}^{\ast }\), m

0.3781

0.3791

0.3791

0.3777

0.3753

0.3752

0.3767

0.3770

\(c_{0}(\tilde {\mathbf {d}}^{\ast })\)b

1.2300

1.2392

1.2391

1.2306

1.2392

1.2392

1.2393

1.1741

\(c_{1}(\tilde {\mathbf {d}}^{\ast })\)b

0.0172

0.0096

0.0096

0.0167

0.0097

0.0096

0.0095

0.0657

\(c_{2}(\tilde {\mathbf {d}}^{\ast })\)b

−0.4882

−0.4911

−0.4910

−0.4889

−0.4948

−0.4950

−0.4935

−0.4650

\(\mathbb {E}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]\)b, kg

12.2684

12.3591

12.3591

12.2732

12.3589

12.3598

12.3608

11.7105

\(\sqrt {\mathrm {var}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]}\)b, kg

2.4666

2.4851

2.4851

2.4677

2.4849

2.4850

2.4852

2.3542

No. of iterations

19

14

14

8

7

7

10

8

No. of \(y_{0}\) evaluations

190

518

896

80

259

448

594

108

Total no. of \(y_{1}\) & \(y_{2}\) evaluations

494

1,876

4,900

208

938

2,450

3,564

270

a The results of TPQ (DSA) and Taylor series were obtained from Lee et al. (2009)

b The objective and constraint functions, \(\mathbb {E}_{\boldsymbol {\tilde {\mathrm d}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]\), and \(\sqrt {\mathrm {var}_{\boldsymbol {\tilde {\mathrm d}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]}\) at respective optima, were evaluated by crude MCS (\(10^{8}\) samples)

Since this problem was also solved by the TPQ and Taylor series methods, comparing their reported solutions (Lee et al. 2009), listed in the last two columns of Table 4, with the PDD solutions should be intriguing. It appears that the TPQ method is also capable of producing a similar optimal solution, but by incurring a computational cost more than most of the PDD methods examined in this work. Comparing the numbers of function evaluations, the TPQ method is more expensive than the univariate direct PDD method by factors of three to seven. These factors grow into 7–17 when graded against the univariate sequential PDD method. The Taylor series method needs only 378 function evaluations, which is slightly more than 288 function evaluations by the univariate sequential PDD, but it violates the first constraint by at least six times more than all PDD and TPQ methods.

When the expansion order and the number of Gauss-quadrature points are increased to \(m=3\) and \(n=4\), respectively, the corresponding optimization results by all PDD, TPQ, and Taylor series methods are summarized in Table 5. The optimal solutions do not change greatly and, therefore, the results of Table 4 are adequate. However, the numbers of function evaluations rise for each method, as they should for larger m or n. In which case, the univariate PDD methods are even more efficient than the TPQ method by orders of magnitude.
Table 5

Optimization results for the two-bar truss problem (\(m=3\), \(n=4\))

Results

Method

Direct PDD (Univariate)

Direct PDD (Bivariate)

Direct PDD (Trivariate)

Sequential PDD (Univariate)

Sequential PDD (Bivariate)

Sequential PDD (Trivariate)

TPQa

Taylor seriesa

\(\tilde {d}_{1}^{\ast }\), \(\mathrm {cm}^{2}\)

11.5516

11.6439

11.6439

11.5650

11.6505

11.6498

11.6476

10.9573

\(\tilde {d}_{2}^{\ast }\), m

0.3805

0.3779

0.3779

0.3754

0.3763

0.3763

0.3767

0.3770

\(c_{0}(\tilde {\mathbf {d}}^{\ast })\)b

1.2393

1.2481

1.2481

1.2386

1.2481

1.2481

1.2480

1.1741

\(c_{1}(\tilde {\mathbf {d}}^{\ast })\)b

0.0095

0.0024

0.0025

0.0101

0.0024

0.0024

0.0025

0.0657

\(c_{2}(\tilde {\mathbf {d}}^{\ast })\)b

−0.4897

−0.4959

−0.4958

−0.4945

−0.4974

−0.4974

−0.4970

−0.4650

\(\mathbb {E}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]\)b, kg

12.3597

12.4480

12.4480

12.3538

12.4482

12.4477

12.4464

11.7150

\(\sqrt {\mathrm {var}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]}\)b, kg

2.4678

2.5025

2.5025

2.4836

2.5029

2.5028

2.5023

2.3542

No. of iterations

15

16

15

7

5

5

10

8

No. of \(y_{0}\) evaluations

195

976

1,875

91

305

625

2,503

108

Total no. of \(y_{1}\) & \(y_{2}\) evaluations

510

3,616

11,070

238

1,130

3,690

15,018

270

a The results of TPQ (DSA) were obtained from Lee et al. (2009)

b The objective and constraint functions, \(\mathbb {E}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]\), and \(\sqrt {\mathrm {var}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]}\) at respective optima, were evaluated by crude MCS (\(10^{8}\) samples)

6.3 Example 3: shape design of a three-hole bracket

The third example involves shape design optimization of a two-dimensional, three-hole bracket, where nine random shape parameters, \(X_{i}\), \(i=1,\ldots ,9\), describe its inner and outer boundaries, while maintaining symmetry about the central vertical axis. The design variables, \(d_{k}={\mathbb {E}_{\mathbf {d}}}[X_{k}]\), \(i=1,\ldots ,\)9, are the means of these independent random variables with Fig. 4a depicting the initial design of the bracket geometry at the mean values of the shape parameters. The bottom two holes are fixed, and a deterministic horizontal force \(F=15{,}000\) N is applied at the center of the top hole. The bracket material has a deterministic mass density \(\rho =7{,}810\)\(\mathrm {kg}/\mathrm {m}^{3}\), deterministic elastic modulus \(E=207.4\) GPa, deterministic Poisson’s ratio \(\nu =0.3\), and deterministic uniaxial yield strength \(S_{y}=800\) MPa. The objective is to minimize the second-moment properties of the mass of the bracket by changing the shape of the geometry such that the maximum von Mises stress \(\sigma _{e,\max }(\mathbf {X})\) does not exceed the yield strength \(S_{y}\) of the material with 99.875 % probability if \(y_{1}\) is Gaussian. Mathematically, the RDO for this problem is defined to
$$\begin{array}{rcl} \min_{\mathbf{d}\in\mathcal{D}}&\;& c_{0}(\mathbf{d})= 0.5\frac{{ \mathbb{E}_{\mathbf{d}}}\left[y_{0}(\mathbf{X})\right]}{{ \mathbb{E}_{\mathbf{d}_{0}}}\left[y_{0}(\mathbf{X})\right]}+0.5\frac{\sqrt{\mathrm{var}_{\mathbf{d}}\left[y_{0}(\mathbf{X})\right]}}{\sqrt{\mathrm{var}_{\mathbf{d}_{0}}\left[y_{0}(\mathbf{X})\right]}},\\ \mathrm{subject\: to}&\;& c_{1}(\mathbf{d})=3\sqrt{\mathrm{\mathrm{var}_{\mathbf{d}}}\left[y_{1}(\mathbf{X})\right]}-{ \mathbb{E}_{\mathbf{d}}}\left[y_{1}(\mathbf{X})\right]\le0,\\ && 0\;\mathrm{mm}\le d_{1}\le14\;\mathrm{mm},\\ &&17\;\mathrm{mm}\le d_{2}\le35\;\mathrm{mm},\\ && 10\;\mathrm{mm}\le d_{3}\le30\;\mathrm{mm},\\ &&30\;\mathrm{mm}\le d_{4}\le40\;\mathrm{mm},\\ && 12\;\mathrm{mm}\le d_{5}\le30\;\mathrm{mm},\\ &&12\;\mathrm{mm}\le d_{6}\le30\;\mathrm{mm},\\ && 50\;\mathrm{mm}\le d_{7}\le140\;\mathrm{mm},\\ &&-15\;\mathrm{mm}\le d_{8}\le10\;\mathrm{mm},\\ && -8\;\mathrm{mm}\le d_{9}\le15\;\mathrm{mm}, \end{array} $$
(47)
where
$$ y_{0}(\mathbf{X})=\rho\int_{\mathcal{D}^{\prime}(\mathbf{X})}d\mathcal{D}^{\prime}\; $$
(48)
and
$$ y_{1}(\mathbf{X})=S_{y}-\sigma_{e,\max}(\mathbf{X}) $$
(49)
are two random response functions, and \({\mathbb {E}_{\mathbf {d}_{0}}}[y_{0}(\mathbf {X})]\) and \({ \mathrm {var}_{\mathbf {d}_{0}}}[y_{0}(\mathbf {X})]\) are the mean and variance, respectively, of \(y_{0}\) at the initial design \(\mathbf {d}_{0}=(0,30,10,40,20,20,75,0,0)\)mm of the design vector \(\mathbf {d}=(d_{1},\ldots ,d_{9})\in \mathcal {D}\subset \mathbb {R}^{9}\). The corresponding mean and standard deviation of \(y_{0}\) of the original design, calculated by first-order bivariate PDD method, are 0.3415 kg and 0.00140 kg, respectively. Figure 4b portrays the contours of the von Mises stress calculated by FEA of the initial bracket design, which comprises 11,908 nodes and 3,914 eight-noded quadrilateral elements. A plane stress condition was assumed. The approximate optimal solution is denoted by \(\tilde {\mathbf {d}}^{\ast }=(\tilde {d}_{1}^{\ast },\ldots ,\tilde {d}_{9}^{\ast })\).
https://static-content.springer.com/image/art%3A10.1007%2Fs00158-013-0883-z/MediaObjects/158_2013_883_Fig4_HTML.gif
Fig. 4

A three-hole bracket; a design parametrization; b von Mises stress at initial design

Due to their finite bounds, the random variables \(X_{i}\), \(i=1,\ldots ,N\), were assumed to follow truncated Gaussian distributions with densities
$$ f_{X_{i}}(x_{i})=\begin{cases} \frac{\phi(\frac{x_{i}-d_{i}}{\sigma_{i}})}{\Phi(D_{i})-\Phi(-D_{i})}, & a_{i}\le x_{i}\le b_{i},\\ 0, & \text{otherwise,} \end{cases} $$
(50)
where \(\Phi (\cdot )\) and \(\phi (\cdot )\) are the cumulative distribution and probability density functions, respectively, of a standard Gaussian variable; \(\sigma _{i}=\sqrt {\mathrm {var}_{\mathbf {d}}[X_{i}]}=\sqrt {{ \mathbb {E}_{\mathbf {d}}[}(X_{i}-d_{i})^{2}]}=0.2\) is the standard deviation of \(X_{i}\); and \(a_{i}=d_{i}-D_{i}\) and \(b_{i}=d_{i}+D_{i}\) are the lower and upper bounds, respectively, of \(X_{i}\). To avoid unrealistic designs, the bounds were chosen to satisfy the following nine conditions: (1) \(D_{1}=(d_{2}-d_{1}-1)/2\); (2) \(D_{2}=\max [\min \{(d_{7}-d_{2}-2)/2,(d_{4}-d_{2}-2)/2\},2\sigma _{2}]\); (3) \(D_{3}=\min \{(d_{3}-2)/2,(30-d_{3}-2)/2\}\); (4) \(D_{4}=\min \{(d_{7}-d_{4}-2)/2,(d_{4}-d_{1}-2)/2\}\); (5) \(D_{5}=(d_{5}-11)/2\); (6) \(D_{6}=(d_{6}-11)/2\); (7) \(D_{7}=\min \{(d_{7}-d_{4}-2)/2,(150-d_{7}-5)/2\}\); (8) \(D_{8}=\max \{(25.57+d_{8}-d_{9})/2,2\sigma _{8}\}\); and (9) \(D_{9}=\max [\min \{(25.57+d_{8}-d_{9})/2,(12.912+d_{9})/2\},2\sigma _{9}]\). These conditions are consistent with the bound constraints of design variables stated in (47).
The proposed multi-point single-step PDD method was applied to solve this problem, employing three univariate and one bivariate PDD approximations for the underlying stochastic analysis: (1) \(S=1\), \(m=1\); (2) \(S=1\), \(m=2\); (3) \(S=1\), \(m=3\); and (4) \(S=2\), \(m=1\). Table 6 summarizes the optimization results by all four choices of the truncation parameters. The optimal design solutions rapidly converge as S or m increases. The univariate, first-order (\(S=1, m=1\)) PDD method, which is the most economical method, produces an optimal solution reasonably close to those obtained from higher-order or bivariate PDD methods. For instance, the largest deviation from the average values of the objective function at four optimum points is only 2.5 %. It is important to note that the coupling between single-step PDD and multi-point approximation is essential to find optimal solutions of this practical problem using low-variate, low-order PDD approximations.
Table 6

Optimization results for the three-hole bracket

Results

Multi-point single-step PDD method

Univariate (\(S=1,m=1\))

Univariate (\(S=1, m=2\))

Univariate (\(S=1, m=3\))

Bivariate (\(S=2, m=1\))

\(\tilde {d}_{1}^{\ast }\), mm

12.8168

13.6828

13.9996

13.9936

\(\tilde {d}_{2}^{\ast }\), mm

17.0112

17.0071

17.5236

17.0133

\(\tilde {d}_{3}^{\ast }\), mm

26.6950

28.3935

28.8053

28.6254

\(\tilde {d}_{4}^{\ast }\), mm

30.1908

30.2860

30.0009

30.0083

\(\tilde {d}_{5}^{\ast }\), mm

12.0069

12.0003

12.0000

12.0000

\(\tilde {d}_{6}^{\ast }\), mm

12.0003

12.0000

12.0000

12.0000

\(\tilde {d}_{7}^{\ast }\), mm

118.1200

118.0900

117.4930

117.7929

\(\tilde {d}_{8}^{\ast }\), mm

−13.7400

−13.8900

−13.8680

−13.9053

\(\tilde {d}_{9}^{\ast }\), mm

14.9124

14.9573

14.9991

14.9966

\(\tilde {c}_{0}(\tilde {\mathbf {d}}^{\ast })\)a

0.6686

0.6430

0.6364

0.6602

\(\tilde {c}_{1}(\tilde {\mathbf {d}}^{\ast })\)a

−1.6671

−0.8289

−1.8599

−8.8978

\(\mathbb {E}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]\)a, kg

0.1230

0.1185

0.1181

0.1176

\(\sqrt {\mathrm {var}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]}\)a, kg

0.00137

0.00132

0.00130

0.00137

No. of iterations

42

43

36

39

No. of FEA

798

1,204

1,332

6,357

a The objective and constraint functions, \(\mathbb {E}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]\), and \(\sqrt {\mathrm {var}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]}\) at respective optima, were evaluated by respective approximations

Figure 5a–d illustrate the contour plots of the von Mises stress for the four optimal designs at the mean values of random shape parameters. Regardless of S or m, the overall area of an optimal design has been substantially reduced, mainly due to significant alteration of the inner boundary and moderate alteration of the outer boundary of the bracket. All nine design variables have undergone moderate to significant changes from their initial values. The optimal masses of the bracket vary as 0.1230 kg, 0.1185 kg, 0.1181 kg, and 0.1186 kg—about a 65 % reduction from the initial mass of 0.3415 kg. Compared with the conservative design in Fig. 4b, larger stresses—for example, 800 MPa—are safely tolerated by the final designs in Fig. 5a–d.
https://static-content.springer.com/image/art%3A10.1007%2Fs00158-013-0883-z/MediaObjects/158_2013_883_Fig5_HTML.gif
Fig. 5

von Mises stress contours at mean values of optimal bracket designs by the multi-point single-step PDD method; a univariate approximation (\(S=1, m=1\)); b univariate approximation (\(S=1,m=2\)); c univariate approximation(\(S=1, m=3\)); d bivariate approximation (\(S=2, m=1\))

6.4 Example 4: shape design of a lever arm

The final example demonstrates the usefulness of the RDO methods advocated in designing an industrial-scale mechanical component, known as a lever arm, commonly found in wheel loaders, as shown in Fig. 6a. Twenty-two random shape parameters, \(X_{i}, i=1,\ldots ,22\), resulting from manufacturing variability, describe the shape of a lever arm in three dimensions, including two rounded quadrilateral holes introduced to reduce the mass of the lever arm as much as possible. The design variables, \(d_{k}={\mathbb {E}_{\mathbf {d}}}[X_{k}]\), \(k=1,\ldots ,\)22, are the means of these independent random variables, with Fig. 6b and c depicting the initial design of the lever arm geometry at mean values of the shape parameters. The centers of the central and right circular holes are fixed, and a deterministic horizontal force, \(F=1{,}600\) kN, was applied at the center of the left circular hole with a \(71.5^{\circ}\) angle from the horizontal line, as shown in Fig. 6b. These boundary conditions are determined from the interaction of the lever arm with other mechanical components of the wheel loader. The lever arm is made of cast steel with deterministic material properties, as follows: mass density \(\rho =7{,}800\)\(\mathrm {kg}/\mathrm {m}^{3}\), elastic modulus \(E=203\) GPa, Poisson’s ratio \(\nu =0.3\), fatigue strength coefficient \(\sigma _{f}^{\prime }=1{,}332\) MPa, fatigue strength exponent \(b=-0.1085\), fatigue ductility coefficient \(\epsilon _{f}^{\prime }=0.375\), and fatigue ductility exponent \(c=-0.6354\). The performance of the lever arm was determined by its fatigue durability obtained by (1) calculating maximum principal strain and mean stress at a point; and (2) calculating the fatigue crack-initiation life at that point from the well-known Coffin–Manson–Morrow equation (Stephens and Fuchs 2001). The objective is to minimize the second-moment properties of the mass of the lever arm by changing the shape of the geometry such that the minimum fatigue crack-initiation life \(N_{\text {min}}(\mathbf {X})\) exceeds a design threshold of \(N_{c}=10^{6}\) loading cycles with 99.875 % probability if \(y_{1}\) is Gaussian. Mathematically, the RDO for this problem is defined to
$$\begin{array}{rcl} \min_{\mathbf{d}\in\mathcal{D}}&\; & c_{0}(\mathbf{d})=0.5\frac{{ \mathbb{E}_{\mathbf{d}}}\left[y_{0}(\mathbf{X})\right]}{{ \mathbb{E}_{\mathbf{d}_{0}}}\left[y_{0}(\mathbf{X})\right]}+0.5\frac{\sqrt{\mathrm{var}_{\mathbf{d}}\left[y_{0}(\mathbf{X})\right]}}{\sqrt{\mathrm{var}_{\mathbf{d}_{0}}\left[y_{0}(\mathbf{X})\right]}},\\ \mathrm{subject\: to} &\;& c_{1}(\mathbf{d})=3\sqrt{\mathrm{\mathrm{var}_{\mathbf{d}}}\left[y_{1}(\mathbf{X})\right]}-{ \mathbb{E}_{\mathbf{d}}}\left[y_{1}(\mathbf{X})\right]\le0,\\ && 382\;\mathrm{mm}\le d_{1}\le458\;\mathrm{mm},\\ &&532\;\mathrm{mm}\le d_{2}\le563\;\mathrm{mm},\\ &&1{,}075\;\mathrm{mm}\le d_{3}\le1{,}185\;\mathrm{mm},\\ &&152\;\mathrm{mm}\le d_{4}\le178\;\mathrm{mm},\\ &&305\;\mathrm{mm}\le d_{5}\le795\;\mathrm{mm},\\ &&55\;\mathrm{mm}\le d_{6}\le357.5\;\mathrm{mm},\\ &&241\;\mathrm{mm}\le d_{7}\le630\;\mathrm{mm},\\ &&435\;\mathrm{mm}\le d_{8}\le689\;\mathrm{mm},\\ &&241\;\mathrm{mm}\le d_{9}\le630\;\mathrm{mm},\\ &&850\;\mathrm{mm}\le d_{10}\le1{,}023\;\mathrm{mm},\\ &&818\;\mathrm{mm}\le d_{11}\le1{,}131\;\mathrm{mm},\\ &&850\;\mathrm{mm}\le d_{12}\le1{,}013\;\mathrm{mm},\\ &&818\;\mathrm{mm}\le d_{13}\le1{,}131\;\mathrm{mm},\\ &&702\;\mathrm{mm}\le d_{14}\le748\;\mathrm{mm},\\ &&637\;\mathrm{mm}\le d_{15}\le755\;\mathrm{mm},\\ &&816\;\mathrm{mm}\le d_{16}\le888\;\mathrm{mm},\\ &&637\;\mathrm{mm}\le d_{17}\le755\;\mathrm{mm},\\ &&1{,}006\;\mathrm{mm}\le d_{18}\le1{,}116\;\mathrm{mm},\\ &&239\;\mathrm{mm}\le d_{19}\le447\;\mathrm{mm},\\ &&947\;\mathrm{mm}\le d_{20}\le1097\;\mathrm{mm},\\ &&257\;\mathrm{mm}\le d_{21}\le447\;\mathrm{mm},\\ &&505\;\mathrm{mm}\le d_{22}\le 833\;\mathrm{mm}, \end{array} $$
(51)
where
$$ y_{0}(\mathbf{X})=\rho\int_{\mathcal{D}^{\prime}(\mathbf{X})}d\mathcal{D}^{\prime}\; $$
(52)
and
$$ y_{1}(\mathbf{X})=N_{\text{min}}(\mathbf{X})-N_{c} $$
(53)
are two random response functions, and \({\mathbb {E}_{\mathbf {d}_{0}}}[y_{0}(\mathbf {X})]\) and \({\mathrm {var}_{\mathbf {d}_{0}}}[y_{0}(\mathbf {X})]\) are the mean and variance, respectively, of \(y_{0}\) at the initial design \(\mathbf {d}_{0}=(450\), 562, \(1{,}075\), 170, 795, 365, 630, 689, 630, 850, 818, 850, 818, 748, 637, 888, 637, \(1{,}006\), 447, 947, 447, \(833)\)mm of the design vector \(\mathbf {d}=(d_{1},\ldots ,d_{22})\in \mathcal {D}\subset \mathbb {R}^{22}\). Figure 7 portrays the FEA mesh for the initial lever arm design, which comprises 126,392 nodes and 75,114 ten-noded, quadratic, tetrahedral elements.
https://static-content.springer.com/image/art%3A10.1007%2Fs00158-013-0883-z/MediaObjects/158_2013_883_Fig6_HTML.gif
Fig. 6

Fatigue durability analysis of a lever arm in a wheel loader; a two lever arms; b design parametrization in front view; c design parametrization in top view

https://static-content.springer.com/image/art%3A10.1007%2Fs00158-013-0883-z/MediaObjects/158_2013_883_Fig7_HTML.gif
Fig. 7

An FEA mesh of a lever arm

As in Example 3, the random variables \(X_{i}, i=1,\ldots ,22\), are truncated Gaussian and have probability densities described by (50) with \(a_{i}=d_{i}-D_{i}\) and \(b_{i}=d_{i}+D_{i}\) denoting the lower and upper bounds, respectively. To avoid unrealistic designs, \(D_{i}=2\) when \(i=1,2,4,14,16\), and \(D_{i}=5\) otherwise.

The proposed multi-point single-step PDD method was applied to solve this lever arm design problem employing only a univariate, first-order PDD approximation, that is, selecting \(S=1\), \(m=1\), for second-moment analyses of \(y_{0}\) and \(y_{1}\). Figure 8a–d show the contour plots of the logarithm of fatigue crack-initiation life at mean shapes of several design iterations, including the initial design, throughout the RDO process. Due to a conservative initial design, with fatigue life contour depicted in Fig. 8a, the minimum fatigue crack-initiation life of \(1.068\times 10^{12}\) cycles is much larger than the required fatigue crack-initiation life of a million cycles. For the tolerance and subregion size parameters selected, 15 iterations and 675 FEA led to a final optimal design with the corresponding mean shape presented in Fig. 8d. The mean optimal mass of the lever arm is 1,263 kg—about a 79 % reduction from the initial mass of 6,036 kg. Correspondingly, the standard deviation of the mass drops from 2.1031 to 1.8016 kg.
https://static-content.springer.com/image/art%3A10.1007%2Fs00158-013-0883-z/MediaObjects/158_2013_883_Fig8_HTML.gif
Fig. 8

Contours of logarithmic fatigue crack-initiation life at mean shapes of the lever arm by the multi-point single-step PDD method; a iteration 1; b iteration 3; c iteration 9; d iteration 15 (optimum)

Figure 9a–d present the iteration histories of the objective function, constraint function, and 22 design variables during the RDO process. The objective function \(c_{0}\) is reduced from 0.9838 at initial design to 0.5238 at optimal design, an almost 50 % change. At optimum, the constraint function \(c_{1}\) is \(-0.0342\times 10^{6}\) cycles and is, therefore, close to being active. The design variables \(d_{5}\), \(d_{6}\), \(d_{7}\), \(d_{9}\), \(d_{19}\), and \(d_{21}\) have undergone the most significant changes from their initial values, prompting substantial modifications of the shapes or sizes of the rounded quadrilateral holes and thickness of the lever arm. The outer boundaries of the profile of the lever arm, controlled by the design variables \(d_{1}\), \(d_{2}\), \(d_{3}\), and \(d_{4}\) have undergone slight changes, because the initial design used is the result of a traditional deterministic optimization. This final example demonstrates that the RDO methods developed—in particular, the multi-point single-step PDD method—are capable of solving industrial-scale engineering design problems using only a few hundred FEA.
https://static-content.springer.com/image/art%3A10.1007%2Fs00158-013-0883-z/MediaObjects/158_2013_883_Fig9_HTML.gif
Fig. 9

RDO iteration histories for the lever arm; a objective function; b constraint function; c normalized design variables. Note: design variables are normalized with respect to their initial values

Table 7 lists percentage changes in the mean and standard deviation of \(y_{0}\) from initial to optimal designs in all four examples. The second-moment statistics at optimal designs are averages of all PDD solutions described earlier. Due to robust design, the largest reduction of the mean is 78.81 %, whereas the standard deviation diminishes by at most 93.80 %. The moderate drop in the standard deviations of Examples 2–4 is attributed to the objective function that combines both the mean and standard deviation of \(y_{0}\).
Table 7

Reductions in the mean and standard deviation of \(y_{0}\) from initial to optimal designs

Example

\(\frac {\mathbb {E}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]-\mathbb {E}_{\mathbf {d}_{0}}\left [y_{0}(\mathbf {X})\right ]}{\mathbb {E}_{\mathbf {d}_{0}}\left [y_{0}(\mathbf {X})\right ]} \)

\( \frac {\sqrt {\mathrm {var}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]}-\sqrt {\mathrm {var}_{\mathbf {d}_{0}}\left [y_{0}(\mathbf {X})\right ]}}{\sqrt {\mathrm {var}_{\mathbf {d}_{0}}\left [y_{0}(\mathbf {X})\right ]}} \)

1a

Not applicable

−93.80 %

2b

−12.51 %

−12.66 %

3c

−65.07 %

−4.35 %

4

−78.81 %

−14.34 %

a The value of \(\mathbb {E}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]\) and \(\sqrt {\mathrm {var}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]}\) is the average of all corresponding PDD results in Table 2

b The value of \(\mathbb {E}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]\) and \(\sqrt {\mathrm {var}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]}\) is the average of all corresponding PDD results in Tables 4 and 5

c The value of \(\mathbb {E}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]\) and \(\sqrt {\mathrm {var}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]}\) is the average of all corresponding PDD results in Table 6

7 Discussions

Since multiple methods and examples are presented in the paper, it is useful to summarize the efficiency and applicability of each method under different conditions. Table 8 presents such a summary, including a few qualitative comments inspired by the examples of the preceding section. Furthermore, the numerical results indicate the following:
  1. (1)

    The direct and single-step PDD methods generate identical optimal solutions for the polynomial functions, but the latter method is substantially more efficient than the former method;

     
  2. (2)

    The direct and sequential PDD methods, both employing univariate, bivariate, and trivariate PDD approximations, produce very close optimal solutions for the non-polynomial functions, but at vastly differing expenses. For either method, the univariate solution is accurate and most economical, even though the stochastic responses are multivariate functions. Given a PDD approximation, the sequential PDD method furnishes an optimal solution incurring at most half the computational cost of the direct PDD method;

     
  3. (3)

    For both polynomial and non-polynomial functions, the TPQ method, although accurate, is more expensive than most variants of the direct, single-step, and sequential PDD methods examined. Considering the non-polynomial functions, the univariate direct PDD and univariate sequential PDD methods are more economical than the TPQ method by an order of magnitude or more;

     
  4. (4)

    The multi-point single-step PDD method employing low-variate or low-order PDD approximations, including a univariate, first-order PDD approximation, is able to solve practical engineering problems with a reasonable computational effort.

     
Table 8

Efficiency and applicability of the four proposed methods

Method

Efficiency

Applicability

Comments

Direct PDD

Low

Both polynomial and non-polynomial functions with small design spaces

Expensive due to recalculation of expansion coefficients. Impractical for complex system designs.

Single-step PDD

Highest

Low-order polynomial functions with small design spaces

Highly economical due to recycling of expansion coefficients, but may produce premature solutions coefficients, but may produce premature solutions for complex system designs.

Sequential PDD

Medium

Polynomial or non-polynomial functions with small to medium design spaces

More expensive than single-step PDD, but substantially more economical than direct PDD. May require high-variate and high-order PDD approximations for complex system designs.

Multi-point single-step PDD

High

Polynomial or non-polynomial functions with large design spaces

Capable of solving complex, practical design problems using low-variate and/or low-order PDD approximations.

8 Conclusions

Four new methods are proposed for robust design optimization of complex engineering systems. The methods involve PDD of a high-dimensional stochastic response for statistical moment analysis, a novel integration of PDD and score functions for calculating the second-moment sensitivities with respect to the design variables, and standard gradient-based optimization algorithms, encompassing direct, single-step, sequential, and multi-point single-step design processes. Because they are rooted in ANOVA dimensional decomposition, the PDD approximations for arbitrary truncations predict the exact mean and generate a convergent sequence of variance approximations for any square-integrable function. When blended with score functions, PDD leads to explicit formulae, expressed in terms of the expansion coefficients, for approximating the second-moment design sensitivities that are also theoretically convergent. More importantly, the statistical moments and design sensitivities are both determined concurrently from a single stochastic analysis or simulation.

Among the four design methods developed, the direct PDD method is the simplest of all, but requires re-calculations of the expansion coefficients at each design iteration and is, therefore, expensive, depending on the cost of evaluating the objective and constraint functions and the requisite number of design iterations. The single-step PDD method eliminates the need to re-calculate the expansion coefficients from scratch by recycling the old expansion coefficients, consequently holding a potential to significantly curtail the computational effort. However, it depends heavily on the quality of a PDD approximation and the accuracy of the estimated expansion coefficients during design iterations. The sequential PDD method upholds the merits of both the direct and single-step PDD methods by re-calculating the expansion coefficients a few times more than the single-step PDD, incurring a computational complexity that is lower than the direct PDD method. However, all three methods just described are global and may not work if the design space is too large for a PDD approximation, with a chosen degree of interaction or expansion order, to be sufficiently accurate. The multi-point single-step PDD method mitigates this problem by adopting a local implementation of PDD approximations, where an RDO problem with a large design space is solved in succession. Precisely for this reason, the method is capable of solving practical engineering problems using low-order and/or low-variate PDD approximations of stochastic responses.

Footnotes
1

The nouns sensitivity, derivative, and gradient are used synonymously in this paper.

 

Copyright information

© Springer-Verlag Berlin Heidelberg 2013