# Robust design optimization by polynomial dimensional decomposition

## Authors

- First Online:

- Received:
- Revised:
- Accepted:

DOI: 10.1007/s00158-013-0883-z

- Cite this article as:
- Ren, X. & Rahman, S. Struct Multidisc Optim (2013) 48: 127. doi:10.1007/s00158-013-0883-z

- 2 Citations
- 356 Views

## Abstract

This paper introduces four new methods for robust design optimization (RDO) of complex engineering systems. The methods involve polynomial dimensional decomposition (PDD) of a high-dimensional stochastic response for statistical moment analysis, a novel integration of PDD and score functions for calculating the second-moment sensitivities with respect to the design variables, and standard gradient-based optimization algorithms. New closed-form formulae are presented for the design sensitivities that are simultaneously determined along with the moments. The methods depend on how statistical moment and sensitivity analyses are dovetailed with an optimization algorithm, encompassing direct, single-step, sequential, and multi-point single-step design processes. Numerical results indicate that the proposed methods provide accurate and computationally efficient optimal solutions of RDO problems, including an industrial-scale lever arm design.

### Keywords

Design under uncertaintyANOVA dimensional decompositionOrthogonal polynomialsScore functionsOptimization## 1 Introduction

Robust design optimization (RDO) constitutes a mathematical framework for solving design problems in the presence of uncertainty, manifested by statistical descriptions of the objective and/or constraint functions (Taguchi 1993; Chen et al. 1996; Du and Chen 2000; Mourelatos and Liang 2006; Zaman et al. 2011; Park et al. 2006). Aimed at improving product quality, it minimizes the propagation of input uncertainty to output responses of interest, leading to an insensitive design. RDO, pioneered by Taguchi (1993), is being increasingly viewed as an enabling technology for design of aerospace, civil, and automotive structures subject to uncertainty (Chen et al. 1996; Du and Chen 2000; Mourelatos and Liang 2006; Zaman et al. 2011; Park et al. 2006).

The objective or constraint functions in RDO often involve second-moment properties, such as means and standard deviations, of stochastic responses, describing the statistical performance of a given design. Therefore, solving an RDO problem draws in uncertainty quantification of random responses and its coupling with gradient-based optimization algorithms, consequently demanding a greater computational effort than that required by a deterministic design optimization. There exist three principal concerns or shortcomings when conducting RDO with existing approaches or techniques. First, the commonly used stochastic methods, including the Taylor series or perturbation expansions (Huang and Du 2007), point estimate method (Huang and Du 2007), polynomial chaos expansion (PCE) (Wang and Kim 2006), tensor-product quadrature rule (Lee et al. 2009), and dimension-reduction methods (Lee et al. 2008, 2009) may not be adequate or applicable for uncertainty quantification of many large-scale practical problems. For instance, the Taylor series expansion and point estimate methods, although simple and inexpensive, begin to break down when the input-output mapping is highly nonlinear or the input uncertainty is arbitrarily large. Furthermore, truly high-dimensional problems are all but impossible to solve using the PCE and tensor-product quadrature rule due to the curse of dimensionality. The dimension-reduction methods, developed by the author’s group (Rahman and Xu 2004; Xu and Rahman 2004), including a modification (Youn et al. 2008), alleviate the curse of dimensionality to some extent, but they are rooted in the referential dimensional decomposition, resulting in sub-optimal approximations of a multivariate function (Rahman 2011, 2012). Second, many of the aforementioned methods invoke finite-difference techniques to calculate design sensitivities of the statistical moments. They demand repeated stochastic analyses for nominal and perturbed values of design parameters and are, therefore, expensive and unwieldy. Although some methods, such as Taylor series expansions, also provide the design sensitivities economically, the sensitivities are either inaccurate or unreliable because they inherit errors from the affiliated second-moment analysis. Therefore, alternative stochastic methods should be explored for calculating the statistical moments and design sensitivities as accurately as possible and simultaneously, but without the computational burden of crude Monte Carlo simulation (MCS). Third, existing methods for solving RDO problems permit the objective and constraint functions and their sensitivities to be calculated only at a fixed design, requiring new statistical moment and sensitivity analyses at every design iteration until convergence is attained. Consequently, the current RDO methods, entailing expensive finite-element analysis (FEA) or similar numerical calculations, are computationally intensive, if not prohibitive, when confronted with a large number of design or random variables. New or significantly improved design paradigms, possibly requiring a single or a few stochastic simulations, are needed for solving the entire RDO problem. Further complications may arise when an RDO problem is formulated in conjunction with a multi-point approximation (Toropov et al. 1993)—a setting frequently encountered when tackling a practical optimization problem with a large design space. In which case, one must integrate stochastic analysis, design sensitivity analysis, and optimization algorithms on a local subregion of the entire design space.

This paper presents four new methods for robust design optimization of complex engineering systems. The methods are based on: (1) polynomial dimensional decomposition (PDD) of a high-dimensional stochastic response for statistical moment analysis; (2) a novel integration of PDD and score functions for calculating the second-moment sensitivities with respect to design variables; and (3) standard gradient-based optimization algorithms, encompassing direct, single-step, sequential, and multi-point single-step design processes. Section 2 formally defines a general RDO problem, including a concomitant mathematical statement. Section 3 starts with a brief exposition of the analysis-of-variance (ANOVA) dimensional decomposition and explains how it leads up to PDD approximations, resulting in explicit formulae for the first two moments of a generic stochastic response. The calculation of the expansion coefficients by dimension-reduction integration is also briefly described. Section 4 defines score functions and unveils new closed-form formulae for the design sensitivities of the first two moments, determined from a single stochastic analysis. Section 5 introduces four new design methods and explains how the stochastic analysis and design sensitivities from a PDD approximation are integrated with a gradient-based optimization algorithm in each method. Section 6 presents four numerical examples involving mathematical functions or solid-mechanics problems, contrasting the accuracy, convergence properties, and computational efforts of the proposed RDO methods. It is followed by Section 7, which discusses the efficiency and applicability of all four methods. Finally, the conclusions are drawn in Section 8.

## 2 Robust design optimization

Let \(\mathbb {N}\), \(\mathbb {N}_{0}\), \(\mathbb {R}\), and \(\mathbb {R}_{0}^{+}\) represent the sets of positive integer (natural), non-negative integer, real, and non-negative real numbers, respectively. For \(k\in \mathbb {N}\), denote by \(\mathbb {R}^{k}\) the *k*-dimensional Euclidean space and by \(\mathbb {N}_{0}^{k}\) the *k*-dimensional multi-index space. These standard notations will be used throughout the paper.

Consider a measurable space \((\Omega ,\mathcal {F})\), where \(\Omega \) is a sample space and \(\mathcal {F}\) is a \(\sigma \)-field on \(\Omega \). Defined over \((\Omega ,\mathcal {F})\), let \(\{P_{\mathbf {d}}:\mathcal {F}\to [0,1]\}\) be a family of probability measures, where for \(M\in \mathbb {N}\) and \(N\in \mathbb {N}\), \(\mathbf {d}=(d_{1},\ldots ,d_{M})\in \mathcal {D}\) is an \(\mathbb {R}^{M}\)-valued design vector with non-empty closed set \(\mathcal {D}\subseteq \mathbb {R}^{M}\) and \(\mathbf {X}:=(X_{1},\ldots ,X_{N}):(\Omega ,\mathcal {F})\to (\mathbb {R}^{N},\mathcal {B}^{N})\) be an \(\mathbb {R}^{N}\)-valued input random vector with \(\mathcal {B}^{N}\) representing the Borel \(\sigma \)-field on \(\mathbb {R}^{N}\), describing the statistical uncertainties in loads, material properties, and geometry of a complex mechanical system. The probability law of \(\mathbf {X}\) is completely defined by a family of the joint probability density functions \(\{f_{\mathbf {X}}(\mathbf {x};\mathbf {d}),\:\mathbf {x}\in \mathbb {R}^{N},\:\mathbf {d}\in \mathcal {D}\}\) that are associated with probability measures \(\{P_{\mathbf {d}},\:\mathbf {d}\in \mathcal {D}\}\), so that the probability triple \((\Omega ,\mathcal {F},P_{\mathbf {d}})\) of \(\mathbf {X}\) depends on \(\mathbf {d}\). A design variable \(d_{k}\) can be any distribution parameter or a statistic–for instance, the mean or standard deviation–of \(X_{i}\).

## 3 Statistical moment analysis

Let \(y(\mathbf {X}):=y(X_{1},\ldots ,X_{N}\)) represent any one of the random functions \(y_{l}\), \(l=0,1,\ldots ,K\), introduced in Section 2, and let \(\mathcal {L}_{2}(\Omega ,\mathcal {F},P_{\mathbf {d}})\) represent a Hilbert space of square-integrable functions, including *y*, with respect to the probability measure \(f_{\mathbf {X}}(\mathbf {x};\mathbf {d})d\mathbf {x}\) supported on \(\mathbb {R}^{N}\). Assuming independent coordinates of \(\mathbf {X}\), its joint probability density is expressed by a product, \(f_{\mathbf {\mathbf {X}}}(\mathbf {x};\mathbf {d})=\prod _{i=1}^{i=N}f_{X_{i}}(x_{i};\mathbf {d})\), of marginal probability density functions \(f_{X_{i}}:\mathbb {R}\to \mathbb {R}_{0}^{+}\) of \(X_{i}\), \(i=1,\ldots ,N\), defined on its probability triple \((\Omega _{i},\mathcal {F}_{i},P_{i,\mathbf {d}})\) with a bounded or an unbounded support on \(\mathbb {R}\). Then, for a given subset \(u\subseteq \{1,\ldots ,N\}\), \(f_{\mathbf {X}_{-u}}(\mathbf {x}_{-u};\mathbf {d}):=\prod _{i=1,i\notin u}^{N}f_{i}(x_{i};\mathbf {d})\) defines the marginal density function of \(\mathbf {X}_{-u}:=\mathbf {X}_{\{1,\ldots ,N\}\backslash u}\).

### 3.1 ANOVA dimensional decomposition

*y*in terms of its input variables with increasing dimensions, where \(u\subseteq \{1,\ldots ,N\}\) is a subset with the complementary set \(-u=\{1,\ldots ,N\}\backslash u\) and cardinality \(0\le |u|\le N\), and \(y_{u}\) is a \(|u|\)-variate component function describing a constant or the interactive effect of \(\mathbf {X}_{u}=(X_{i_{1}},\ldots ,X_{i_{|u|}})\), \(1\leq i_{1}<\ldots <i_{|u|}\leq N\), a subvector of \(\mathbf {X}\), on

*y*when \(|u|=0\) or \(|u|>0\). The summation in (3) comprises \(2^{N}\) terms, with each term depending on a group of variables indexed by a particular subset of \(\{1,\ldots ,N\}\), including the empty set \(\emptyset \). In (5), \((\mathbf {X}_{u},\mathbf {x}_{-u})\) denotes an

*N*-dimensional vector whose

*i*th component is \(X_{i}\) if \(i\in u\) and \(x_{i}\) if \(i\notin u\). When \(u=\emptyset \), the sum in (5) vanishes, resulting in the expression of the constant function \(y_{\emptyset }\) in (4). When \(u=\{1,\ldots ,N\}\), the integration in (5) is on the empty set, reproducing (3) and hence finding the last function \(y_{\{1,\ldots ,N\}}\). Indeed, all component functions of

*y*can be obtained by interpreting literally (5).

### Remark 1

### Remark 2

The coefficient \(y_{\emptyset }=\mathbb {E}_{\mathbf {d}}[y(\mathbf {X})]\) in (4) is a function of the design vector \(\mathbf {d}\), which describes the probability distribution of the random vector \(\mathbf {X}\). Therefore, the adjective “constant” used to describe \(y_{\emptyset }\) should be interpreted with respect to \(\mathbf {X}\), not \(\mathbf {d}\). A similar condition applies for the component functions \(y_{u}\), \(\emptyset \ne u\subseteq \{1,\ldots ,N\}\), which also depend on \(\mathbf {d}\).

### 3.2 Polynomial dimensional decomposition

#### 3.2.1 Orthonormal polynomials

Let \(\{\psi _{ij}(x_{i};\mathbf {d});\; j=0,1,\ldots \}\) be a set of univariate, orthonormal polynomial basis functions in the Hilbert space \(\mathcal {L}_{2}(\Omega _{i},\mathcal {F}_{i},P_{i,\mathbf {d}})\) that is consistent with the probability measure \(P_{i,\mathbf {d}}\) or \(f_{X_{i}}(x_{i};\mathbf {d})dx_{i}\) of \(X_{i}\) for a given design \(\mathbf {d}\). For \(\emptyset \ne u=\{i_{1},\ldots ,i_{|u|}\}\subseteq \{1,\ldots ,N\}\), where 1 \(\le \)\(|u|\)\(\le \)*N* and 1 \(\le \)\(i_{1}\)\(<\)\(\ldots \)\(<\)\(i_{|u|}\)\(\le \)*N*, let \((\times _{p=1}^{p=|u|}\Omega _{i_{p}}\), \(\times _{p=1}^{p=|u|}\mathcal {F}_{i_{p}}\), \(\times _{p=1}^{p=|u|}P_{i_{p}, \mathbf {d}})\) be the product probability triple of \(\mathbf {X}_{u}=(X_{i_{1}}\), \(\ldots \), \(X_{i_{|u|}})\). Denote the associated space of the \(|u|\)-variate component functions of *y* by \(\mathcal {L}_{2}(\times _{p=1}^{p=|u|}\Omega _{i_{p}}\), \(\times _{p=1}^{p=|u|}\mathcal {F}_{i_{p}}\), \(\times _{p=1}^{p=|u|}P_{i_{p}, \mathbf {d}})\)\(:=\)\(\{y_{u}:\int _{\mathbb {R}^{|u|}}y_{u}^{2}\)\((\mathbf {x}_{u};\mathbf {d})f_{\mathbf {X}_{u}}(\mathbf {x}_{u};\mathbf {d})d\mathbf {x}_{u}<\infty \}\), which is a Hilbert space. Since the joint density of \(\mathbf {X}_{u}\) is separable (independence of \(X_{i},i\in u\)), that is, \(f_{\mathbf {X}_{u}}(\mathbf {x}_{u};\mathbf {d})={\textstyle \prod _{p=1}^{|u|}}f_{X_{i_{p}}}(x_{i_{p}};\mathbf {d})dx_{i_{p}}\), the product \(\psi _{u\mathbf {j}_{|u|}}(\mathbf {X}_{u};\mathbf {d}):=\prod _{p=1}^{|u|}\psi _{i_{p}j_{p}}(X_{i_{p}};\mathbf {d})\), where \(\mathbf {j}_{|u|}=\)\((j_{1},\ldots ,j_{|u|})\in \mathbb {N}_{0}^{|u|}\), a \(|u|\)-dimensional multi-index with \(\infty \)-norm \(||\mathbf {j}_{|u|}||_{\infty }=\max (j_{1},\ldots ,j_{|u|}\)), constitutes a multivariate orthonormal polynomial basis in \(\mathcal {L}_{2}(\times _{p=1}^{p=|u|}\Omega _{i_{p}}\), \(\times _{p=1}^{p=|u|}\mathcal {F}_{i_{p}}\), \(\times _{p=1}^{p=|u|}P_{i_{p},\mathbf {d}})\). Two important properties of these product polynomials from tensor products of Hilbert spaces are as follows.

### Proposition 1

### Proposition 2

### Proof

The results of Propositions 1 and 2 follow by recognizing independent coordinates of \(\mathbf {X}\) and using the second-moment properties of univariate orthonormal polynomials: (1) \(\mathbb {E}_{\mathbf {d}}[\psi _{ij}(X_{i};\mathbf {d})]=1\) when \(j=0\) and *zero* when \(j\ge 1\); and (2) \(\mathbb {E}_{\mathbf {d}}[\psi _{ij_{1}}(X_{i};\mathbf {d})\psi _{ij_{2}}(X_{i};\mathbf {d})]=1\) when \(j_{1}=j_{2}\) and *zero* when \(j_{1}\neq j_{2}\) for an arbitrary random variable \(X_{i}\). □

### Remark 3

Given a probability measure \(P_{i,\mathbf {d}}\) of any random variable \(X_{i}\), the well-known three-term recurrence relation is commonly used to construct the associated orthogonal polynomials (Rahman 2009a; Gautschi 2004). For \(m\in \mathbb {N}\), the first *m* recursion coefficient pairs are uniquely determined by the first \(2m\) moments of \(X_{i}\) that must exist. When these moments are exactly calculated, they lead to exact recursion coefficients, some of which belong to classical orthogonal polynomials. For an arbitrary probability measure, approximate methods, such as the Stieltjes procedure, can be employed to obtain the recursion coefficients (Rahman 2009a; Gautschi 2004).

#### 3.2.2 Stochastic expansion

*y*in terms of an infinite number of coefficients or orthonormal polynomials. In practice, the number of coefficients or polynomials must be finite, say, by retaining at most

*m*th-order polynomials in each variable. Furthermore, in many applications, the function

*y*can be approximated by a sum of at most

*S*-variate component functions, where \(S\in \mathbb {N};1\le S\le N\), resulting in the

*S*-variate,

*m*th-order PDD approximation

*S*input variables \(X_{i_{1}},\ldots ,X_{i_{S}}\), \(1\le i_{1}\le \ldots \le i_{S}\le N\). For instance, by selecting \(S=1\) and 2, the functions

*m*th-order PDD approximations, contain contributions from all input variables, and should not be viewed as first- and second-order approximations, nor as limiting the nonlinearity of

*y*. Depending on how the component functions are constructed, arbitrarily high-order univariate and bivariate terms of

*y*could be lurking inside \(\tilde {y}_{1,m}\) and \(\tilde {y}_{2,m}\). When \(S\to N\) and \(m\to \infty \), \(\tilde {y}_{S,m}\) converges to

*y*in the mean-square sense, permitting (12) to generate a hierarchical and convergent sequence of approximations of

*y*. Readers interested in further details of PDD are referred to the authors’ past works (Rahman 2008, 2009a).

### 3.3 Statistical moments

*S*-variate,

*m*th-order PDD approximation matches the exact mean \(\mathbb {E}_{\mathbf {d}}\left [y(\mathbf {X})\right ]=y_{\emptyset }(\mathbf {d})\), regardless of

*S*or

*m*. Applying the expectation operator again, this time on \([\tilde {y}_{S,m}(\mathbf {X})-y_{\emptyset }(\mathbf {d})]^{2}\), and recognizing Propositions 1 and 2, results in the approximate variance (Rahman 2010)

*S*-variate,

*m*th-order PDD approximation of \(y(\mathbf {X})\). Clearly, the approximate variance in (16) approaches the exact variance

*y*when \(S\to N\) and \(m\to \infty \). The mean-square convergence of \(\tilde {y}_{S,m}\) is guaranteed as

*y*and its component functions are all members of the associated Hilbert spaces.

*m*, and progressively improve as

*S*becomes larger. Recent works on error analysis indicate that the second-moment properties obtained from the ANOVA dimensional decomposition, which leads to PDD approximations, are superior to those derived from dimension-reduction methods that are grounded in the referential dimensional decomposition (Rahman 2011, 2012). Therefore, employing PDD for solving RDO problems contributes to development of a new, significant design paradigm.

### 3.4 Expansion coefficients

The determination of the expansion coefficients \(y_{\emptyset }\) and \(C_{u\mathbf {j}_{|u|}}\) in (4) and (10), respectively, of the stochastic responses involves various *N*-dimensional integrals over \(\mathbb {R}^{N}\). For large *N*, a full numerical integration employing an *N*-dimensional tensor product of a univariate quadrature formula is computationally prohibitive. Instead, a dimension-reduction integration scheme can be applied to estimate the coefficients efficiently.

The dimension-reduction integration, originally developed by Xu and Rahman (2004), entails approximating a high-dimensional integral of interest by finite-sum lower-dimensional integrations. For calculating the expansion coefficients \(y_{\emptyset }\) and \(C_{u\mathbf {j}_{|u|}}\), this is accomplished by replacing the *N*-variate function *y* in (4) and (10) with an *R*-variate truncation, where \(R<N\), of its referential dimensional decomposition at a chosen reference point (Rahman 2011, 2012). The result is a reduced integration scheme, requiring evaluations of at most *R*-dimensional integrals. The scheme facilitates calculation of the coefficients approaching their exact values as \(R\to N\), and is significantly more efficient than performing one *N*-dimensional integration, particularly when \(R\ll N\). Hence, the computational effort is significantly lowered using the dimension-reduction integration. When \(R=1\) or 2, the scheme involves one-, or, at most, two-dimensional integrations, respectively. Nonetheless, numerical integration is still required for a general integrand. The Gauss-type quadrature rule was used to perform integrations. The integration points and associated weights, which depend on the probability distribution of \(X_{i}\), are readily available when the basis functions are polynomials (Gautschi 2004; Rahman 2009a). Further details are available elsewhere (Xu and Rahman 2004).

The *S*-variate, *m*th-order PDD approximation requires evaluations of \(Q_{S,m}=\sum\limits_{k=0}^{k=S}\binom {N}{S-k}m^{S-k}=\sum\limits_{k=0}^{k=S}\binom {N}{k}m^{k}\) expansion coefficients, including \(y_{\emptyset }\). If these coefficients are estimated by dimension-reduction integration with \(R=S<N\) and, therefore, involve at most *S*-dimensional tensor product of an *n*-point univariate quadrature rule depending on *m*, then the total cost for the *S*-variate, *m*th-order approximation entails a maximum of \(\sum_{k=0}^{k=S}\binom {N}{S-k}n^{S-k}(m)=\sum _{k=0}^{k=S}\binom {N}{k}n^{k}(m)\) function evaluations. If the integration points include a common point in each coordinate – a special case of symmetric input probability density functions and odd values of *n* – the number of function evaluations reduces to \(\sum _{k=0}^{k=S}\binom {N}{S-k}(n(m)-1)^{S-k}=\sum _{k=0}^{k=S}\binom {N}{k}(n(m)-1)^{k}\). Nonetheless, the computational complexity of the *S*-variate PDD approximation is *S*th-order polynomial with respect to the number of random variables or integration points. Therefore, PDD alleviates the curse of dimensionality to some extent.

## 4 Proposed methods for design sensitivity analysis^{1}

When solving design problems using gradient-based optimization algorithms, at least first-order derivatives of both the objective and constraint functions with respect to each design variable are required. For an RDO problem defined by (2), calculating such derivatives is trivial once the derivatives of the first two moments of \(y_{l}(\mathbf {X})\), \(l=0,1,\ldots ,K\), are known. In this subsection, a new, analytical method, developed by blending PDD with score functions, for design sensitivity analysis is presented.

### 4.1 Score functions

### Remark 4

The evaluation of score functions, \(\partial \ln f_{\mathbf {X}}(\mathbf {X}\); \(\mathbf {d})\left /\partial d_{k}\right .\); \(k=1,\ldots ,M\), requires differentiating only the probability density function of \(\mathbf {X}\). Therefore, the resulting score functions can be determined easily and, in many cases, analytically—for instance, when \(\mathbf {X}\) follows classical probability distributions (Rahman 2009b). If the density function of \(\mathbf {X}\) is arbitrarily prescribed, the score functions can be calculated numerically, yet inexpensively, since no evaluation of the response function is involved.

*k*th design variable simplifies to \(\partial \ln f_{\mathbf {X}}(\mathbf {x};\mathbf {d})\left /\partial d_{k}\right .=\partial \ln f_{X_{i_{k}}}(X_{i_{k}};\mathbf {d})\left /\partial d_{k}\right .\), which is a univariate function of \(X_{i_{k}}\). Defining \(s_{k}(X_{i_{k}};\mathbf {d}):=\partial \ln f_{X_{i_{k}}}(X_{i_{k}};\mathbf {d})\left /\partial d_{k}\right .\) as the

*k*th first-order score function, the sensitivity is obtained from

### 4.2 Exact sensitivities

*k*th score function

### 4.3 Approximate sensitivities

*S*-variate,

*m*th-order PDD and \(m^{\prime}\)th-order Fourier-polynomial approximations, respectively, the resultant sensitivity equations, expressed by

*S*,

*m*, and \(m^{\prime}\) in general. It is elementary to show that when \(S=N\) and \(m=m^{\prime}=\infty \), \(\mathrm {var}_{\mathbf {d}}[\tilde {y}_{S,m}(\mathbf {X})]=\mathrm {var}_{\mathbf {d}}[y(\mathbf {X})]\) and \(\tilde {T}_{k,m,m^{\prime}}=T_{k}\). Therefore, the approximate sensitivities of the moments also converge to exactness when \(S\to N\) and \(m_{\min }\to \infty \).

*S*, meaning that both the univariate (\(S=1\)) and bivariate (\(S=2\)) approximations, given the same \(m_{\min }<\infty \), form the same result, as displayed in (30). However, the sensitivity equations of \(\partial \mathbb {E}_{\mathbf {d}}[\tilde {y}_{S,m}^{2}(\mathbf {X})]/\partial d_{k}\) for the univariate and bivariate approximations vary with respect to

*S*,

*m*, and \(m^{\prime}\). For instance, the univariate approximation results in

*y*, respectively. Since the expansion coefficients of the score function do not involve the response function, no additional cost is incurred from response analysis. In other words, the effort required to obtain the statistical moments of a response also furnish the sensitivities of moments, a highly desirable trait for efficiently solving RDO problems.

### Remark 5

Since the score functions are univariate functions, their expansion coefficients require only univariate integration for their evaluations. When \(X_{i}\) follows classical distributions—for instance, the Gaussian distribution—then the coefficients can be calculated exactly or analytically. Otherwise, numerical quadrature is required. Nonetheless, there is no need to employ dimension-reduction integration for calculating the expansion coefficients of the score functions.

## 5 Proposed methods for design optimization

The PDD approximations described in the preceding section provide a means to approximate the objective and constraint functions, including their design sensitivities, from a single stochastic analysis. Therefore, any gradient-based algorithm employing PDD approximations should render a convergent solution of the RDO problem in (2). However, there exist multiple ways to dovetail stochastic analysis with an optimization algorithm. Four such design optimization methods, all anchored in PDD, are presented in this section.

### 5.1 Direct PDD

The direct PDD method involves straightforward integration of the PDD-based stochastic analysis with design optimization. Given a design vector at the current iteration and the corresponding values of the objective and constraint functions and their sensitivities, the design vector at the next iteration is generated from a suitable gradient-based optimization algorithm. However, new statistical moment and sensitivity analyses, entailing re-calculations of the PDD expansion coefficients, are needed at every design iteration. Therefore, the direct PDD method is expensive, depending on the cost of evaluating the objective and constraint functions and the requisite number of design iterations.

### 5.2 Single-step PDD

The single-step PDD method is motivated on solving the entire RDO problem from a single stochastic analysis by sidestepping the need to recalculate the PDD expansion coefficients at every design iteration. It subsumes two important assumptions: (1) an *S*-variate, *m*th-order PDD approximation \(\tilde {y}_{S,m}\) of *y* at the initial design is acceptable for all possible designs; and (2) the expansion coefficients for one design, derived from those generated for another design, are accurate.

*y*with the right side of (11) in (4) and (10). However, in practice, when the

*S*-variate,

*m*th-order PDD approximation (12) is used to replace

*y*in (4) and (10), then the new expansion coefficients, and which are applicable for \(\emptyset \ne u\subseteq \{1,\ldots ,N\}\), \(1\le |u|\le S\), become approximate, although convergent. In the latter case, the integrals in (37) and (38) consist of finite-order polynomial functions of at most

*S*variables and can be evaluated inexpensively without having to compute the original function

*y*for the new design. Therefore, new stochastic analyses, all employing

*S*-variate,

*m*th-order PDD approximation of

*y*, are conducted with little additional cost during all design iterations, drastically curbing the computational effort in solving the RDO problem.

### 5.3 Sequential PDD

When the truncations parameters, *S* and/or *m*, of a PDD approximation are too low, the assumptions of the single-step PDD method are likely to be violated, resulting in a premature or an inaccurate optimal solution. To overcome this problem, a sequential PDD method, combining the ideas of the single-step PDD and direct PDD methods, was developed. It forms a sequential design process, where each sequence begins with a single-step PDD using the expansion coefficients calculated at an optimal design solution generated from the previous sequence. Although more expensive than the single-step PDD method, the sequential PDD method is expected to be more economical than the direct PDD method.

- Step 1:
Select an initial design vector \(\mathbf {d}_{0}\). Define a tolerance \(\epsilon >0\). Set the iteration \(q=1\),

*q*th initial design vector \(\mathbf {d}_{0}^{(q)}=\mathbf {d}_{0}\), and approximate optimal solution \(\mathbf {d}_{\ast }^{(0)}=\mathbf {d}_{0}\) at \(q=0\). - Step 2:
Select (\(q=1\)) or use (\(q>1\)) the PDD and Fourier truncation parameters

*S*,*m*, and \(m^{\prime}\). At \(\mathbf {d}=\mathbf {d}_{0}^{(q)}\), generate the PDD expansion coefficients, \(y_{\emptyset }(\mathbf {d})\) and \(C_{u\mathbf {j}_{|u|}}(\mathbf {d})\), where \(\emptyset \ne u\subseteq \{1,\ldots ,N\}\), \(1\le |u|\le S\), \(\mathbf {j}_{|u|}\in \mathbb {N}_{0}^{|u|}\), \(||\mathbf {j}_{|u|}||_{\infty }\le m\), \(j_{1},\ldots ,j_{|u|}\neq 0\), using dimension-reduction integration with \(R=S\), \(n=m+1\), leading to*S*-variate,*m*th-order PDD approximations of \(y_{l}(\mathbf {X})\), \(l=0,1,\ldots ,K\), in (2). Calculate the expansion coefficients of the score functions, \(s_{k,\emptyset }(\mathbf {d})\) and \(D_{i_{k},j}(\mathbf {d})\), where \(k=1,\ldots ,M\) and \(j=1,\ldots ,m^{\prime}\), analytically, if possible, or numerically, resulting in \(m^{\prime}\)th-order Fourier-polynomial approximations of \(s_{k}(X_{i_{k}};\mathbf {d})\), \(k=1,\ldots ,M\). - Step 3:
Solve the design problem in (2) employing PDD approximations of \(y_{l}\), \(l=0,1,\ldots ,K\) and a standard gradient-based optimization algorithm. In so doing, recycle the PDD expansion coefficients obtained from Step 2 in (37) and (38), producing approximations of the objective and constraint functions that stem from single calculation of these coefficients. To evaluate the gradients, recalculate the Fourier expansion coefficients of score functions as needed. Denote the approximate optimal solution by \(\mathbf {d}_{\ast }^{(q)}\). Set \(\mathbf {d}_{0}^{(q+1)}=\mathbf {d}_{\ast }^{(q)}\).

- Step 4:
If \(||\mathbf {d}_{\ast }^{(q)}-\mathbf {d}_{\ast }^{(q-1)}||_{2}<\epsilon \), then stop and denote the final approximate optimal solution as \(\tilde {\mathbf {d}^{\ast }}=\mathbf {d}_{\ast }^{(q)}\). Otherwise, update \(q=q+1\) and go to Step 2.

### 5.4 Multi-point single-step PDD

The optimization methods described in the preceding subsections are founded on PDD approximations of stochastic responses, supplying surrogates of objective and constraint functions for the entire design space. Therefore, these methods are global and may not be cost-effective when the truncation parameters of PDD are required to be exceedingly large to capture high-order responses or high-variate interactions of input variables. Furthermore, a global method using a truncated PDD, obtained by retaining only low-order or low-variate terms, may not even find a true optimal solution. An attractive alternative method, developed in this work and referred to as the multi-point single-step PDD method, involves local implementations of the single-step PDD approximation that are built on a local subregion of the design space. According to this method, the original RDO problem is exchanged with a succession of simpler RDO sub-problems, where the objective and constraint functions in each sub-problem represent their multi-point approximations (Toropov et al. 1993). The design solution of an individual sub-problem, obtained by the single-step PDD method, becomes the initial design for the next sub-problem. Then, the move limits are updated, and the optimization is repeated iteratively until the optimal solution is attained. Due to its local approach, the multi-point single-step PDD method should solve practical engineering problems using low-order and/or low-variate PDD approximations.

*q*th subregion for \(q=1,2,\ldots \). According to the multi-point single-step PDD method, the RDO problem in (2) is reformulated to

*S*-variate,

*m*th-order PDD approximations of \(y_{l}(\mathbf {X})\) and \(c_{l}(\mathbf {d})\), respectively, at iteration

*q*, and \(d_{k,0}^{(q)}-\beta _{k}^{(q)}(d_{k,U}-d_{k,L})/2\) and \(d_{k,0}^{(q)}+\beta _{k}^{(q)}(d_{k,U}-d_{k,L})/2\), also known as the move limits, are the lower and upper bounds, respectively, of the subregion \(\mathcal {D}^{(q)}\). The multi-point single-step PDD method solves the optimization problem in (39) for \(q=1,2,\ldots \) by successively employing the single-step PDD approximation at each subregion or iteration until convergence is attained. When \(S\to N\) and \(m\to \infty \), the second-moment properties of PDD approximations converge to their exact values, yielding coincident solutions of the optimization problems described by (2) and (39). However, if the subregions are sufficiently small, then for finite and possibly low values of

*S*and

*m*, (39) is expected to generate an accurate solution of (2), the principal motivation of this method.

- Step 1:
Select an initial design vector \(\mathbf {d}_{0}\). Define tolerances \(\epsilon _{1}>0\), \(\epsilon _{2}>0\), and \(\epsilon _{3}>0\). Set the iteration \(q=1\), \(\mathbf {d}_{0}^{(q)}=(d_{1,0}^{(q)},\ldots ,d_{M,0}^{(q)})=\mathbf {d}_{0}\). Define the subregion size parameters \(0<\beta _{k}^{(q)}\le 1\), \(k=1,\ldots ,M\), describing \(\mathcal {D}^{(q)}=\times _{k=1}^{k=M}[d_{k,0}^{(q)}-\beta _{k}^{(q)}(d_{k,U}-d_{k,L})/2,d_{k,0}^{(q)}+\beta _{k}^{(q)}(d_{k,U}-d_{k,L})/2]\). Denote the subregion’s increasing history by a set \(H^{(0)}\) and set it to empty. Set two designs \(\mathbf {d}_{f}=\mathbf {d}_{0}\) and \(\mathbf {d}_{f,last}\ne \mathbf {d}_{0}\) such that \(||\mathbf {d}_{f}-\mathbf {d}_{f,last}||_{2}>\epsilon _{1}\). Set \(\mathbf {d}_{\ast }^{(0)}=\mathbf {d}_{0}\) , \(q_{f,last}=1\) and \(q_{f}=1\). Usually, a feasible design should be selected to be the initial design \(\mathbf {d}_{0}\). However, when an infeasible initial design is chosen, a new feasible design can be obtained during the iteration if the initial subregion size parameters are large enough.

- Step 2:
Select (\(q=1\)) or use (\(q>1\)) the PDD truncation parameters

*S*and*m*. At \(\mathbf {d}=\mathbf {d}_{0}^{(q)}\), generate the PDD expansion coefficients, \(y_{\emptyset }(\mathbf {d})\) and \(C_{u\mathbf {j}_{|u|}}(\mathbf {d})\), where \(\emptyset \ne u\subseteq \{1,\ldots ,N\}\), \(1\le |u|\le S\), \(\mathbf {j}_{|u|}\in \mathbb {N}_{0}^{|u|}\), \(||\mathbf {j}_{|u|}||_{\infty }\le m\), \(j_{1},\ldots ,j_{|u|}\neq 0\), using dimension-reduction integration with \(R=S\), \(n=m+1\), leading to*S*-variate,*m*th-order PDD approximations \(\tilde {y}_{l,S,m}^{(q)}(\mathbf {X})\) of \(y_{l}(\mathbf {X})\) and \(\tilde {c}_{l,S,m}^{(q)}(\mathbf {d})\) of \(c_{l}(\mathbf {d})\), \(l=0,1,\ldots ,K\), in (2). Calculate the expansion coefficients of score functions, \(s_{k,\emptyset }(\mathbf {d})\) and \(D_{i_{k},j}(\mathbf {d})\), where \(k=1,\ldots ,M\) and \(j=1,\ldots ,m^{\prime}\), analytically, if possible, or numerically, resulting in \(m^{\prime}\)th-order Fourier-polynomial approximations of \(s_{k}(X_{i_{k}};\mathbf {d})\), \(k=1,\ldots ,M\). - Step 3:
If \(q=1\) and \(\tilde {c}_{l}^{(q)}(\mathbf {d}_{0}^{(q)})<0\) for \(l=1,\ldots ,K\), then go to Step 4. If \(q>1\) and \(\tilde {c}_{l}^{(q)}(\mathbf {d}_{0}^{(q)})<0\) for \(l=1,\ldots ,K\), then set \(\mathbf {d}_{f,\mathrm {last}}=\mathbf {\mathbf {d}}_{f}\), \(\mathbf {\mathbf {d}}_{f}=\mathbf {d}_{0}^{(q)}\) , \(q_{f,\mathrm {last}}=q_{f}\), \(q_{f}=q\) and go to Step 4. Otherwise, go to Step 5.

- Step 4:
If \(||\mathbf {\mathbf {d}}_{f}-\mathbf {d}_{f,last}||_{2}<\epsilon _{1}\) or \(\left |\left [\tilde {c}_{0}^{(q)}(\mathbf {d}_{f})-\tilde {c}_{0}^{(q_{f,\mathrm {last}})}(\mathbf {d}_{f,\mathrm {last}})\right ]/\tilde {c}_{0}^{(q)}(\mathbf {d}_{f})\right |<\epsilon _{3}\), then stop and denote the final optimal solution as \(\tilde {\mathbf {d}}^{\ast }=\mathbf {d}_{f}\). Otherwise, go to Step 6.

- Step 5:
Compare the infeasible design \(\mathbf {d}_{0}^{(q)}\) with the feasible design \(\mathbf {d}_{f}\) and interpolate between \(\mathbf {d}_{0}^{(q)}\) and \(\mathbf {d}_{f}\) to obtain a new feasible design and set it as \(\mathbf {d}_{0}^{(q+1)}\). For dimensions with large differences between \(\mathbf {d}_{0}^{(q)}\) and\(\mathbf {d}_{f}\), interpolate aggressively. Reduce the size of the subregion \(\mathcal {D}^{(q)}\) to obtain new subregion \(\mathcal {D}^{(q+1)}\). For dimensions with large differences between \(\mathbf {d}_{0}^{(q)}\) and \(\mathbf {d}_{f}\), reduce aggressively. Also, for dimensions with large differences between the sensitivities of \(\tilde {c}_{l,Sm}^{(q)}(\mathbf {d}_{0}^{(q)})\) and \(\tilde {c}_{l,Sm}^{(q-1)}(\mathbf {d}_{0}^{(q)})\), reduce aggressively. Update \(q=q+1\) and go to Step 2.

- Step 6:
If the subregion size is small, that is, \(\beta _{k}^{(q)}(d_{k,U}-d_{k,L})<\epsilon _{2}\), and \(\mathbf {d}_{\ast }^{(q-1)}\) is located on the boundary of the subregion, then go to Step 7. Otherwise, go to Step 9.

- Step 7:
If the subregion centered at \(\mathbf {d}_{0}^{(q)}\) has been enlarged before, that is, \(\mathbf {d}_{0}^{(q)}\in H^{(q-1)}\), then set \(H^{(q)}=H^{(q-1)}\) and go to Step 9. Otherwise, set \(H^{(q)}=H^{(q-1)}\bigcup \{\mathbf {d}_{0}^{(q)}\}\) and go to Step 8.

- Step 8:
For coordinates of \(\mathbf {d}_{0}^{(q)}\) located on the boundary of the subregion and \(\beta _{k}^{(q)}(d_{k,U}-d_{k,L})<\epsilon _{2}\), increase the sizes of corresponding components of \(\mathcal {D}^{(q)}\); for other coordinates, keep them as they are. Set the new subregion as \(\mathcal {D}^{(q+1)}\).

- Step 9:
Solve the design problem in (39) employing the single-step PDD method. In so doing, recycle the PDD expansion coefficients obtained from Step 2 in (37) and (38), producing approximations of the objective and constraint functions that stem from single calculation of these coefficients. To evaluate the gradients, recalculate the Fourier expansion coefficients of score functions as needed. Denote the optimal solution by \(\mathbf {d}_{\ast }^{(q)}\) and set \(\mathbf {d}_{0}^{(q+1)}=\mathbf {d}_{\ast }^{(q)}\). Update \(q=q+1\) and go to Step 2.

Summary of features of the four proposed methods

Feature | Direct PDD | Single-step PDD | Sequential PDD | Multi-point single-step PDD |
---|---|---|---|---|

Design space | Global | Global | Global | Local |

Frequency of PDD approximations | Every iteration | Only first iteration | A few iterations | For every subproblem |

Problem solved in every iteration | Original problem | Original problem | Original problem | Subproblems |

## 6 Numerical examples

Four examples are presented to illustrate the PDD methods developed in solving various RDO problems. The objective and constraint functions are either elementary mathematical functions or relate to engineering problems, ranging from simple structural to complex FEA-aided mechanical designs. Both size and shape design problems are included. In Examples 1–4, orthonormal polynomials, consistent with the probability distributions of input random variables, were used as bases. For the Gaussian distribution, the Hermite polynomials were used. For random variables following non-Gaussian probability distributions, such as the Lognormal, Beta, and Gumbel distributions in Example 2, the orthonormal polynomials were obtained either analytically when possible or numerically, exploiting the Stieltjes procedure (Rahman 2009a; Gautschi 2004). However, in Examples 3 and 4, the original random variables were transformed into standard Gaussian random variables, facilitating the use of classical Hermite polynomials as orthonormal polynomials. The PDD truncation parameters *S* and *m* vary, depending on the function or the example, but in all cases the PDD expansion coefficients were calculated using dimension-reduction integration with \(R=S\) and the number of integration points \(n=m+1\). The Gauss-quadrature rules are consistent with the polynomial basis functions employed. Since the design variables are the means of Gaussian random variables, the order \(m^{\prime}\) used for Fourier expansion coefficients of score functions in Examples 1, 3, and 4 is one. However, in Example 2, where the design variables describe both means and standard deviations of random variables, \(m^{\prime}\) is two. The tolerances and initial subregion size parameters are as follows: (1) \(\epsilon =0.001\); \(\epsilon _{1}=0.1\), \(\epsilon _{2}=2\); \(\epsilon _{3}=0\) (Example 3), \(\epsilon _{3}=0.005\) (Example 4); and (2) \(\beta _{1}^{(1)}=\ldots =\beta _{M}^{(1)}=0.5\). The optimization algorithm selected is sequential quadratic programming (DOT 2001) in all examples.

### 6.1 Example 1: optimization of a mathematical function

Two proposed RDO methods, the direct PDD and single-step PDD methods, were applied to solve this problem. Since \(y_{0}\) and \(y_{1}\) are both univariate functions, only univariate (\(S=1\)) PDD approximations are required. The chosen PDD expansion orders are \(m=4\) for \(y_{0}\) and \(m=1\) for \(y_{1}\). The initial design vector \(\mathbf {d}_{0}=(5,5)\) and, correspondingly, \(\sqrt {\mathrm {var}_{\mathbf {d}_{0}}\left [y_{0}(\mathbf {X})\right ]}=18.2987\). The approximate optimal solution is denoted by \(\tilde {\mathbf {d}}^{\ast }=(\tilde {d}_{1}^{\ast },\tilde {d}_{2}^{\ast })\).

Optimization results for the mathematical example

Results | Method | |||
---|---|---|---|---|

Direct PDD | Single-Step PDD | TPQ | Taylor series | |

\(\tilde {d}_{1}^{\ast }\) | 3.3508 | 3.3508 | 3.4449 | 3.4983 |

\(\tilde {d}_{1}^{\ast }\) | 4.9856 | 4.9856 | 5.000 | 4.9992 |

\(c_{0}(\tilde {\mathbf {d}}^{\ast })\) | 0.0756 | 0.0756 | 0.0861 | 0.0902 |

\(c_{1}(\tilde {\mathbf {d}}^{\ast })\) | −0.1873 | −0.1599 | −0.2978 | −0.3504 |

\(\sqrt {\mathrm {var}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]}\) | 1.1340 | 1.1340 | 1.2915 | 1.3535 |

No. of iterations | 5 | 5 | 4 | 4 |

No. of \(y_{0}\) evaluations | 66 | 11 | 81 | 45 |

No. of \(y_{1}\) evaluations | 30 | 5 | 81 | 45 |

### 6.2 Example 2: size design of a two-bar truss

Statistical properties of random input for the two-bar truss problem

Random variable | Mean | Standard deviation | Probability distribution |
---|---|---|---|

Cross-sectional area (\(X_{1}\)), \(\mathrm {cm}{}^{2}\) | \(d_{1}\) | \(0.02d_{1}\) | Gaussian |

Half-horizontal span (\(X_{2}\)), m | \(d_{2}\) | \(0.02d_{2}\) | Gaussian |

Mass density (\(X_{3}\)), \(\mathrm {kg}/\mathrm {m}^{3}\) | 10,000 | 2,000 | Beta |

Load magnitude (\(X_{4}\)), kN | 800 | 200 | Gumbel |

Yield strength (\(X_{5}\)), MPa | 1,050 | 250 | Lognormal |

Optimization results for the two-bar truss problem (\(m=2\), \(n=3\))

Results | Direct PDD (Univariate) | Direct PDD (Bivariate) | Direct PDD (Trivariate) | Sequential PDD (Univariate) | Sequential PDD (Bivariate) | Sequential PDD (Trivariate) | TPQ | Taylor series |
---|---|---|---|---|---|---|---|---|

\(\tilde {d}_{1}^{\ast }\), \(\mathrm {cm}^{2}\) | 11.4749 | 11.5561 | 11.5561 | 11.4811 | 11.5710 | 11.5714 | 11.5669 | 10.9573 |

\(\tilde {d}_{2}^{\ast }\), m | 0.3781 | 0.3791 | 0.3791 | 0.3777 | 0.3753 | 0.3752 | 0.3767 | 0.3770 |

\(c_{0}(\tilde {\mathbf {d}}^{\ast })\) | 1.2300 | 1.2392 | 1.2391 | 1.2306 | 1.2392 | 1.2392 | 1.2393 | 1.1741 |

\(c_{1}(\tilde {\mathbf {d}}^{\ast })\) | 0.0172 | 0.0096 | 0.0096 | 0.0167 | 0.0097 | 0.0096 | 0.0095 | 0.0657 |

\(c_{2}(\tilde {\mathbf {d}}^{\ast })\) | −0.4882 | −0.4911 | −0.4910 | −0.4889 | −0.4948 | −0.4950 | −0.4935 | −0.4650 |

\(\mathbb {E}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]\) | 12.2684 | 12.3591 | 12.3591 | 12.2732 | 12.3589 | 12.3598 | 12.3608 | 11.7105 |

\(\sqrt {\mathrm {var}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]}\) | 2.4666 | 2.4851 | 2.4851 | 2.4677 | 2.4849 | 2.4850 | 2.4852 | 2.3542 |

No. of iterations | 19 | 14 | 14 | 8 | 7 | 7 | 10 | 8 |

No. of \(y_{0}\) evaluations | 190 | 518 | 896 | 80 | 259 | 448 | 594 | 108 |

Total no. of \(y_{1}\) & \(y_{2}\) evaluations | 494 | 1,876 | 4,900 | 208 | 938 | 2,450 | 3,564 | 270 |

Since this problem was also solved by the TPQ and Taylor series methods, comparing their reported solutions (Lee et al. 2009), listed in the last two columns of Table 4, with the PDD solutions should be intriguing. It appears that the TPQ method is also capable of producing a similar optimal solution, but by incurring a computational cost more than most of the PDD methods examined in this work. Comparing the numbers of function evaluations, the TPQ method is more expensive than the univariate direct PDD method by factors of three to seven. These factors grow into 7–17 when graded against the univariate sequential PDD method. The Taylor series method needs only 378 function evaluations, which is slightly more than 288 function evaluations by the univariate sequential PDD, but it violates the first constraint by at least six times more than all PDD and TPQ methods.

*m*or

*n*. In which case, the univariate PDD methods are even more efficient than the TPQ method by orders of magnitude.

Optimization results for the two-bar truss problem (\(m=3\), \(n=4\))

Results | Method | |||||||
---|---|---|---|---|---|---|---|---|

Direct PDD (Univariate) | Direct PDD (Bivariate) | Direct PDD (Trivariate) | Sequential PDD (Univariate) | Sequential PDD (Bivariate) | Sequential PDD (Trivariate) | TPQ | Taylor series | |

\(\tilde {d}_{1}^{\ast }\), \(\mathrm {cm}^{2}\) | 11.5516 | 11.6439 | 11.6439 | 11.5650 | 11.6505 | 11.6498 | 11.6476 | 10.9573 |

\(\tilde {d}_{2}^{\ast }\), m | 0.3805 | 0.3779 | 0.3779 | 0.3754 | 0.3763 | 0.3763 | 0.3767 | 0.3770 |

\(c_{0}(\tilde {\mathbf {d}}^{\ast })\) | 1.2393 | 1.2481 | 1.2481 | 1.2386 | 1.2481 | 1.2481 | 1.2480 | 1.1741 |

\(c_{1}(\tilde {\mathbf {d}}^{\ast })\) | 0.0095 | 0.0024 | 0.0025 | 0.0101 | 0.0024 | 0.0024 | 0.0025 | 0.0657 |

\(c_{2}(\tilde {\mathbf {d}}^{\ast })\) | −0.4897 | −0.4959 | −0.4958 | −0.4945 | −0.4974 | −0.4974 | −0.4970 | −0.4650 |

\(\mathbb {E}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]\) | 12.3597 | 12.4480 | 12.4480 | 12.3538 | 12.4482 | 12.4477 | 12.4464 | 11.7150 |

\(\sqrt {\mathrm {var}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]}\) | 2.4678 | 2.5025 | 2.5025 | 2.4836 | 2.5029 | 2.5028 | 2.5023 | 2.3542 |

No. of iterations | 15 | 16 | 15 | 7 | 5 | 5 | 10 | 8 |

No. of \(y_{0}\) evaluations | 195 | 976 | 1,875 | 91 | 305 | 625 | 2,503 | 108 |

Total no. of \(y_{1}\) & \(y_{2}\) evaluations | 510 | 3,616 | 11,070 | 238 | 1,130 | 3,690 | 15,018 | 270 |

### 6.3 Example 3: shape design of a three-hole bracket

*S*or

*m*increases. The univariate, first-order (\(S=1, m=1\)) PDD method, which is the most economical method, produces an optimal solution reasonably close to those obtained from higher-order or bivariate PDD methods. For instance, the largest deviation from the average values of the objective function at four optimum points is only 2.5 %. It is important to note that the coupling between single-step PDD and multi-point approximation is essential to find optimal solutions of this practical problem using low-variate, low-order PDD approximations.

Optimization results for the three-hole bracket

Results | Multi-point single-step PDD method | |||
---|---|---|---|---|

Univariate (\(S=1,m=1\)) | Univariate (\(S=1, m=2\)) | Univariate (\(S=1, m=3\)) | Bivariate (\(S=2, m=1\)) | |

\(\tilde {d}_{1}^{\ast }\), mm | 12.8168 | 13.6828 | 13.9996 | 13.9936 |

\(\tilde {d}_{2}^{\ast }\), mm | 17.0112 | 17.0071 | 17.5236 | 17.0133 |

\(\tilde {d}_{3}^{\ast }\), mm | 26.6950 | 28.3935 | 28.8053 | 28.6254 |

\(\tilde {d}_{4}^{\ast }\), mm | 30.1908 | 30.2860 | 30.0009 | 30.0083 |

\(\tilde {d}_{5}^{\ast }\), mm | 12.0069 | 12.0003 | 12.0000 | 12.0000 |

\(\tilde {d}_{6}^{\ast }\), mm | 12.0003 | 12.0000 | 12.0000 | 12.0000 |

\(\tilde {d}_{7}^{\ast }\), mm | 118.1200 | 118.0900 | 117.4930 | 117.7929 |

\(\tilde {d}_{8}^{\ast }\), mm | −13.7400 | −13.8900 | −13.8680 | −13.9053 |

\(\tilde {d}_{9}^{\ast }\), mm | 14.9124 | 14.9573 | 14.9991 | 14.9966 |

\(\tilde {c}_{0}(\tilde {\mathbf {d}}^{\ast })\) | 0.6686 | 0.6430 | 0.6364 | 0.6602 |

\(\tilde {c}_{1}(\tilde {\mathbf {d}}^{\ast })\) | −1.6671 | −0.8289 | −1.8599 | −8.8978 |

\(\mathbb {E}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]\) | 0.1230 | 0.1185 | 0.1181 | 0.1176 |

\(\sqrt {\mathrm {var}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]}\) | 0.00137 | 0.00132 | 0.00130 | 0.00137 |

No. of iterations | 42 | 43 | 36 | 39 |

No. of FEA | 798 | 1,204 | 1,332 | 6,357 |

*S*or

*m*, the overall area of an optimal design has been substantially reduced, mainly due to significant alteration of the inner boundary and moderate alteration of the outer boundary of the bracket. All nine design variables have undergone moderate to significant changes from their initial values. The optimal masses of the bracket vary as 0.1230 kg, 0.1185 kg, 0.1181 kg, and 0.1186 kg—about a 65 % reduction from the initial mass of 0.3415 kg. Compared with the conservative design in Fig. 4b, larger stresses—for example, 800 MPa—are safely tolerated by the final designs in Fig. 5a–d.

### 6.4 Example 4: shape design of a lever arm

As in Example 3, the random variables \(X_{i}, i=1,\ldots ,22\), are truncated Gaussian and have probability densities described by (50) with \(a_{i}=d_{i}-D_{i}\) and \(b_{i}=d_{i}+D_{i}\) denoting the lower and upper bounds, respectively. To avoid unrealistic designs, \(D_{i}=2\) when \(i=1,2,4,14,16\), and \(D_{i}=5\) otherwise.

Reductions in the mean and standard deviation of \(y_{0}\) from initial to optimal designs

Example | \(\frac {\mathbb {E}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]-\mathbb {E}_{\mathbf {d}_{0}}\left [y_{0}(\mathbf {X})\right ]}{\mathbb {E}_{\mathbf {d}_{0}}\left [y_{0}(\mathbf {X})\right ]} \) | \( \frac {\sqrt {\mathrm {var}_{\boldsymbol {\tilde {\mathrm {d}}}^{\ast }}\left [y_{0}(\mathbf {X})\right ]}-\sqrt {\mathrm {var}_{\mathbf {d}_{0}}\left [y_{0}(\mathbf {X})\right ]}}{\sqrt {\mathrm {var}_{\mathbf {d}_{0}}\left [y_{0}(\mathbf {X})\right ]}} \) |
---|---|---|

1 | Not applicable | −93.80 % |

2 | −12.51 % | −12.66 % |

3 | −65.07 % | −4.35 % |

4 | −78.81 % | −14.34 % |

## 7 Discussions

- (1)
The direct and single-step PDD methods generate identical optimal solutions for the polynomial functions, but the latter method is substantially more efficient than the former method;

- (2)
The direct and sequential PDD methods, both employing univariate, bivariate, and trivariate PDD approximations, produce very close optimal solutions for the non-polynomial functions, but at vastly differing expenses. For either method, the univariate solution is accurate and most economical, even though the stochastic responses are multivariate functions. Given a PDD approximation, the sequential PDD method furnishes an optimal solution incurring at most half the computational cost of the direct PDD method;

- (3)
For both polynomial and non-polynomial functions, the TPQ method, although accurate, is more expensive than most variants of the direct, single-step, and sequential PDD methods examined. Considering the non-polynomial functions, the univariate direct PDD and univariate sequential PDD methods are more economical than the TPQ method by an order of magnitude or more;

- (4)
The multi-point single-step PDD method employing low-variate or low-order PDD approximations, including a univariate, first-order PDD approximation, is able to solve practical engineering problems with a reasonable computational effort.

Efficiency and applicability of the four proposed methods

Method | Efficiency | Applicability | Comments |
---|---|---|---|

Direct PDD | Low | Both polynomial and non-polynomial functions with small design spaces | Expensive due to recalculation of expansion coefficients. Impractical for complex system designs. |

Single-step PDD | Highest | Low-order polynomial functions with small design spaces | Highly economical due to recycling of expansion coefficients, but may produce premature solutions coefficients, but may produce premature solutions for complex system designs. |

Sequential PDD | Medium | Polynomial or non-polynomial functions with small to medium design spaces | More expensive than single-step PDD, but substantially more economical than direct PDD. May require high-variate and high-order PDD approximations for complex system designs. |

Multi-point single-step PDD | High | Polynomial or non-polynomial functions with large design spaces | Capable of solving complex, practical design problems using low-variate and/or low-order PDD approximations. |

## 8 Conclusions

Four new methods are proposed for robust design optimization of complex engineering systems. The methods involve PDD of a high-dimensional stochastic response for statistical moment analysis, a novel integration of PDD and score functions for calculating the second-moment sensitivities with respect to the design variables, and standard gradient-based optimization algorithms, encompassing direct, single-step, sequential, and multi-point single-step design processes. Because they are rooted in ANOVA dimensional decomposition, the PDD approximations for arbitrary truncations predict the exact mean and generate a convergent sequence of variance approximations for any square-integrable function. When blended with score functions, PDD leads to explicit formulae, expressed in terms of the expansion coefficients, for approximating the second-moment design sensitivities that are also theoretically convergent. More importantly, the statistical moments and design sensitivities are both determined concurrently from a single stochastic analysis or simulation.

Among the four design methods developed, the direct PDD method is the simplest of all, but requires re-calculations of the expansion coefficients at each design iteration and is, therefore, expensive, depending on the cost of evaluating the objective and constraint functions and the requisite number of design iterations. The single-step PDD method eliminates the need to re-calculate the expansion coefficients from scratch by recycling the old expansion coefficients, consequently holding a potential to significantly curtail the computational effort. However, it depends heavily on the quality of a PDD approximation and the accuracy of the estimated expansion coefficients during design iterations. The sequential PDD method upholds the merits of both the direct and single-step PDD methods by re-calculating the expansion coefficients a few times more than the single-step PDD, incurring a computational complexity that is lower than the direct PDD method. However, all three methods just described are global and may not work if the design space is too large for a PDD approximation, with a chosen degree of interaction or expansion order, to be sufficiently accurate. The multi-point single-step PDD method mitigates this problem by adopting a local implementation of PDD approximations, where an RDO problem with a large design space is solved in succession. Precisely for this reason, the method is capable of solving practical engineering problems using low-order and/or low-variate PDD approximations of stochastic responses.