Numerical Methods for Metamaterial Design pp 7196  Cite as
Metamaterial Design by Mesh Adaptive Direct Search
Abstract
In the design of optical metamaterials, some optimization problems require launching a numerical simulation. The Mesh Adaptive Direct Search algorithm is designed for such problems. The MADS algorithm does not require any derivative information about the problem being optimized, and no continuity or differentiability assumptions are made by MADS on the functions defining the simulation. A detailed discussion of the method is provided in the second section of the chapter, followed by a discussion of the NOMAD implementation of the method and its features. The final section of the chapter lists three instances of combining NOMAD with FiniteDifference TimeDomain electromagnetic simulations to tailor the broadband spectral response and nearfield interactions of Split Ring Resonator metamaterials.
Keywords
Variable Neighborhood Search Trial Point Incumbent Solution Poll Point Library Mode4.1 Introduction
This chapter describes methods for solving constrained optimization problems using the Mesh Adaptive Direct Search (Mads) algorithm, which belongs to the more broad class of DerivativeFree Optimization methods. Because small changes in the geometry of a metamaterial can results in large changes in the overall behavior of the structure, these techniques are wellsuited for the design of optical metamaterials, and can handle the large discontinuities in the “cost function space” that often arise. The Mads algorithm does not require any derivative information about the problem being optimized, and no continuity or differentiability assumptions are made on the functions defining the simulation. Out of the many applicable techniques that can be used for metamaterial design, the NOMAD implementation of the Mads algorithm discussed in this chapter has many advantages, including built in capabilities to handle nonlinear constraints, biobjective function optimization, sensitivity analysis, and up to 50 variables. For even larger problems, the PsdMads parallel version of the method is able to solve problems with 50–500 variables. Lastly, the NOMAD implementation has the added benefit that it is constantly being updated and improved within a number of programming languages including C++ and Matlab (http://www.gerad.ca/nomad).
4.1.1 Structuring the Optimization Problem
The sets X and Ω define the feasible region and each one of them corresponds to a specific type of constraint for which different treatments are described in Sect. 4.2.2. The quantifiable constraints c _{ j }(x)≤0, j∈J, defining Ω provide a distance to feasibility and/or to infeasibility. Violating these constraints is also permitted as long as these violations occur only at the intermediate candidates considered by an optimization method. The set X may contain any constraint for which a measure of the violation is not available, and/or constraints that cannot be relaxed. Typically, X contains bound constraints necessary to run the simulation, but can also include hidden constraints [20], which occur when the simulation fails to evaluate. In [5], the objective function failed to return a value on approximately 43 % of the simulations, and in [18], the failure rate climbed to 60 %. Such problems pose a challenge to optimization methods that use function evaluations to estimate derivatives.
Different approaches exist to tackle Problem (4.1), and this chapter discusses DerivativeFree Optimization (DFO) methods. This choice is justified by the fact that these methods are backed by rigorous convergence analyses based on different levels of assumptions on the nature of the functions defining the problem. This type of analysis marks the difference between DFO methods and heuristics. While this chapter focuses on the Mads algorithm to address these types of problems, a review of DFO methods may be consulted in the recent book [23], which does not focus on the constrained case. The present chapter aims at describing a practical and recent method and software for constrained blackbox optimization.
The chapter is divided as follows. Section 4.2 summarizes the general organization of the Mads algorithm, describes strategies to handle various types of constraints, discusses the use of surrogates and models to guide the optimization, and details the type of nonsmooth convergence analyses on which these methods rely. Section 4.3 describes our C++ implementation NOMAD of Mads and highlights some features that make the code versatile. Finally, Sect. 4.4 describes a metamaterial design optimization problem, shows how to formulate it as a blackbox optimization problem, and numerical experiments are conducted using the NOMAD software.
4.2 The Mesh Adaptive Direct Search Class of Algorithms
The Mesh Adaptive Direct Search (Mads) is presented in [9] as a generalization of several existing direct search methods.
4.2.1 General Organization of the Mads Algorithm
In order to use a Mads algorithm, the user must provide an initial point denoted \(x_{0} \in {\mathbb{R}}^{n}\). It does not need to be feasible with respect to the quantifiable constraints c _{ j }(x)≤0, j∈J, but must belong to the set X. Mads algorithms are iterative and the iteration counter is denoted by the index k. At each iteration, the algorithm deploys some effort to improve the incumbent solution x _{ k }, i.e., the current best solution. For now, we will remain vague about the word best as it takes different meanings if it refers to a feasible or an infeasible solution. This clarification is made in Sect. 4.2.2
At each iteration, the algorithm tries to improve the incumbent solution by launching the simulation at a finite number of carefully selected trial points. This is done in two steps called the search and the poll. The search is very flexible and allows the user to take advantage of his knowledge of the problem to propose some candidates where the simulation will be launched. Some search strategies are tailored to specific problems, while others are generic (e.g., speculative search [9], Latin hypercube sampling [36], variable neighborhood searches [6], surrogates). The poll needs to satisfy more rigorous restrictions. It consists of a local exploration near the current incumbent solution, and its definition varies from one instantiation of Mads to another. In practice, the search can greatly improve the quality of the final solution, while the poll structure allows a rigorous convergence analysis.
Each Mads iteration goes as follows. Given an incumbent solution x _{ k }∈X, the search step produces a list of tentative trial mesh points. Any mechanism can be used to created the list, as long as it contains a finite number of points located on the mesh. The list may even be empty. Then, the simulation is launched at the trial points until all trial points are tested, or until one trial point is found to be better than the incumbent x _{ k }. In the latter case, the poll step can be skipped, and the algorithm may continue directly to the updates.
Following an unsuccessful search step, the poll step generates a list of mesh points near the incumbent x _{ k }. The term near is tied to the socalled poll size parameter \(\varDelta _{k}^{p} >0\). Again, the poll step may be interrupted as soon as an improvement over the incumbent is found. During an iteration, the simulations can be launched sequentially or in parallel. Synchronous and asynchronous strategies are described in Sect. 4.3.3 when multiple processors are available.
Parameters are updated at the end of each iteration. There are two possibilities. If either the search or the poll step generated a mesh point t∈M _{ k } which is better than x _{ k }, then the next incumbent x _{ k+1} is set to t and both the mesh size and poll size parameters are increased or kept to the same value. For example, \(\varDelta ^{p}_{k+1} \leftarrow 2\varDelta ^{p}_{k}\) and \(\varDelta ^{m}_{k+1} \leftarrow\min\{1, \sqrt{\varDelta ^{p}_{k}}\}\). Otherwise, x _{ k+1} is set to x _{ k } and the poll size parameter is decreased and the mesh size parameter decreased or kept the same. For example, \(\varDelta ^{p}_{k+1} \leftarrow \frac{1}{2}\varDelta ^{p}_{k}\) and \(\varDelta ^{m}_{k+1} \leftarrow\min\{1, \sqrt{\varDelta ^{p}_{k}}\}\).
At any iteration of a Mads algorithm, the poll size parameter \(\varDelta ^{p}_{k}\) must be greater than or equal to the mesh size parameter \(\varDelta ^{m}_{k}\). In Fig. 4.1, \(\varDelta ^{p}_{k} = \frac{1}{2}\) and \(\varDelta ^{m}_{k} = \frac{1}{4}\), and the search points are {t _{1},t _{2},t _{3}}. In OrthoMads, the poll points are obtained by generating a pseudorandom orthogonal basis H _{ k } and by completing it to a maximal positive basis D _{ k }=[H _{ k } −H _{ k }]. The poll points are then obtained from the incumbent x _{ k } in the directions of the columns of D _{ k } while remaining in the frame (the shaded region in the figure) defined by the poll size parameter \(\varDelta ^{p}_{k}\).
4.2.2 Handling of Constraints
The extreme barrier is essentially a mechanism that allows infeasible trial points whose constraint violation function value is below a threshold \(h_{k}^{\max} > 0\). But as the algorithm is deployed and the iteration number increases, the threshold is progressively reduced. This is accomplished with the Mads algorithm by having two incumbent solutions around which polling is conducted. One poll center is the feasible incumbent solution \(x_{k}^{F}\), i.e., the feasible solution found so far with the least objective function value. The second poll center is the infeasible incumbent solution \(x_{k}^{I}\), i.e., the infeasible solution found so far with a constraint violation value under the threshold \(h^{\max}_{k}\) having the least objective function value. Under this strategy, the infeasible trial points approach the feasible region by prioritizing the infeasible points with a low objective function value. This strategy differs from the ones in [2, 8, 24] where priority was given to feasibility at the expense of the objective function value.
With the progressive barrier, the iterations are categorized into more than two types. Dominating iterations are those that either generate a feasible trial point with a lower objective function value than that the feasible incumbent, or those that generate an infeasible trial point with better objective and constraint violation function values than the infeasible incumbent. Improving iterations are those that are not dominating, but generate an infeasible trial point with a better constraint violation value. Otherwise, the iteration is said to be unsuccessful.
At the end of an unsuccessful iteration the incumbents remain unchanged, and the poll size parameter is reduced as this suggests that we are near a locally optimal solution. At the end of a dominating solution, a new incumbent solution is identified, and the poll size parameter is increased to allow farreaching explorations in the space of variables. Finally, after an improving iteration, the poll size parameter is kept unchanged, but the constraint violation threshold is reduce in such a way that the next iteration will not have the same infeasible incumbent as the one from the previous iteration.
Another strategy to handle relaxable quantifiable constraints is the progressive to extreme barrier [12]. As its name suggests, this strategy consists in first handling the constraint by the progressive barrier. But as soon as a trial point which satisfies the constraint is generated, then the treatment of the constraint is switched to the extreme barrier. This last strategy allows infeasible initial trial points, but forces the satisfaction of individual constraints as soon as they are satisfied for the first time.
4.2.3 Surrogates and Models
Surrogates are functions that can be considered as substitutes for the true functions defining the optimization problem, f and c _{ j }, j∈J. A surrogate function shares some similarities with the original one, but has the advantage of being significantly less expensive to evaluate. Surrogate functions can be classified into static surrogates and dynamic surrogates.
4.2.3.1 Static Surrogates
Static surrogates are approximations that are provided by the user with some knowledge of the problem. A surrogate consists of a model of the true function, and is fixed during the optimization process. For example, static surrogates may be obtained by simplified physics models, or by allowing more relaxed stopping criteria within the blackbox simulation, or by replacing complicated subproblems by simpler ones. Straightforward uses of static surrogates within an optimization method are described in [15]. A first possibility is to order a list of tentative trial points by their surrogate values, and then to launch the expensive simulation defining the truth on the most promising trial points first. The process terminates as soon as a better point is found. This is called the opportunist strategy and can save a considerable amount of time.
A second strategy using static surrogates consists in defining a search step that optimizes the surrogates in order to determine one or two candidates for the true evaluations. In some situations, static surrogates may be parametrized with controllable precision. The use of such surrogates within a Gps framework is described in [39].
4.2.3.2 Dynamic Surrogates
In contrast to static surrogates, dynamic surrogates are not provided by the user. They correspond to models dynamically built within the optimization method, based on past evaluations from the cache. Any interpolation method can be used for this task, as, for example, quadratic models, neural networks, radial basis functions, or statistical methods. The Surrogate Management Framework [18] proposes ways to exploit such surrogates within direct search methods, and Gps in particular, for the unconstrained case. Successful applications of this framework include unsteady fluid mechanics problems [34, 35], helicopter rotor blade design [17] and multiobjective liquidrocket injector design [40].
Recent developments propose the use of these surrogates for the constrained case within Mads. The first of these developments considers quadratic models and is summarized in the next section. The second approach is ongoing research and currently considers statistical methods [27], namely tree Gaussian processes [26]. Quadratic model and statistical surrogates share some similarities. They can be used to sort a list of trial points before launching the expensive true blackbox simulation, as proposed above for the static surrogates. Dynamic surrogates may also define a search step named the model search, enabled as soon as a sufficient number of true evaluations is available (typically n+1). These points are denoted the data points and are used to build one model for f and m models for the quantifiable constraints c _{ j },j∈J. The model of the objective function is then optimized subject to the models of the constraints. This provides one or two mesh candidates, one feasible and possibly one infeasible, at which the true functions are evaluated. These are called oracle points. In addition to the nature of the surrogates, some subtleties remain: The quadratic models are kept local, while statistical surrogates consider the whole space and attempt to escape local solutions. They can also provide additional candidates based on statistics such as the Expected Improvement (EI) [29].
4.2.3.3 Quadratic Models in Mads
This section discusses the quadratic models described in [22] and currently used in NOMAD. The framework is inspired from the work of Conn et al. summarized in the DFO book [23] where more focus is put on modelbased methods.
Quadratic models are employed at two different levels: First, the model search exploits the flexibility of the search by allowing the generation of trial points anywhere on the mesh. Candidates of the model search are the result of an optimization process considering the model of the objective constrained to the models of the constraints.
The other way of using models is to sort a list of candidates prior to their evaluations (model ordering), so that the most promising—from the model point of view—points will be evaluated first. The impact of this strategy is important because of the opportunistic strategy.
To construct a model, a set Y={y ^{0},…,y ^{ p }} of p+1 data points is collected from the cache. The objective function f(y) and the constraint functions c _{ j }(y), j∈J are known and finite at each data point y∈Y. Since quadratic models are more suited for local interpolation, data points are collected in the neighborhood of the current iterate: \(y^{i} \in B_{\infty}(x_{k};\rho \varDelta _{k}^{p})\) with \(B_{\infty}(x;r) = \{ y \in {\mathbb{R}}^{n} : \ yx \_{\infty} \leq r \}\), where the poll size parameter \(\varDelta _{k}^{p}\) bounds the distance between x _{ k } and the poll trial points, and ρ is a parameter called the radius factor, typically set to two.
After Problem (4.4) is solved (in practice, heuristically), its feasible and infeasible incumbent solutions define new candidates at which evaluate the true functions f and c _{ j }, j∈J. In order to satisfy the Mads convergence analysis described in Sect. 4.2.4, these candidates are projected on the mesh before they are evaluated.
4.2.4 Convergence Analysis
Even if Mads is designed to be applied to the general optimization problem (4.1) without exploiting any of its structure, Mads is supported by a rigorous hierarchical convergence analysis. The analysis reveals that depending on the properties of the objective function f and the domain Ω, Mads will produce a limit point \(\hat{x}\) at which some necessary optimality conditions are satisfied. Of course, we do not expect our target problems to satisfy any smoothness properties, but the convergence analysis can be seen as a validation of the behavior of the algorithm on smoother problems.
The entire convergence analysis relies on the following assumptions. Suppose that Mads was launched on a test problem, without any stopping criteria, and suppose that the union of all trial points generated by the algorithm belongs to a bounded subset of \({\mathbb{R}}^{n}\). The assumption that Mads is launched indefinitely is not realistic, as in practice it necessarily terminates after a finite amount of time. But for the analysis, we are interested in seeing where the iterates would lead, it the algorithm were not stopped. The second assumption on the bounded subset can be satisfied in multiple ways. For example, it is true when the variables are bounded, or when level sets of f are bounded. In practice, it is not frequent that real problems have unbounded solutions.
The convergence analysis then focuses on limits of incumbent solutions. Torczon [43] showed for pattern searches that the hypotheses on bounded trial points implies that there are infinitely many unsuccessful iterations and that the limit inferior of the mesh size parameters \(\varDelta ^{m}_{k}\) converges to zero. These results were adapted in [9] in the context of the Mads algorithm. Let U denote the indices of the unsuccessful iterations and let \(\hat{x}\) be an accumulation point {x _{ k }}_{ k∈U }. Such an accumulation point exists because of the assumption that the iterates belong to a bounded set.
An unsuccessful iteration occurs when the poll step was conducted around the incumbent x _{ k }, and no better solution was found. The mesh size parameter is reduced only after an unsuccessful iteration. We say that the incumbent solution x _{ k } is a mesh local optimizer. At the low end of the convergence analysis [7], we have the zeroth order result: \(\hat{x}\) is the limit of mesh local optimizers on meshes that get infinitely fine. At the other end of the analysis, we have that if f is strictly differentiable near \(\hat{x}\) then \(\nabla f(\hat{x}) =0\) in the unconstrained case, and in the constrained case, the result ensures that the directional derivatives \(f'(\hat{x},; d)\) are nonnegative for every direction \(d \in {\mathbb{R}}^{n}\) that points in the contingent cone to the feasible region Ω, provided that Ω is regular [9]. The contingent cone generalizes the notion of tangent cone.
There are several intermediate results in the convergence analysis that involve different assumptions on f such as lowersemi continuity and regularity, and on the constraints such as properties of the hypertangent, Clarke tangent, and Bouligand cones. The fundamental theoretical result in the analysis was shown in [7, 9] and relies on Clarke’s [21] generalization of the directional derivative f ^{∘} for nonsmooth functions. The result states that \(f^{\circ}(\hat{x}; d) \geq0\) for every direction d in the hypertangent cone to Ω at \(\hat{x}\). A generalization of this result for discontinuous functions was recently shown in [45]. In the case where the progressive barrier [10] fails to generate feasible solutions, the analysis ensures that the constraint violation function satisfies \(h^{\circ}(\hat{x}; d) \geq0\) for every direction d in the hypertangent cone to X at \(\hat{x}\).
4.3 NOMAD: A C++ Implementation of the Mads Algorithm
This section describes the NOMAD software [32] which implements the Mads algorithm. We list several of its features, but do not expect to cover all of them in this chapter. NOMAD is a C++ code freely distributed under the LGPL license. The package is found at http://www.gerad.ca/nomad. It includes a complete documentation, a doxygen [44] interactive manual, and many examples and tools. As for other derivativefree codes, we expect as a rule of thumb that NOMAD will be efficient for problems with up to 50 variables.
4.3.1 Batch and Library Modes
NOMAD can be used in two different modes having various advantages. The user must choose with care the appropriate mode depending on its problem.
The second way to use the NOMAD algorithm is through the library mode. The user must write a C++ code which will be linked to the NOMAD static library included in the package. This way, interactions with NOMAD are directly performed via C++ function calls and objects manipulations. The optimization problem is described as a class and is written in C++ or in a compatible language such as C, FORTRAN, R, etc. The problem and algorithmic parameters are given as objects, and no parameters file is necessary. The library mode must be considered only by users with basic C++ knowledge. The problem must also be C++compatible in order to be expressed as a class, and hidden constraints need to be explicitly treated. If these points are addressed, the advantages of the library modes are numerous. First, when the blackbox is not costly, the execution will be much faster than the batch mode since no temporary files and no system calls are used. Second, numerical precision is not an issue because the communications between the algorithm and the problem occur at memory level. Third, more flexibility is possible with the use of callback functions that are userdefined and automatically called by NOMAD at key events, such as after a new success, or at the end of a Mads iteration. Last, the library mode is convenient when NOMAD is repeatedly called as a subroutine. Note finally that a hybrid use of the batch and library modes is possible. For example, one can define its problem as a C++ class and use a parameters file.
4.3.2 Important Algorithmic Parameters
 Starting point(s):

As for every optimization method, the starting point choice is crucial. The user has to provide his/her best guess so far for the method to be as efficient as possible. Within NOMAD, it is possible to define multiple starting points. This might be useful, for example, if in addition to a feasible initial solution, an infeasible one corresponding to a different and promising design is known.
 Initial mesh size and scaling:

In our opinion, the second most important algorithmic choice concerns the initial mesh size. Some defaults related to the scale of the starting point are used, but the user is encouraged to determine a good problemrelated value. Some automatic scaling is performed, but there again, users should sometimes consider changing the scale of their variables and study the impact.
 Poll directions:

The default type of poll directions is OrthoMads [3] but sometimes other direction types, such as LTMads [9], may perform well.
 Surrogates:

Static and dynamic surrogates can be used with NOMAD. Static surrogates are indicated by the user with the same format as the true blackbox. Concerning dynamic surrogates, the current NOMAD version 3.5 includes only quadratic models, but statistical surrogates will be available in a future release. Dynamic surrogates may be employed at two different levels: as a search, and as a way to sort a list of candidates before they are evaluated. The current NOMAD default is to use quadratic models at both places.
 Projection to bounds:

When generating candidates outside of the hyperrectangle defined by the bounds on the variables, NOMAD projects these points to the boundary, by default. For some problems, this strategy might not be the appropriate one.
 Seeds:

If the context allows multiple executions, changing the random seed for LTMads, or the Halton seed for OrthoMads [3], will lead to different executions.
 Termination criteria:

We finish this expose by indicating that many termination criteria are available, in addition to the obvious choice of a budget of evaluations.
4.3.3 Extensions of Mads
 Parallel and variable decomposition methods:

Three different parallel versions are available, using MPI. These methods are called PMads, CoopMads, and PsdMads. PMadssimply performs the evaluations in parallel, and two variants are available. First, the synchronous version waits for all ongoing parallel evaluations before iterating. This is opposed to the asynchronous variant inspired from [28] which iterates as soon as a new success is made, even if some evaluations are still not finished. In case one of these evaluations results in an improvement, the current iterate and the current mesh size are adjusted accordingly during a special update step. The two other parallel methods, CoopMads and PsdMads, are provided as tools in the package. The first executes several Mads instances in parallel with different seeds. Some cooperative actions are performed in order to guide the search. PsdMads performs the same collaborative process, but in addition, subgroups of variables are considered for each process. This technique is described in [11] and aims at solving larger problems (50 to 500 variables).
 Groups of variables:

The user with some knowledge of the problem can create groups of variables. Directions are then relative to these groups and variables from separate groups are not going to vary at the same time. This proved useful for localization problems and in particular the one presented in [4].
 Different types of variables:

It is possible to define integer and binary variables, which are treated by special meshes with a minimal size of one. Categorical variables may also be used. They are handled with the extended poll defined in [1, 31]. For such problems, the user must define a neighborhood structure, which may contain a different number of variables. NOMAD defines the concept of signature that allows such heterogeneous points. Other types of variables include fixed variables and periodic variables (angles, for example). The strategy used for the periodic variables is described in [14].
 Biobjective optimization:

In some situations, the user is interested in considering the tradeoffs between two conflicting objective functions. The method from [16] executes a series of singleobjective optimizations on reformulated versions of the original problem. These reformulations are not linear combinations of the objective, and ensure that nonconvex Pareto fronts can be identified.
 Variable Neighborhood Search (VNS):

For problems with many local optima, it is possible to enable the generic VNS search. This strategy has been described in [6] and uses a variable neighborhood search metaheuristic in order to escape local optima.
 Sensitivity analysis:

The last feature described here is a tool that uses biobjective optimization in order to conduct sensitivity analyses on the constraints, including bounds as well as nonrelaxable constraints. This is described in [13] with plots illustrating the impact on the objective of changing the righthand side of a given constraint.
4.4 Metamaterial Design Using NOMAD
This section describes the implementation of the NOMAD optimization routine [32] in combination with fullfield electromagnetic simulations to tailor the broadband spectral response of gold and silver split ring resonator metamaterials. By allowing NOMAD to “drive” finitedifference timedomain simulations, the spectral position of resonant reflection peaks and nearfield interactions within the metamaterial were tuned over a wide range of the nearinfrared spectrum. While this section discusses the design problems studied and the optimized results, a detailed discussion of the implementation used to communicate between the different software packages is provided in the Appendix.
4.4.1 Split Ring Resonator Optimization
In this section, the SRRs were illuminated with a broadband plane wave source from 1–4 μm, and the structure was parameterized based on the height, width, ring thickness (t _{1}), bar thickness (t _{2}), and gap width. For all simulations, the thickness of the metal was 100 nm, the Efield was perpendicular to the arms of the SRR, and the width of the bar was kept the same as the width of the SRR. Also, linear constraints were imposed to ensure that the parameters made physical sense (e.g., 2t _{1} ≥ width), and a gap was always present between the two arms of the SRRs.
SRR initial and optimized values  

Initial value  Gold optimized value  Silver optimized value  Minimum  Maximum  
Width  400  450  474  200  600 
Height  400  394  412  200  600 
t _{1}  100  125  152  50^{∗}  200^{∗} 
t _{2}  100  114  193  50  200 
Gap  100  192  184  50  200 
Width  500  455  438  200  600 
Height  500  476  480  200  600 
t _{1}  125  190  182  50^{∗}  200^{∗} 
t _{2}  125  94  105  50  200 
Gap  125  190  173  50  200 
Here “λ _{ P1}” is the shorter wavelength peak position, “λ _{ P2}” is the longer wavelength peak position, “λ _{ V }” is the position of the reflection minima between λ _{ P1} and λ _{ P2}, “I _{ P1}” is the intensity of the shorter wavelength peak, “I _{ P2}” is the intensity of the longer wavelength peak, and “I _{ V }” is the intensity of the reflection minima between λ _{ P1} and λ _{ P2}.
Figure 4.7 shows the optimization results for a matrix of initial conditions. The left panels (a & b) are simulations of gold SRRs, while the right panels (c & d) are silver SRRs. Table 4.1 details two sets of initial values for each of the resonators’ geometrical parameters. The top row of Fig. 4.7, panels (a & c), correspond to the first set of initial conditions. The bottom row, panels (b & d), correspond to the second starting point. For all four panels, the dashed curve represents the double Lorentzian target spectrum, the dotted curve represents the reflection spectrum from the starting point, and the solid curve represents the reflection spectrum from the optimized result. A close look at the results in Table 4.1 shows that for a each metal, there is some variability in the optimized results based on the starting point. This is not necessarily surprising considering the degeneracy in potential solutions for this fiveparameter optimization, and a relatively relaxed convergence tolerance specified during the optimizations. For all four cases, the results are especially encouraging based on the fact that the ideal double Lorentzian to which the curves were fit was arbitrarily chosen, and a perfect fit cannot necessarily be obtained with the given geometrical constraints and materials.
4.4.2 Split Ring Resonator Filters
Starting and optimized dimensions (in nm) for the SRR structures in Sect. 4.4.2 designed to act as a notch filter at λ=1310,1550, and 1800 nm
SRR filter initial and optimized values  

Initial value  1310 nm  1550 nm  1800 nm  Minimum  Maximum  
Width  500  348  424  508  200  600 
Height  500  356  420  492  200  600 
t _{1}  200  100  164  208  50^{∗}  200^{∗} 
t _{2}  100  124  140  104  50  200 
Gap  100  51  60  96  50  200 
x _{spacing}  200  128  120  192  50  1000 
y _{spacing}  200  330  392  436  50  1000 
As a final check of the solutions’ robustness in the previous sections, a systematic variation of each parameter of the 400 nm Au SRR in Sect. 4.4.1 and the 1500 nm SRR filter in Sect. 4.4.2 was performed near the optimized value. Variations from 1–3 % produced a corresponding change in the objective function of <0.3 % for the 400 nm Au SRR and <4 % for the 1500 nm notch filter. We conclude from this that the method is robust to local perturbations (which is an attribute of the relationship between the SRR geometry and the objective function, not the optimization method used).
4.4.3 Coupling Quantum Dots to SplitRing Resonators
While there exist a wide range of energy transitions for this type of system, for this example we consider second harmonic generation studies with resonances at both λ=3600 nm and λ=1800 nm, which would enhance both the absorption of light λ=3600 nm (ω _{Au}) and the coupling of light between the resonator and quantum dots at λ=1800 nm (∼2ω _{Au}), Fig. 4.9(b). For the case of gold resonators patterned on a sapphire substrate, the resonance wavelengths were chosen to closely match those of the lead selenide quantum dots.^{3}
The diagram in Fig. 4.9(b) shows typical broadband absorption spectra of lead selenide quantum dots (dotdashed line) overlaid on top of a target resonance spectrum of a metamaterial tuned to the wavelengths of interest (solid line). While a wide range of resonator geometries would satisfy the performance requirements specified above, throughout this chapter, variations on the basic SRR geometry have been studied, and will again be used in this section. Using λ=3600 nm and λ=1800 nm as the target spectrum, initial simulations showed that the basic SRR structure shown in Sects. 4.4.1 and 4.4.2 was unable to span the wavelength region of interest; however, the SRR variant shown in Fig. 4.9(a) would. In this case, the array was on a sapphire substrate and a sevenparameter NOMAD optimization was performed. While the array optimization in this situation was similar to that in Sect. 4.4.2; here the gap between the two SRRs was a design parameter with the lower bound set by fabrication constraints (50 nm), and the height of each SRR was varied independently. This last parameter is not to be confused with resonator thickness, which was set at 50 nm for all structures studied in this example. Finally, the total width and arm width for both SRRs was kept equal for all designs. As in Sects. 4.4.1 and 4.4.2, a nonlinear constraint was imposed within NOMAD to maintain a minimum gap between the arms of each SRR to maintain the general shape of the broadband reflection spectra.
Objective functions used for the design optimization of an array of Split Ring Resonators with resonances at λ=3600 nm and λ=1800 nm, coupled to quantum dots. For the above functions, \(I^{*}_{P1,2}=500(1  \textnormal{Intensity}_{P1,2})\). The resulting broadband resonance spectra are shown in Fig. 4.10
O.F. #  SRR initial and optimized values 

Objective function  
1  λ _{ P1}−1800+λ _{ P2}−3600/2 
2  \(\vert \lambda_{P1}  1800\vert + \vert \lambda_{P2}  3600\vert /2 + I^{*}_{P1}\) 
3  \(\vert \lambda_{P1}  1800\vert + \vert \lambda_{P2}  3600\vert /2 + I^{*}_{P2}\) 
4  \(\vert \lambda_{P1}  1800\vert + \vert \lambda_{P2}  3600\vert /2 + I^{*}_{P1} + I^{*}_{P2}\) 
5  λ _{ P1}−1800⋅λ _{ P2}−3600 
6  \(\vert \lambda_{P1}  1800\vert \cdot \vert \lambda_{P2}  3600\vert \cdot I^{*}_{P2}\) 
7  \(\vert \lambda_{P1}  1800\vert \cdot \vert \lambda_{P2}  3600\vert \cdot I^{*}_{P1}\) 
8  \(\vert \lambda_{P1}  1800\vert \cdot \vert \lambda_{P2}  3600\vert \cdot I^{*}_{P1} \cdot I^{*}_{P2}\) 
Starting and optimized dimensions (in nm) for an array of SRRs shown in Fig. 4.9(a) with dimensions optimized to resonate at both λ=3600 nm and λ=1800 nm. The columns “O.F. 1–6”, “O.F. 7”, and “O.F. 8”, correspond to the optimized dimensions obtained using Objective Functions #1–6, #7, and #8 from Table 4.3. The resulting broadband resonance spectrum is shown in Fig. 4.10
SRR initial and optimized values  

Initial value  O.F. 1–6  O.F. 7  O.F. 8  Minimum  Maximum  
Width  500  467±2  579  592  300  600 
Length 1  500  494±1  530  573  300  600 
t _{1}  100  134±1  198  196  50^{∗}  200^{∗} 
Length 2  500  494±1  494  549  300  600 
Gap  100  114±1  183  113  50  200 
x _{spacing}  100  306±8  374  296  100  500 
y _{spacing}  100  299±1  299  298  100  500 
While these resonances are close, but not a perfect match to the desired resonances, the fullwidth halfmaxima of the resonances for O.F. #8 at λ=1800 nm and λ=3600 nm are ∼240 nm and ∼675 nm respectively, and that of quantum dots such is ∼150 nm. Hence, while the inherently broad nature of the resonances from metamaterial arrays is normally considered a drawback; in this situation, it can compensate for some amount of discrepancy between the optimized peaks and the ω and 2ω targets.
Finally, it should be noted that this example is only a firstorder result. Here, the resonances of the quantum dots and the metamaterial are considered independently when setting the objective function targets. The addition of quantum dots into the nearfield of the SRRs, will introduce perturbations in the local dielectric environment and as a result, shift the actual resonances from those predicted in the simulations. While nonnegligible, this change is a secondorder effect and combined with the substantial bandwidth of the individual resonances, should not significantly affect the results of the example.
Footnotes
Notes
Acknowledgements
Work of the first author was supported by NSERC Discovery Grant 23943605 and AFOSR FA95500910160.
References
 1.M.A. Abramson, C. Audet, J.W. Chrissis, J.G. Walston, Mesh adaptive direct search algorithms for mixed variable optimization. Optim. Lett. 3(1), 35–47 (2009) MathSciNetzbMATHCrossRefGoogle Scholar
 2.M.A. Abramson, C. Audet, J.E. Dennis Jr., Filter pattern search algorithms for mixed variable constrained optimization problems. Pac. J. Optim. 3(3), 477–500 (2007) MathSciNetzbMATHGoogle Scholar
 3.M.A. Abramson, C. Audet, J.E. Dennis Jr., S. Le Digabel, OrthoMADS: a deterministic MADS instance with orthogonal directions. SIAM J. Optim. 20(2), 948–966 (2009) MathSciNetzbMATHCrossRefGoogle Scholar
 4.S. Alarie, C. Audet, V. Garnier, S. Le Digabel, L.A. Leclaire, Snow water equivalent estimation using blackbox optimization. Pac. J. Optim. 9(1), 1–21 (2013) zbMATHGoogle Scholar
 5.C. Audet, V. Béchard, J. Chaouki, Spent potliner treatment process optimization using a MADS algorithm. Optim. Eng. 9(2), 143–160 (2008) zbMATHCrossRefGoogle Scholar
 6.C. Audet, V. Béchard, S. Le Digabel, Nonsmooth optimization through mesh adaptive direct search and variable neighborhood search. J. Glob. Optim. 41(2), 299–318 (2008) zbMATHCrossRefGoogle Scholar
 7.C. Audet, J.E. Dennis Jr., Analysis of generalized pattern searches. SIAM J. Optim. 13(3), 889–903 (2003) MathSciNetzbMATHCrossRefGoogle Scholar
 8.C. Audet, J.E. Dennis Jr., A pattern search filter method for nonlinear programming without derivatives. SIAM J. Optim. 14(4), 980–1010 (2004) MathSciNetzbMATHCrossRefGoogle Scholar
 9.C. Audet, J.E. Dennis Jr., Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17(1), 188–217 (2006) MathSciNetzbMATHCrossRefGoogle Scholar
 10.C. Audet, J.E. Dennis Jr., A progressive barrier for derivativefree nonlinear programming. SIAM J. Optim. 20(1), 445–472 (2009) MathSciNetzbMATHCrossRefGoogle Scholar
 11.C. Audet, J.E. Dennis Jr., S. Le Digabel, Parallel space decomposition of the mesh adaptive direct search algorithm. SIAM J. Optim. 19(3), 1150–1170 (2008) MathSciNetzbMATHCrossRefGoogle Scholar
 12.C. Audet, J.E. Dennis Jr., S. Le Digabel, Globalization strategies for mesh adaptive direct search. Comput. Optim. Appl. 46(2), 193–215 (2010) MathSciNetzbMATHCrossRefGoogle Scholar
 13.C. Audet, J.E. Dennis Jr., S. Le Digabel, Tradeoff studies in blackbox optimization. Optim. Methods Softw. 27(4–5), 613–624 (2012) MathSciNetzbMATHCrossRefGoogle Scholar
 14.C. Audet, S. Le Digabel, The mesh adaptive direct search algorithm for periodic variables. Pac. J. Optim. 8(1), 103–119 (2012) MathSciNetzbMATHGoogle Scholar
 15.C. Audet, D. Orban, Finding optimal algorithmic parameters using derivativefree optimization. SIAM J. Optim. 17(3), 642–664 (2006) MathSciNetzbMATHCrossRefGoogle Scholar
 16.C. Audet, G. Savard, W. Zghal, Multiobjective optimization through a series of singleobjective formulations. SIAM J. Optim. 19(1), 188–210 (2008) MathSciNetzbMATHCrossRefGoogle Scholar
 17.A.J. Booker, J.E. Dennis Jr., P.D. Frank, D.B. Serafini, V. Torczon, Optimization using surrogate objectives on a helicopter test example, in Optimal Design and Control, ed. by J. Borggaard, J. Burns, E. Cliff, S. Schreck. Progress in Systems and Control Theory (Birkhäuser, Basel, 1998), pp. 49–58 Google Scholar
 18.A.J. Booker, J.E. Dennis Jr., P.D. Frank, D.B. Serafini, V. Torczon, M.W. Trosset, A rigorous framework for optimization of expensive functions by surrogates. Struct. Multidiscip. Optim. 17(1), 1–13 (1999) CrossRefGoogle Scholar
 19.A. Chipouline, V. Fedotov, Coupling plasmonics with quantum systems, in SPIE Newsroom (2011) Google Scholar
 20.T.D. Choi, C.T. Kelley, Superlinear convergence and implicit filtering. SIAM J. Optim. 10(4), 1149–1162 (2000) MathSciNetzbMATHCrossRefGoogle Scholar
 21.F.H. Clarke, Optimization and Nonsmooth Analysis (Wiley, New York, 1983). Reissued in 1990 by SIAM Publications, Philadelphia, as Vol. 5 in the series Classics in Applied Mathematics zbMATHGoogle Scholar
 22.A.R. Conn, S. Le Digabel, Use of quadratic models with mesh adaptive direct search for constrained black box optimization. Optim. Methods Softw. 28(1), 139–158 (2013) MathSciNetzbMATHCrossRefGoogle Scholar
 23.A.R. Conn, K. Scheinberg, L.N. Vicente, Introduction to DerivativeFree Optimization. MOS/SIAM Series on Optimization (SIAM, Philadelphia, 2009) zbMATHCrossRefGoogle Scholar
 24.J.E. Dennis Jr., C.J. Price, I.D. Coope, Direct search methods for nonlinearly constrained optimization using filters and frames. Optim. Eng. 5(2), 123–144 (2004) MathSciNetzbMATHCrossRefGoogle Scholar
 25.R. Fletcher, S. Leyffer, Nonlinear programming without a penalty function. Math. Program., Ser. A 91, 239–269 (2002) MathSciNetzbMATHCrossRefGoogle Scholar
 26.R.B. Gramacy, tgp: an R package for Bayesian nonstationary, semiparametric nonlinear regression and design by treed Gaussian process models. J. Stat. Softw. 19(9), 1–46 (2007). http://www.jstatsoft.org/v19/i09 Google Scholar
 27.R.B. Gramacy, S. Le Digabel, The mesh adaptive direct search algorithm with treed Gaussian process surrogates. Technical Report G201137, Les cahiers du GERAD, 2011 Google Scholar
 28.G.A. Gray, T.G. Kolda, Algorithm 856: APPSPACK 4.0: asynchronous parallel pattern search for derivativefree optimization. ACM Trans. Math. Softw. 32(3), 485–507 (2006) MathSciNetzbMATHCrossRefGoogle Scholar
 29.D.R. Jones, M. Schonlau, W.J. Welch, Efficient global optimization of expensive black box functions. J. Glob. Optim. 13(4), 455–492 (1998) MathSciNetzbMATHCrossRefGoogle Scholar
 30.S. Kim, J. Jin, Y.J. Kim, I.Y. Park, Y. Kim, S.W. Kim, Highharmonic generation by resonant plasmonic field enhancement. Nature 453, 757–760 (2008) ADSCrossRefGoogle Scholar
 31.M. Kokkolaras, C. Audet, J.E. Dennis Jr., Mixed variable optimization of the number and composition of heat intercepts in a thermal insulation system. Optim. Eng. 2(1), 5–29 (2001) MathSciNetzbMATHCrossRefGoogle Scholar
 32.S. Le Digabel, Algorithm 909: NOMAD: nonlinear optimization with the MADS algorithm. ACM Trans. Math. Softw. 37(4), 44 (2011) CrossRefGoogle Scholar
 33.L.Y. Lin, C.J. Wang, M.C. Hegg, L. Huang, Quantum dot nanophotonics—from waveguiding to integration. J. Nanophotonics 3, 031603 (2009) ADSCrossRefGoogle Scholar
 34.A.L. Marsden, J.A. Feinstein, C.A. Taylor, A computational framework for derivativefree optimization of cardiovascular geometries. Comput. Methods Appl. Mech. Eng. 197(21–24), 1890–1905 (2008) MathSciNetADSzbMATHCrossRefGoogle Scholar
 35.A.L. Marsden, M. Wang, J.E. Dennis Jr., P. Moin, Trailingedge noise reduction using derivativefree optimization and largeeddy simulation. J. Fluid Mech. 572, 13–36 (2007) MathSciNetADSzbMATHCrossRefGoogle Scholar
 36.M.D. McKay, W.J. Conover, R.J. Beckman, A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2), 239–245 (1979) MathSciNetzbMATHGoogle Scholar
 37.E.D. Palik, Handbook of Optical Constants of Solids (Academic Press, New York, 1997) Google Scholar
 38.E. Plum, V.A. Fedotov, P. Kuo, D.P. Tsai, N.I. Zheludev, Towards the lasing spaser: controlling metamaterial optical response with towards the lasing spaser: controlling metamaterial optical response with semiconductor quantum dots. Opt. Express 17(10), 8548–8551 (2009) ADSCrossRefGoogle Scholar
 39.E. Polak, M. Wetter, Precision control for generalized pattern search algorithms with adaptive precision function evaluations. SIAM J. Optim. 16(3), 650–669 (2006) MathSciNetzbMATHCrossRefGoogle Scholar
 40.N. Queipo, R. Haftka, W. Shyy, T. Goel, R. Vaidyanathan, P. Kevintucker, Surrogatebased analysis and optimization. Prog. Aerosp. Sci. 41(1), 1–28 (2005) CrossRefGoogle Scholar
 41.J. Renger, R. Quidant, N.V. Hulst, L. Novotny, Surface enhanced nonlinear fourwave mixing. Phys. Rev. Lett. 104(4), 046803 (2010) ADSCrossRefGoogle Scholar
 42.K. Tanaka, E. Plum, J.Y. Ou, T. Uchino, N.I. Zheludev, Multifold enhancement of quantum dot luminescence in plasmonic metamaterials. Phys. Rev. Lett. 105(22), 227403 (2010) ADSCrossRefGoogle Scholar
 43.V. Torczon, On the convergence of pattern search algorithms. SIAM J. Control Optim. 7(1), 1–25 (1997) MathSciNetzbMATHCrossRefGoogle Scholar
 44.D. van Heesch, Doxygen manual (1997). http://www.doxygen.org/manual.html
 45.L.N. Vicente, A.L. Custódio, Analysis of direct searches for discontinuous functions, in Mathematical Programming (2010) Google Scholar
 46.C.J. Wang, L.Y. Lin, B.A. Parviz, 100nm quantum dot waveguides by twolayer selfassembly, in Lasers and ElectroOptics Society, LEOS 2005. the 18th Annual Meeting of the IEEE (2005), pp. 194–195 Google Scholar