Abstract
This paper introduces a novel methodology for the global optimization of general constrained grey-box problems. A grey-box problem may contain a combination of black-box constraints and constraints with a known functional form. The novel features of this work include (i) the selection of initial samples through a subset selection optimization problem from a large number of faster low-fidelity model samples (when a low-fidelity model is available), (ii) the exploration of a diverse set of interpolating and non-interpolating functional forms for representing the objective function and each of the constraints, (iii) the global optimization of the parameter estimation of surrogate functions and the global optimization of the constrained grey-box formulation, and (iv) the updating of variable bounds based on a clustering technique. The performance of the algorithm is presented for a set of case studies representing an expensive non-linear algebraic partial differential equation simulation of a pressure swing adsorption system for \(\hbox {CO}_{2}\). We address three significant sources of variability and their effects on the consistency and reliability of the algorithm: (i) the initial sampling variability, (ii) the type of surrogate function, and (iii) global versus local optimization of the surrogate function parameter estimation and overall surrogate constrained grey-box problem. It is shown that globally optimizing the parameters in the parameter estimation model, and globally optimizing the constrained grey-box formulation has a significant impact on the performance. The effect of sampling variability is mitigated by a two-stage sampling approach which exploits information from reduced-order models. Finally, the proposed global optimization approach is compared to existing constrained derivative-free optimization algorithms.
Similar content being viewed by others
References
Audet, C., Bechard, V., Chaouki, J.: Spent potliner treatment process optimization using a MADS algorithm. Optim. Eng. 9(2), 143–160 (2008)
Bartholomew-Biggs, M.C., Parkhurst, S.C., Wilson, S.P.: Using DIRECT to solve an aircraft routing problem. Comput. Optim. Appl. 21(3), 311–323 (2002)
Boukouvala, F., Ierapetritou, M.G.: Surrogate-based optimization of expensive flowsheet modeling for continuous pharmaceutical manufacturing. J. Pharm. Innov. 8(2), 131–145 (2013)
Caballero, J.A., Grossmann, I.E.: An algorithm for the use of surrogate models in modular flowsheet optimization. AIChE J. 54(10), 2633–2650 (2008)
Egea, J.A., Rodriguez-Fernandez, M., Banga, J.R., Marti, R.: Scatter search for chemical and bio-process optimization. J. Glob. Optim. 37(3), 481–503 (2007)
Fahmi, I., Cremaschi, S.: Process synthesis of biodiesel production plant using artificial neural networks as the surrogate models. Comput. Chem. Eng. 46, 105–123 (2012)
Fowler, K.R., Reese, J.P., Kees, C.E., Dennis Jr, J.E., Kelley, C.T., Miller, C.T., Audet, C., Booker, A.J., Couture, G., Darwin, R.W., Farthing, M.W., Finkel, D.E., Gablonsky, J.M., Gray, G., Kolda, T.G.: Comparison of derivative-free optimization methods for groundwater supply and hydraulic capture community problems. Adv. Water Resour. 31(5), 743–757 (2008)
Graciano, J.E.A., Roux, G.A.C.L.: Improvements in surrogate models for process synthesis. Application to water network system design. Comput. Chem. Eng. 59, 197–210 (2013)
Hemker, T., Fowler, K., Farthing, M., Stryk, O.: A mixed-integer simulation-based optimization approach with surrogate functions in water resources management. Optim. Eng. 9(4), 341–360 (2008)
Henao, C.A., Maravelias, C.T.: Surrogate-based superstructure optimization framework. AIChE J. 57(5), 1216–1232 (2011)
Kleijnen, J.P.C., van Beers, W., van Nieuwenhuyse, I.: Constrained optimization in expensive simulation: Novel approach. Eur. J. Oper. Res. 202(1), 164–174 (2010)
Wan, X.T., Pekny, J.F., Reklaitis, G.V.: Simulation-based optimization with surrogate models—application to supply chain management. Comput. Chem. Eng. 29(6), 1317–1328 (2005)
Espinet, A., Shoemaker, C., Doughty, C.: Estimation of plume distribution for carbon sequestration using parameter estimation with limited monitoring data. Water Resour. Res. 49(7), 4442–4464 (2013)
Hasan, M.M.F., Baliban, R.C., Elia, J.A., Floudas, C.A.: Modeling, simulation, and pptimization of postcombustion \({\rm CO}_2\) capture for variable feed concentration and flow rate. 2. Pressure swing adsorption and vacuum swing adsorption processes. Ind. Eng. Chem. Res. 51(48), 15665–15682 (2013)
Hasan, M.M.F., Baliban, R.C., Elia, J.A., Floudas, C.A.: Modeling, simulation, and optimization of postcombustion \(\text{ CO }_2\) capture for variable feed concentration and flow rate. 1. Chemical absorption and membrane processes. Ind. Eng. Chem. Res. 51(48), 15642–15664 (2013)
Hasan, M.M.F., Boukouvala, F., First, E.L., Floudas, C.A.: Nationwide, regional, and statewide \(\text{ CO }_2\) capture, utilization, and sequestration supply chain network optimization. Ind. Eng. Chem. Res. 53(18), 7489–7506 (2014)
Li, S., Feng, L., Benner, P., Seidel-Morgenstern, A.: Using surrogate models for efficient optimization of simulated moving bed chromatography. Comput. Chem. Eng. 67, 121–132 (2014)
Forrester, A.I.J., Sóbester, A., Keane, A.J.: Engineering Design Via Surrogate Modelling—A Practical Guide. Wiley, Chichester (2008)
Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to Derivative-Free Optimization. MPS-SIAM Series on Optimization, vol. 8. SIAM, Philadelphia (2009)
Martelli, E., Amaldi, E.: PGS-COM: a hybrid method for constrained non-smooth black-box optimization problems: brief review, novel algorithm and comparative evaluation. Comput. Chem. Eng. 63, 108–139 (2014)
Rios, L.M., Sahinidis, N.V.: Derivative-free optimization: a review of algorithms and comparison of software implementations. J. Glob. Optim. 56(3), 1247–1293 (2013)
Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev. 45(3), 385–482 (2003)
Bjorkman, M., Holmstrom, K.: Global optimization of costly nonconvex functions using radial basis functions. Optim. Eng. 1(4), 373–397 (2000)
Booker, A.J., Dennis, J.E., Frank, P.D., Serafini, D.B., Torczon, V., Trosset, M.W.: A rigorous framework for optimization of expensive functions by surrogates. Struct. Multidiscip. Optim. 17(1), 1–13 (1999)
Boukouvala, F., Ierapetritou, M.G.: Derivative-free optimization for expensive constrained problems using a novel expected improvement objective function. AIChE J. 60(7), 2462–2474 (2014)
Conn, A.R., Le Digabel, S.: Use of quadratic models with mesh-adaptive direct search for constrained black box optimization. Optim. Methods Softw. 28(1), 139–158 (2013)
Forrester, A.I.J., Keane, A.J.: Recent advances in surrogate-based optimization. Prog. Aerosp. Sci. 45(1), 50–79 (2009)
Jakobsson, S., Patriksson, M., Rudholm, J., Wojciechowski, A.: A method for simulation based optimization using radial basis functions. Optim. Engi. 11(4), 501–532 (2010)
Jones, D.R.: A taxonomy of global optimization methods based on response surfaces. J. Global Optim. 21(4), 345–383 (2001)
Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. J. Global Optim. 13(4), 455–492 (1998)
Regis, R.G.: Constrained optimization by radial basis function interpolation for high-dimensional expensive black-box problems with infeasible initial points. Eng. Optim. 46(2), 218–243 (2014)
Regis, R.G., Shoemaker, C.A.: Constrained global optimization of expensive black box functions using radial basis functions. J. Glob. Optim. 31(1), 153–171 (2005)
Yao, W., Chen, X.Q., Huang, Y.Y., van Tooren, M.: A surrogate-based optimization method with RBF neural network enhanced by linear interpolation and hybrid infill strategy. Optim. Methods Softw. 29(2), 406–429 (2014)
Muller, J., Shoemaker, C.A.: Influence ensemble surrogate models and sampling strategy on the solution quality of algorithms for computationally expensive black-box global optimization methods. J. Glob. Optim. 60(2), 123–144 (2014)
Viana, F.A.C., Haftka, R.T., Watson, L.T.: Efficient global optimization algorithm assisted by multiple surrogate techniques. J. Glob. Optim. 56(2), 669–689 (2013)
Davis, E., Ierapetritou, M.: A kriging method for the solution of nonlinear programs with black-box functions. AIChE J. 53(8), 2001–2012 (2007)
Floudas, C.A.: Deterministic Global Optimization, vol. 37. Springer, Berlin (1999)
Floudas, C.A., Gounaris, C.E.: A review of recent advances in global optimization. J. Glob. Optim. 45(1), 3–38 (2009)
First, E.L., Hasan, M.M.F., Floudas, C.A.: Discovery of novel zeolites for natural gas purification through combined material screening and process optimization. AICHE J. 60(5), 1767–1785 (2014)
Hasan, M.M.F., First, E.L., Floudas, C.A.: Cost-effective \(\text{ CO }_2\) capture based on in silico screening of zeolites and process optimization. Phys. Chem. Chem. Phys. 15(40), 17601–17618 (2013)
Abramson, M.: Pattern Search Algorithms for Mixed Variable General Constrained Optimization Problems. Rice University, Houston (2002)
Audet, C., Dennis, J.: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17(1), 188–217 (2006)
Audet, C., Dennis Jr, J.E.: A progressive barrier for derivative-free nonlinear programming. SIAM J. Optim. 20(1), 445–472 (2009)
Holmstrom, K., Quttineh, N.-H., Edvall, M.M.: An adaptive radial basis algorithm (ARBF) for expensive black-box mixed-integer constrained global optimization. Optim. Eng. 9(4), 311–339 (2008)
Audet, C., Dennis Jr, J.E.: A pattern search filter method for nonlinear programming without derivatives. SIAM J. Optim. 14(4), 980–1010 (2004)
Parr, J.M., Keane, A.J., Forrester, A.I.J., Holden, C.M.E.: Infill sampling criteria for surrogate-based optimization with constraint handling. Eng. Optim. 44(10), 1147–1166 (2012)
Sasena, M.J., Papalambros, P., Goovaerts, P.: Exploration of metamodeling sampling criteria for constrained global optimization. Eng. Optim. 34(3), 263–278 (2002)
Regis, R.G.: Stochastic radial basis function algorithms for large-scale optimization involving expensive black-box objective and constraint functions. Comput. Oper. Res. 38(5), 837–853 (2011)
Abramson, M., Audet, C., Dennis, J., Digabel, S.: OrthoMADS: a deterministic MADS instance with orthogonal directions. SIAM J. Optim. 20(2), 948–966 (2009)
Audet, C., Bechard, V., Digabel, S.: Nonsmooth optimization through mesh adaptive direct search and variable neighborhood search. J. Glob. Optim. 41(2), 299–318 (2008)
Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev. 45(3), 385–482 (2003)
Wild, S.M., Regis, R.G., Shoemaker, C.A.: Orbit: optimization by radial basis function interpolation in trust-regions. SIAM J. Sci. Comput. 30(6), 3197–3219 (2008)
Sankaran, S., Audet, C., Marsden, A.L.: A method for stochastic constrained optimization using derivative-free surrogate pattern search and collocation. J. Comput. Phys. 229(12), 4664–4682 (2010)
Le Digabel, S.: Algorithm 909: NOMAD: nonlinear optimization with the MADS algorithm. ACM Trans. Math. Softw. (TOMS) 37(4), 1–15 (2011)
Oeuvray, R.: Trust-region methods based on radial basis functions with application to biomedical imaging. PhD in Mathematics, Ecole Polytechnique Federale de Lausanne (2005)
Powell, M.J.D.: The BOBYQA Algorithm for Bound Constrained Optimization Without Derivatives. Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge (2009)
Regis, R.G., Shoemaker, C.A.: Parallel radial basis function methods for the global optimization of expensive functions. Eur. J. Oper. Res. 182(2), 514–535 (2007)
Regis, R.G., Shoemaker, C.A.: A stochastic radial basis function method for the global optimization of expensive functions. INFORMS J. Comput. 19(4), 497–509 (2007)
Regis, R.G., Shoemaker, C.A.: Improved strategies for radial basis function methods for global optimization. J. Glob. Optim. 37(1), 113–135 (2007)
Regis, R.G., Shoemaker, C.A.: A quasi-multistart framework for global optimization of expensive functions using response surface models. J. Glob. Optim. 56(4), 1719–1753 (2013)
Powell, M.J.D.: A direct search optimization method that models the objective and constraint functions by linear interpolation. In: Gomez, S., Hennart, J.-P. (eds.) Advances in Optimization and Numerical Analysis: Mathematics and its Applications, vol. 275. Mathematics and Its Applications, pp. 51–67. Springer, Berlin (1994)
Quan, N., Yin, J., Ng, S.H., Lee, L.H.: Simulation optimization via kriging: a sequential search using expected improvement with computing budget constraints. IIE Trans. 45(7), 763–780 (2013)
Torn, A., Zilinskas, A.: Global optimization. In: Lecture Notes in Computer Science, vol. 350. Springer, Berlin (1989)
Regis, R.G.: Evolutionary programming for high-dimensional constrained expensive black-box optimization using radial basis functions. IEEE Trans. Evol. Comput. 18(3), 326–347 (2014). doi:10.1109/TEVC.2013.2262111
Misener, R., Floudas, C.A.: Global optimization of mixed-integer models with quadratic and signomial functions: a review. Appl. Comput. Math. 11, 317–336 (2012)
Misener, R., Floudas, C.A.: GloMIQO: global mixed-integer quadratic optimizer. J. Glob. Optim. 57(1), 3–50 (2013)
Misener, R., Floudas, C.: ANTIGONE: algorithms for continuous/integer global optimization of nonlinear equations. J. Glob. Optim. 59(2–3), 503–526 (2014)
Mckay, M.D., Beckman, R.J., Conover, W.J.: A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 42(1), 55–61 (2000)
Kahrs, O., Marquardt, W.: The validity domain of hybrid models and its application in process optimization. Chem. Eng. Process. 46(11), 1054–1066 (2007)
Sobieszczanski-Sobieski, J., Haftka, R.T.: Multidisciplinary aerospace design optimization: survey of recent developments. Struct. Optim. 14(1), 1–23 (1997). doi:10.1007/BF01197554
Willcox, K., Peraire, J.: Balanced model reduction via the proper orthogonal decomposition. AIAA J. 40(11), 2323–2330 (2002)
Lucia, D.J., Beran, P.S., Silva, W.A.: Reduced-order modeling: new approaches for computational physics. Prog. Aerosp. Sci. 40(1–2), 51–117 (2004)
Li, Z., Floudas, C.A.: Optimal scenario reduction framework based on distance of uncertainty distribution and output performance: I. Single reduction via mixed integer linear optimization. Comput. Chem. Eng. 70, 50–65 (2014)
Cressie, N.: Statistics for Spatial Data. Wiley Series in Probability and Statistics. Wiley-Interscience, New York (1993)
Kleijnen, J.P.C.: Kriging metamodeling in simulation: a review. Eur. J. Oper. Res. 192(3), 707–716 (2009)
Sacks, J., Welch, W.J., Toby, J.M., Wynn, H.P.: Design and analysis of computer experiments. Stat. Sci. 4(4), 409–423 (1989)
Myers, R.H., Montgomery, D.C.: Response Surface Methodology: Process and Product in Optimization Using Designed Experiments. Wiley, New York (1995)
Bjork, K.-M., Lindberg, P.O., Westerlund, T.: Some convexifications in global optimization of problems containing signomial terms. Comput. Chem. Eng. 27(5), 669–679 (2003)
Gramacy, R.B., Lee, H.K.H.: Optimization Under Unknown Constraints. University of Cambridge, Cambridge (2010)
Lin, L.-C., Berger, A., Martin, R., Kim, J., Swisher, J., Jariwala, K., Rycroft, C., Bhown, A., Deem, M., Haranczyk, M., Smit, B.: In silico screening of carbon-capture materials. Nat. Mater. 11(7), 633–641 (2012)
Siriwardane, R.V., Shen, M.-S., Fisher, E.P., Poston, J.A.: Adsorption of \(\text{ CO }_2\) on molecular sieves and activated carbon. Energy Fuels 15(2), 279–284 (2001)
Zhang, J., Webley, P.A., Xiao, P.: Effect of process parameters on power requirements of vacuum swing adsorption technology for \(\text{ CO }_2\) capture from flue gas. Energy Convers. Manag. 49(2), 346–356 (2008)
Drud, A.: CONOPT—a large-scale GRG code. ORSA J. Comput. 6, 207–216 (1992)
Egea, J.A., Martí, R., Banga, J.R.: An evolutionary method for complex-process optimization. Comput. Oper. Res. 37(2), 315–324 (2010)
Johnson, S.G.: The NLopt nonlinear-optimization package. http://ab-initio.mit.edu/nlopt
Holmstrom, K., Goran, A.O., Edvall, M.M.: Users Guide for TOMLAB CGO. http://tomopt.com/docs/TOMLAB_CGO.pdf (2008)
Acknowledgments
The authors acknowledge financial support from the National Science Foundation (CBET-0827907, CBET-1263165).
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix 1: Model equations
Appendix 2: Nomenclature
Indices
- i :
-
Component
- n :
-
Packed column
Sets
The set of all components is given as follows: \(i\in I=\{ CO _2,N_2\}\)
Parameters
The following parameters are defined:
- F :
-
Feed flow rate
- \(y_{i,f}\) :
-
Mole fraction of component i in the feed gas
- \(T_o\) :
-
Feed temperature
- \(P_f\) :
-
Feed flow rate
- \(u_o\) :
-
Feed velocity
- \(T_a\) :
-
Outside air temperature
- \(y_{i,0}\) :
-
Initial mole fraction of component i in the bed
- \(P_{atm}\) :
-
Atmospheric pressure
- \(K_z\) :
-
Effective heat conductivity
- \(K_w\) :
-
Thermal heat conductivity of column wall
- \(h_{in}\) :
-
Heat transfer coefficient inside the column
- \(h_o\) :
-
Heat transfer coefficient for outside the column
- \(C_{pw}\) :
-
Specific heat capacity of column wall
- \(C_{pg}\) :
-
Specific heat capacity of gas mixture
- \(C_{pa}\) :
-
Specific heat capacity of adsorbed gas in solid
- \(C_{ps}\) :
-
Specific heat capacity of adsorbent
- R :
-
Gas constant
- \(q_s\) :
-
Saturation capacity
- \(b_i^o\) :
-
Isotherm parameter for component i
- \(\Delta U_i\) :
-
Heat of adsorption for component i
- \(\tau _p\) :
-
Tortuosity factor
- \(D_M\) :
-
Molecular diffusivity of \({ CO }_2 -N_2\,mixture\)
- \(D_p \) :
-
Macropore diffusivity, \(D_p =\frac{D_M }{\tau _p }\)
- \(D_L\) :
-
Axial dispersion coefficient defined by the correlation: \(D_L =0.7D_M +0.5u_o d_p \)
- UC :
-
Unit operating cost
- \(d_p\) :
-
Particle diameter
- \(r_p\) :
-
Particle radius, \(\frac{d_p }{2}\)
- \(\rho _w\) :
-
Density of column
- \(\rho _s\) :
-
Density of solid particle
- \(\mu \) :
-
Gas viscosity
- \(\varepsilon _p\) :
-
Particle porosity
- \(\varepsilon \) :
-
Bed porosity
- \(\phi \) :
-
Annualization factor
- \(\alpha \) :
-
Unit capital cost
- \(\beta \) :
-
Exponent of power cost
- \(k_p\) :
-
Bed permeability defined by the correlation: \(k_p =\frac{d_p^2 }{150}\left( {\frac{\varepsilon }{1-\varepsilon }} \right) ^{2}\)
Variables
We consider the variation both in time and along the column length. To denote this, we define:
- l :
-
Bed length
- t :
-
Time
The following system variables are defined to describe the system:
- P(l, t):
-
Pressure
- T(l, t):
-
Temperature
- \(T_W (l,t)\) :
-
Column wall temperature
- \(u_z (l,t)\) :
-
Velocity
- \(c_i (l,t)\) :
-
Concentration of component i in the gas phase
- \(y_i (l,t)\) :
-
Mole fraction of component i in the gas phase
- \(x_i (l,t)\) :
-
Fractional loading of component i in the solid phase
- \(x_i^*(l,t)\) :
-
Equilibrium fractional loading of component i in the solid phase
- \(k_i (l,t)\) :
-
Mass transfer coefficient of component i
- \(b_i (l,t)\) :
-
Temperature dependent parameter for component i
- \(B_i (l,t)\) :
-
An isotherm parameter for component i
- \(\Delta H_i (l,t)\) :
-
Change in internal energy for component i
- \(\rho _g (l,t)\) :
-
Density of gas obtained from the ideal gas law
The decision variables for process optimization are as follows:
- N :
-
Number of columns
- \(P_{ads}\) :
-
Highest pressure of the system
- L :
-
Column length
- D :
-
Column diameter
- \(P_{bd}\) :
-
Blowdown pressure
- \(P_{evac}\) :
-
Evacuation pressure
- \(t_{pr}\) :
-
Duration of the pressurization step, 20
- \(\hbox {s}t_{ads}\) :
-
Duration of the adsorption step
- \(t_{bd}\) :
-
Duration of the blowdown step
- \(t_{evac}\) :
-
Duration of the evacuation step
Variable scaling
The model is described in terms of these dimensionless variables:
We also need appropriate boundary and initial conditions to describe the four steps, since they are different for different steps. These are defined subsequently.
Initial and boundary conditions
Pressurization The initial conditions for the first pressurization step are given by the following equation:
In a 4-step cyclic process, the initial conditions for each subsequent step are the final conditions of the previous step, so initial conditions are not specified for the remaining steps.
Let \(\hbox {Z} = 0^{+}\) and \(\hbox {Z} = 1^{-}\) be the two ends of an adsorption column where the boundary conditions must be applied. The boundary conditions at \(\hbox {Z} = 0^{+}\) for the pressurization step are as follows:
The boundary conditions at \(\hbox {Z} = 1^{-}\) are:
Adsorption The boundary conditions at \(\hbox {Z}= 0^{+}\) and \(\hbox {Z}= 1^{-}\) for the adsorption step are the following:
where \(Pe_H =\frac{\varepsilon u_o L\rho _g C_{pg} }{K_z }\)
Blowdown The boundary conditions at \(\hbox {Z} = 0^{+}\) and \(\hbox {Z} = 1^{-}\) for the blowdown step are given by:
Evacuation The boundary conditions at \(\hbox {Z} = 0^{+}\) and \(\hbox {Z} = 1^{-}\) for the evacuation step are:
Rights and permissions
About this article
Cite this article
Boukouvala, F., Hasan, M.M.F. & Floudas, C.A. Global optimization of general constrained grey-box models: new method and its application to constrained PDEs for pressure swing adsorption. J Glob Optim 67, 3–42 (2017). https://doi.org/10.1007/s10898-015-0376-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10898-015-0376-2