Abstract
In the context of algorithmic parameter optimization, there is much room for efficient usage of computational resources. We consider the Opal framework in which a nonsmooth optimization problem models the parameter identification task, and is solved by a mesh adaptive direct search solver. Each evaluation of trial parameters requires the processing of a potentially large number of independent tasks. We describe and evaluate several strategies for using parallelism in this setting. Our test scenario consists in optimizing five parameters of a trust-region method for smooth unconstrained minimization.
Similar content being viewed by others
References
Abramson, M., Audet, C., Couture, G., Dennis, J. Jr., Le Digabel, S.: The NOMAD project. Software available at http://www.gerad.ca/nomad
Abramson M., Audet C., Dennis J. Jr., Le Digabel S.: OrthoMADS: a deterministic MADS instance with orthogonal directions. SIAM. J. Optim. 20(2), 948–966 (2009). doi: 10.1137/080716980
Audet C., Béchard V., Le Digabel S.: Nonsmooth optimization through mesh adaptive direct search and variable neighborhood search. J. Glob. Optim. 41(2), 299–318 (2008). doi:10.1007/s10898-007-9234-1
Audet, C., Dang, C.K., Orban, D.: Algorithmic parameter optimization of the DFO method with the OPAL framework. In: Naono, K., Teranishi, K, Cavazos, J, Suda, R. (eds.) Software Automatic Tuning: From Concepts to State-of-the-Art Results, chap. 15, pp. 255–274. Springer, New york (2010)
Audet C., Dennis J. Jr: Analysis of generalized pattern searches. SIAM J. Optim 13(3), 889–903 (2003). doi:10.1137/S1052623400378742
Audet C., Dennis J. Jr: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17(1), 188–217 (2006)
Audet C., Dennis J. Jr: A progressive barrier for derivative-free nonlinear programming. SIAM J. Optim. 20(4), 445–472 (2009). doi:10.1137/070692662
Audet C., Dennis J. Jr, Le Digabel S.: Parallel space decomposition of the mesh adaptive direct search algorithm. SIAM J. Optim. 19(3), 1150–1170 (2008). doi:10.1137/070707518
Audet C., Dennis J. Jr, Le Digabel S.: Globalization strategies for mesh adaptive direct search. Comput. Optim. Appl. 46(2), 193–215 (2010). doi:10.1007/s10589-009-9266-1
Audet C., Orban D.: Finding optimal algorithmic parameters using derivative-free optimization. SIAM J. Optim. 17(3), 642–664 (2006). doi:10.1137/040620886
Choi T., Kelley C.: Superlinear convergence and implicit filtering. SIAM J. Optim. 10(4), 1149–1162 (2000)
Clarke, F.: Optimization and Nonsmooth Analysis. Wiley, New York (1983) (reissued in 1990 by SIAM Publications, Philadelphia, as Vol. 5 in the series Classics in Applied Mathematics)
Conn A., Gould N., Toint P.: Trust-Region Methods. MPS-SIAM Series on Optimization. SIAM, Philadelphia (2000)
Conn, A., Scheinberg, K., Vicente, L.: Introduction to Derivative-Free Optimization. MOS/SIAM Series on Optimization. SIAM, Philadelphia (2009)
Gilmore, P., Choi, T., Eslinger, O., Kelley, C., Patrick, H., Gablonsky, J.: IFFCO (implicit filtering for constrained optimization). Software available at: http://www4.ncsu.edu/~ctk/iffco.html
Gould N., Lucidi S., Toint P.: Solving the trust-region subproblem using the Lanczos method. SIAM J. Optim. 9(2), 504–525 (1999)
Gould N., Orban D., Toint P.: CUTEr (and SifDec): a constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29(4), 373–394 (2003). doi:10.1145/962437.962439
Gould N., Orban D., Toint P.: GALAHAD, a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization. ACM Trans. Math. Softw. 29(4), 353–372 (2003). doi:10.1145/962437.962438
Gray G., Kolda T.: Algorithm 856: APPSPACK 4.0: Asynchronous parallel pattern search for derivative-free optimization. ACM Trans. Math. Softw. 32(3), 485–507 (2006). doi:10.1145/1163641.1163647
Griffin J., Kolda T., Lewis R.: Asynchronous parallel generating set search for linearly-constrained optimization. SIAM J. Sci. Comp. 30(4), 1892–1924 (2008). doi:10.1137/060664161
Gropp W., Lusk E., Skjellum A.: Using MPI: portable parallel programming with the message- passing interface, 2nd edn. Scientific and Engineering Computation Series. MIT Press, Cambridge (1994)
Le Digabel, S.: NOMAD user guide. Tech. Rep. G-2009-37, Les cahiers du GERAD (2009)
Le Digabel S.: Algorithm 909: NOMAD: Nonlinear optimization with the MADS algorithm. ACM Trans. Math. Softw. 37(4), 44:1–44:15 (2011). doi:10.1145/1916461.1916468
Pardalos P.: Parallel Processing of Discrete Problems, IMA Volumes in Mathematics and its Applications, vol. 106. Springer, Berlin (1999)
Shylo O., Middelkoop T., Pardalos P.: Restart strategies in optimization: parallel and serial cases. Parallel Comput. 37(1), 60–68 (2011)
Author information
Authors and Affiliations
Corresponding author
Additional information
Research of the first author is partially supported by NSERC Discovery Grant 239436-05, Afosr FA9550-09-1-0160, and ExxonMobil Upstream Research Company EM02562. Research of the third author is partially supported by NSERC Discovery Grant 299010-04.
Rights and permissions
About this article
Cite this article
Audet, C., Dang, CK. & Orban, D. Efficient use of parallelism in algorithmic parameter optimization applications. Optim Lett 7, 421–433 (2013). https://doi.org/10.1007/s11590-011-0428-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11590-011-0428-6