Skip to main content
SpringerLink
Log in

Efficient use of parallelism in algorithmic parameter optimization applications

  • Original Paper
  • Published:
Optimization Letters Aims and scope Submit manuscript

Abstract

In the context of algorithmic parameter optimization, there is much room for efficient usage of computational resources. We consider the Opal framework in which a nonsmooth optimization problem models the parameter identification task, and is solved by a mesh adaptive direct search solver. Each evaluation of trial parameters requires the processing of a potentially large number of independent tasks. We describe and evaluate several strategies for using parallelism in this setting. Our test scenario consists in optimizing five parameters of a trust-region method for smooth unconstrained minimization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Abramson, M., Audet, C., Couture, G., Dennis, J. Jr., Le Digabel, S.: The NOMAD project. Software available at http://www.gerad.ca/nomad

  2. Abramson M., Audet C., Dennis J. Jr., Le Digabel S.: OrthoMADS: a deterministic MADS instance with orthogonal directions. SIAM. J. Optim. 20(2), 948–966 (2009). doi: 10.1137/080716980

    Article  MathSciNet  MATH  Google Scholar 

  3. Audet C., Béchard V., Le Digabel S.: Nonsmooth optimization through mesh adaptive direct search and variable neighborhood search. J. Glob. Optim. 41(2), 299–318 (2008). doi:10.1007/s10898-007-9234-1

    Article  MATH  Google Scholar 

  4. Audet, C., Dang, C.K., Orban, D.: Algorithmic parameter optimization of the DFO method with the OPAL framework. In: Naono, K., Teranishi, K, Cavazos, J, Suda, R. (eds.) Software Automatic Tuning: From Concepts to State-of-the-Art Results, chap. 15, pp. 255–274. Springer, New york (2010)

  5. Audet C., Dennis J. Jr: Analysis of generalized pattern searches. SIAM J. Optim 13(3), 889–903 (2003). doi:10.1137/S1052623400378742

    Article  MathSciNet  MATH  Google Scholar 

  6. Audet C., Dennis J. Jr: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17(1), 188–217 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  7. Audet C., Dennis J. Jr: A progressive barrier for derivative-free nonlinear programming. SIAM J. Optim. 20(4), 445–472 (2009). doi:10.1137/070692662

    Article  MathSciNet  MATH  Google Scholar 

  8. Audet C., Dennis J. Jr, Le Digabel S.: Parallel space decomposition of the mesh adaptive direct search algorithm. SIAM J. Optim. 19(3), 1150–1170 (2008). doi:10.1137/070707518

    Article  MathSciNet  MATH  Google Scholar 

  9. Audet C., Dennis J. Jr, Le Digabel S.: Globalization strategies for mesh adaptive direct search. Comput. Optim. Appl. 46(2), 193–215 (2010). doi:10.1007/s10589-009-9266-1

    Article  MathSciNet  MATH  Google Scholar 

  10. Audet C., Orban D.: Finding optimal algorithmic parameters using derivative-free optimization. SIAM J. Optim. 17(3), 642–664 (2006). doi:10.1137/040620886

    Article  MathSciNet  MATH  Google Scholar 

  11. Choi T., Kelley C.: Superlinear convergence and implicit filtering. SIAM J. Optim. 10(4), 1149–1162 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  12. Clarke, F.: Optimization and Nonsmooth Analysis. Wiley, New York (1983) (reissued in 1990 by SIAM Publications, Philadelphia, as Vol. 5 in the series Classics in Applied Mathematics)

  13. Conn A., Gould N., Toint P.: Trust-Region Methods. MPS-SIAM Series on Optimization. SIAM, Philadelphia (2000)

    Book  Google Scholar 

  14. Conn, A., Scheinberg, K., Vicente, L.: Introduction to Derivative-Free Optimization. MOS/SIAM Series on Optimization. SIAM, Philadelphia (2009)

  15. Gilmore, P., Choi, T., Eslinger, O., Kelley, C., Patrick, H., Gablonsky, J.: IFFCO (implicit filtering for constrained optimization). Software available at: http://www4.ncsu.edu/~ctk/iffco.html

  16. Gould N., Lucidi S., Toint P.: Solving the trust-region subproblem using the Lanczos method. SIAM J. Optim. 9(2), 504–525 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  17. Gould N., Orban D., Toint P.: CUTEr (and SifDec): a constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29(4), 373–394 (2003). doi:10.1145/962437.962439

    Article  MathSciNet  MATH  Google Scholar 

  18. Gould N., Orban D., Toint P.: GALAHAD, a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization. ACM Trans. Math. Softw. 29(4), 353–372 (2003). doi:10.1145/962437.962438

    Article  MathSciNet  MATH  Google Scholar 

  19. Gray G., Kolda T.: Algorithm 856: APPSPACK 4.0: Asynchronous parallel pattern search for derivative-free optimization. ACM Trans. Math. Softw. 32(3), 485–507 (2006). doi:10.1145/1163641.1163647

    Article  MathSciNet  MATH  Google Scholar 

  20. Griffin J., Kolda T., Lewis R.: Asynchronous parallel generating set search for linearly-constrained optimization. SIAM J. Sci. Comp. 30(4), 1892–1924 (2008). doi:10.1137/060664161

    Article  MathSciNet  Google Scholar 

  21. Gropp W., Lusk E., Skjellum A.: Using MPI: portable parallel programming with the message- passing interface, 2nd edn. Scientific and Engineering Computation Series. MIT Press, Cambridge (1994)

    Google Scholar 

  22. Le Digabel, S.: NOMAD user guide. Tech. Rep. G-2009-37, Les cahiers du GERAD (2009)

  23. Le Digabel S.: Algorithm 909: NOMAD: Nonlinear optimization with the MADS algorithm. ACM Trans. Math. Softw. 37(4), 44:1–44:15 (2011). doi:10.1145/1916461.1916468

    Article  MathSciNet  Google Scholar 

  24. Pardalos P.: Parallel Processing of Discrete Problems, IMA Volumes in Mathematics and its Applications, vol. 106. Springer, Berlin (1999)

    Book  Google Scholar 

  25. Shylo O., Middelkoop T., Pardalos P.: Restart strategies in optimization: parallel and serial cases. Parallel Comput. 37(1), 60–68 (2011)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. GERAD and Département de Mathématiques et de Génie Industriel, École polytechnique de Montréal, Montreal, Canada

    C. Audet, C.-K. Dang & D. Orban

Authors
  1. C. Audet
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. C.-K. Dang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. D. Orban
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to C. Audet.

Additional information

Research of the first author is partially supported by NSERC Discovery Grant 239436-05, Afosr FA9550-09-1-0160, and ExxonMobil Upstream Research Company EM02562. Research of the third author is partially supported by NSERC Discovery Grant 299010-04.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Audet, C., Dang, CK. & Orban, D. Efficient use of parallelism in algorithmic parameter optimization applications. Optim Lett 7, 421–433 (2013). https://doi.org/10.1007/s11590-011-0428-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11590-011-0428-6

Keywords

Search

Navigation