Advertisement

Parallel Computing in Optimization

  • Robert B. Schnabel
Part of the NATO ASI Series book series (volume 15)

Abstract

One of the major developments in computing in recent years has been the introduction of a variety of parallel computers, and the development of algorithms that effectively utilize their capabilities. Very little of this parallel algorithm development, however, has been in numerical optimization. Nevertheless, significant opportunities exist for the utilization of parallelism in optimization, especially on computers that support independent concurrent processes. This paper first gives a very brief survey of parallel architectures and general characteristics of parallel algorithms. Next we indicate what we see as the leading opportunities for the utilization of parallelism in optimization. Then we survey the small amount of existing research in parallel optimization; most of this has been conducted at The Hatfield Polytechnic. Finally we discuss some recently initiated research at the University of Colorado concerned with solving optimization problems by parallel algorithms suitable for implementation on a local area network of computers; we focus on a new parallel algorithm for global optimization.

Keywords

Parallel Algorithm Conjugate Gradient Method Local Area Network Parallel Architecture Sequential Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. L. Adams [1983], “An M-step preconditioned conjugate gradient method for parallel computation”, Proceedings of the 1983 International Conference on Parallel Processing, pp. 36–43.Google Scholar
  2. L. Adams and J. Ortega [1982], “A multi-color SOR method for parallel computation”, Proceedings of the 1982 International Conference on Parallel Processing, pp. 53–56.Google Scholar
  3. G. Baudet [1978], “Asynchronous iterative methods for multiprocessors”, Journal of the Association for Computing Machinery 25, pp. 226–244.Google Scholar
  4. C. G. E. Boender, A. H. G. Rinnooy Kan, L. Stougie, and G. T. Timmer [1982], “A stochastic method for global optimization”, Mathematical Programming 22, pp. 125–140.Google Scholar
  5. J. E. Dennis Jr. and R. B. Schnabel [1983], Numerical Methods for Nonlinear Equations and Unconstrained Optimization, Prentice-Hall, Englewood Cliffs, New Jersey.Google Scholar
  6. L. C. W. Dixon [1981], “The place of parallel computation in numerical optimization I, the local problem”, Technical Report No. 118, Numerical Optimisation Centre, The Hatfield Polytechnic.Google Scholar
  7. L. C. W. Dixon and K. D. Patel [1981], “The place of parallel computation in numerical optimization II, the multiextremal global optimisation problem”, Technical Report No. 119, Numerical Optimisation Centre, The Hatfield Polytechnic.Google Scholar
  8. L. C. W. Dixon and K. D. Patel [1982], “The place of parallel computation in numerical optimization IV, parallel algorithms for nonlinear optimisation”, Technical Report No. 125, Numerical Optimisation Centre, The Hatfield Polytechnic.Google Scholar
  9. L. C. W. Dixon, K. D. Patel, and P. G. Ducksbury [1983], “Experience running optimisation algorithms on parallel processing systems”, Technical Report No. 138, Numerical Optimisation Centre, The Hatfield Polytechnic.Google Scholar
  10. L. C. W. Dixon, P. G. Ducksbury, and P. Singh [1982], “A parallel version of the conjugate gradient algorithm for finite element problems”, Technical Report No. 132, Numerical Optimisation Centre, The Hatfield Polytechnic.Google Scholar
  11. J. J. Dongarra and R. E. Hiromoto [1983], “A collection of parallel linear equations routines for the Denelcor HEP”, Technical Report ANL/MCS-TM-15, Mathematics and Computer Science Divsion, Argonne National Laboratory.Google Scholar
  12. P. G. Ducksbury [1982], “The implementation of a parallel version of Price’s (CRS) algorithm on an ICL DAP”, Technical Report No. 127, Numerical Optimisation Centre, The Hatfield Polytechnic.Google Scholar
  13. I. S. Duff [1983], “The solution of sparse linear systems on the Cray-1”, Technical Report CSS 125 (revised), Computer Science and Systems Divsion, AERE Harwell.Google Scholar
  14. B. Feijoo and R. R. Meyer [1984], “Piecewise-linear approximation methods for non-separable convex optimization”, Technical Report No. 521, Computer Sciences Department, University of Wisconsin — Madison (to appear).Google Scholar
  15. M. J. Flynn [1966], “Very high-speed computing systems”, Proceedings of the IEEE 54, pp. 1901–1909.Google Scholar
  16. K. W. Fong and T. L. Jordan [1977], “Some linear algebraic algorithms and their performance on the Cray-1”, Report LA-6774, Los Alamos National Laboratory.Google Scholar
  17. P. E. Gill, W. Murray, and M. H. Wright [1981], Practical Optimization, Academic Press, London.Google Scholar
  18. A. O. Griewank and Ph. L. Toint [1982], “On the unconstrained optimization of partially separable functions”, in Nonlinear Optimization 1981, M. J. D. Powell ed., Academic Press, London, pp. 301–312.Google Scholar
  19. D. Heller [1978], “A survey of parallel algorithms in numerical linear algebra”, SIAM Review 20, pp. 740-777. pp. 409–436.Google Scholar
  20. R. W. Hockney and C. R. Jesshope [1981], Parallel Computers, Adam-Hilger Ltd.,. Bristol, England.Google Scholar
  21. E. C. Housos and O. Wing [1980], “Parallel nonlinear minimization by conjugate directions”, Proceedings of the 1980 International Conference on Parallel Processing, pp. 157–158.Google Scholar
  22. K. Hwang and F. A. Briggs [1984], Computer Architecture and Parallel Processing, McGraw-Hill, New York.Google Scholar
  23. T. L. Jordan [1979], “A performance evaluation of linear algebra software in parallel architectures”, in Performance Evaluation of Numerical Software, L. D. Fosdick, ed., North-Holland, Amsterdam, pp. 59–76.Google Scholar
  24. R. Kapur and J. Browne [1981], “Block tridiagonal system solution on reconfigurable array computers”, Proceedings of the 1981 International Conference on Parallel Processing, pp. 92–99.Google Scholar
  25. J. Kowalik and S. P. Kumar [1982], “An efficient parallel block conjugate gradient method for linear equations”, Proceedings of the 1982 International Conference on Parallel Processing, pp. 47–52.Google Scholar
  26. H. T. Kung [976], “Synchronized and asynchronous parallel algorithms for multiprocessors”, in Algorithms and Complexity: Recent Results and New Directions, J. E. Traub, ed., Addison-Wesley, pp. 153–200.Google Scholar
  27. B. W. Lampson, M. Paul, and H. J. Siegert [1981], eds., Distributed Systems — Architecture and Implementation, Springer-Verlag, Berlin.Google Scholar
  28. A. V. Levy, A. Montalvo, S. Gomez, and A. Calderon [1981], “Topics in global optimization”, in Proceedings of the Third IIMAS Workshop, Cocoyoc, Mexico, January 1981, J. P. Hennart, ed.Google Scholar
  29. R. Lord, J. Kowalik, and S. Kumar [1980], “Solving linear algebraic equations on a MIMD computer”, Proceedings of the 1980 International Conference on Parallel Processing, pp. 205–210.Google Scholar
  30. J. J. McKeown [1979], “Experiments in implementing a nonlinear least-squares algorithm on a dual-processor computer”, Technical Report No. 102, Numerical Optimisation Centre, The Hatfield Polytechnic.Google Scholar
  31. J. Mohan [1982], “A study in parallel computation — the traveling salesman problem”, Technical Report CMU-CS-82-136, Department of Computer Science, Carnegie-Mellon University.Google Scholar
  32. K. D. Patel [1982b], “Implementation of a parallel (SIMD) modified Newton method on the ICL DAP”, Technical Report No. 131, Numerical Optimisation Centre, The Hatfield Polytechnic.Google Scholar
  33. K. D. Patel [1982a], “Parallel Computation and Numerical Optimisation”, Technical Report No. 129, Numerical Optimisation Centre, The Hatfield Polytechnic.Google Scholar
  34. W. L. Price [1981], “A new version of the controlled random search procedure for global optimisation”, Technical Report, Engineering Department, University of Leicester, England.Google Scholar
  35. A. H. G. Rinnooy Kan and G. T. Timmer [1983], “Stochastic methods for global optimization”, Report 8317/0, Econometric Institute, Erasmus University, Rotterdam.Google Scholar
  36. G. Rodrigue [1982], ed., Parallel Computations, Academic Press, New York.Google Scholar
  37. J. B. Rosen [1983], “Global minimization of a linearly constrained concave function by partition of feasible domain”, Mathematics of Operations Research 8, pp.Google Scholar
  38. A. Samen [1977], “Numerical parallel algorithms — a survey”, in High Speed Computer and Algorithm Organization, D. Kuck, D. Lawrie, and A. Sameh, eds., Academic Press, pp. 207–228.Google Scholar
  39. L. J. Siegel [1983], “Characteristics of Parallel Algorithms”, presented at the Taxonomy of Parallel Algorithms Workshop, Santa Fe, New Mexico, December 1983.Google Scholar
  40. T. A. Straeter [1973], “A parallel variable metric optimization algorithm,” NASA Technical Note D-7329, Langley Reseach Center, Hampton, Virginia.Google Scholar
  41. T. A. Straeter and A. T. Markos [1975], “A parallel Jacobson-Oksman optimization algorithm,” NASA Technical Note D-8020, Langley Reseach Center, Hampton, Virginia.Google Scholar
  42. H. van der Vorst [1982], “A vectorizable variant of some ICCG methods”, SIAM Journal on Scientific and Statistical Computing 3, pp. 350–356.Google Scholar
  43. P. van Laarhoven [1984], “Parallel algorithms for unconstrained optimization,” Mathematical Programming, to appear.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1985

Authors and Affiliations

  • Robert B. Schnabel
    • 1
  1. 1.Department of Computer ScienceUniversity of ColoradoBoulderUSA

Personalised recommendations