Introduction to Parallel Computers: Architecture and Algorithms

  • D. R. Emerson
Part of the ERCOFTAC Series book series (ERCO, volume 3)


The term “parallel processing” refers to performing computational tasks concurrently i e at the same time. Each individual task can be either identical, such as adding two elements of an array, or unique. It should be made clear at this stage that parallel processing is not a new concept. Indeed, it has been around in practice for several decades and its concept has been understood for over 150 years (Menabrea, 1842). However, it is only since 1985 that commercially available machines have been in use outside of purely academic or research establishments. Furthermore, it is now widely recognised that only parallel processing offers the potential of solving very large scale scientific problems, such as those encountered in Computational Fluid Dynamics (CFD), Computational ElectroMagnetics (CEM), Quantum Chemistry, Quantum ChromoDynamics (QCD), and many other computationally demanding fields, in an “acceptable” time and at a “reasonable” cost. Of course, what is acceptable or reasonable will very much depend upon each individual’s requirements.


Clock Cycle Peak Performance Cache Line Floating Point Operation Loop Unroll 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Amdahl, G., “Validity of the Single Processor Approach to Achieving Large Scale Computing Capabilities”, In AFIPS Conference Proceedings, 1967, pp 483-485.Google Scholar
  2. Bailey, D., “Twelve Ways to Fool the Masses When Giving Performance Results on Parallel Computers”, Supercomputing Review, August 1991, pp 54-55.Google Scholar
  3. Bailey, D., “Misleading Performance in the Supercomputing Field”, Supercomputing 92 Proceedings, Published by IEEE, held at Minneapolis, November 16-20, 1992, pp 155-158.Google Scholar
  4. Bailey, D., “RISC Microprocessors and Scientific Computing”, RNR Technical Report RNR-93-004, 1993.Google Scholar
  5. Chandi, K.M. and Taylor, S., An Introduction to Parallel Programming, Jones and Bartlett Publishers, Boston, 1992, ISBN 0-86720-208-4.Google Scholar
  6. Dongarra, J. J., Duff, I. S., Sorensen, D. C. and Van der Vorst, H. A., Linear System Solving on Vector and Shared Memory Computers, SIAM, 1991, ISBN 0-89871-270-X.Google Scholar
  7. Dongarra, J. J., Meuer, H. W. and Strohmaier, E., The TOP500 Report, WWW Scholar
  8. Edmondson, J. H., Rubinfeld, P., Preston, R. and Rajagopalan, V., “Superscalar Instruction Execution in the 21164 Alpha Microprocessor”, IEEE Micro, April 1995, pp 33 - 43.Google Scholar
  9. Emerson, D. R., Blake, R. J. and Allan, R. J.,“Unsteady Flow Over Multiple Cavities on MIMD and Virtual Shared Memory Architectures”, in CFD Algorithms and Applications for Parallel Processors, ASME Fluids Engineering Conference FED-Vol 156, June 20-24, 1993, Washington, pp 33-41.Google Scholar
  10. Flynn, M.J., “Some Computer Organisations and Their Effectiveness”, IEEE Transactions on Computers, Vol. C-21, No. 9, Sept. 1972, pp 948–960.MathSciNetCrossRefGoogle Scholar
  11. Gajski, D. D. and Pier, J-K, “A Comparison of Five Multiprocessor Systems”, Parallel Computing, Vol 2, 1985, pp 265–282.CrossRefGoogle Scholar
  12. Gustafson, J.L., “Reevaluating Amdahl’s Law”, Communications of the ACM, Vol 31, No. 5, 1988, pp 532–533.CrossRefGoogle Scholar
  13. Heath, M. and Worley, P., “Once Again, Amdahl’s Law”, Communications of the ACM, Vol 32, No. 2, February 1989, pp 262–264.Google Scholar
  14. Hill, M. D., “What is Scalabilty?”, Computer Architecture News, December, 1990, pp 18-21.Google Scholar
  15. Hockney, R. W. and Jesshope, C. R., Parallel Computers 2, IOP Publishing, 1988, ISBN 0-85274-812-4.Google Scholar
  16. Hoffmann, K. A., Computational Fluid Dynamics for Engineers, Engineering Education System, 1989, ISBN 0-9623731-4-1.Google Scholar
  17. Hord, R. M., The Uliac IV - The First Supercomputer, Springer-Verlag, 1982, ISBN 0-387-11765-2.Google Scholar
  18. Hwang, K. and Briggs, F. A., Computer Architecture and Parallel Processing, McGraw-Hill, 1985, ISBN 0-07-031556-6.Google Scholar
  19. Koelbel, C. H., Loveman, D. B., Schreiber, R. S., Steele, G. L. and Zosel, M. E., The High Performance Fortran Handbook, MIT Press, 1994, ISBN 0-262-11185-3.Google Scholar
  20. Kuck, D. J., The Structure of Computers and Computations, John Wiley and Sons, 1978, ISBN 0-471-02716-2.Google Scholar
  21. Lawrie, D. H., “Access and Alignment of Data in an Array Processor”, IEEE Trans, on Computers, Vol C-24, No. 12, December 1975, pp 1145–1155.MathSciNetCrossRefGoogle Scholar
  22. Leiserson, C. E., “Fat-Trees: Universal Networks for Hardware-Efficient Supercomputing”, IEEE Trans, on Computers, Vol C-34, No. 10, October 1985, pp 892–901.Google Scholar
  23. Menabrea, L. F., “Sketch of the Analytical Engine Invented by Charles Babbage”, ESP. Biblioth&que Universale de Genfcve, 82, 1842.Google Scholar
  24. Mirapuri, S., Woodacre, M. and Vasseghi, N., “The MIPS R4000 Processor”, IEEE Micro, April 1992, pp 10 - 22.Google Scholar
  25. Nussbaum, D. and Agarwal, A., “Scalabilty of Parallel Machines”, Communications of the ACM, Vol 34, No. 3, March 1991, pp 57–61.Google Scholar
  26. Oed, W.,“The Cray Research Massively Parallel Processor System: CRAY T3D”, Cray Research, November, 1993.Google Scholar
  27. Pease, M. C., “The Indirect Binary n-Cube Microprocessor Array”, IEEE Trans, on Computers, Vol C-26, No. 5, May 1977, pp 458–473.CrossRefGoogle Scholar
  28. Randell, B., The Origins of Digital Computers, Springer-Verlag, 1982, ISBN 3-540-11319-3.Google Scholar
  29. Singh, J. P., Hennessy, J. L. and Gupta, A., “Scaling Parallel Programs for Multiprocessors: Methodology and Examples”, IEEE Computer, July, 1993, pp42-50.Google Scholar
  30. Trew, A. and Wilson, G. (Eds.), Past, Present, Parallel - A Survey of Available Parallel Computing Systems, Springer-Verlag, 1991, ISBN 0-387-19664-1.Google Scholar
  31. Ware, W., “The Ultimate Computer”, IEEE Spectrum, March 1973, pp 89-91.Google Scholar

Copyright information

© Kluwer Academic Publishers 1996

Authors and Affiliations

  • D. R. Emerson
    • 1
  1. 1.Computational Engineering GroupDaresbury LaboratoryCheshireUK

Personalised recommendations