Advertisement

Communication techniques in parallel processing

  • Howard Jay Siegel
  • William Tsun-yuk Hsu
  • Menkae Jeng
  • Wayne G. Nation
Part II - Parallel Computer Architectures
Part of the Lecture Notes in Computer Science book series (LNCS, volume 295)

Abstract

Large-scale parallel processing is one basis for the design of the supercomputer systems needed for many scientific, industrial, and military applications. The interconnection network in a parallel processing system provides the vehicle for communications among the processors and memories. Eight interconnection techniques for supporting large-scale parallelism (e.g., 26 to 216 processors) are overviewed. These are the Cube, Shuffle-Exchange, PM21 (Plus-Minus 2i), and FNN (Four Nearest Neighbor) single stage networks, and the Generalized Cube, Extra Stage Cube, Augmented Data Manipulator, and Dynamic Redundancy multistage networks.

Keywords

Interconnection Network Parallel Processing System Primary Path Secondary Path Multistage Interconnection Network 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

8. References

  1. [AdA87]
    G. B. Adams III, D. P. Agrawal, and H. J. Siegel “A survey and comparison of fault-tolerant multistage interconnection networks,” Computer, Vol. 20, June 1987, pp. 14–27.Google Scholar
  2. [AdS82]
    G. B. Adams III and H. J. Siegel “The extra stage cube: a fault-tolerant interconnection network for super-systems,” IEEE Trans. Computers, Vol. C-31, May 1982, pp. 443–454.Google Scholar
  3. [AdS84]
    G. B. Adams III and H. J. Siegel “Modifications to improve the fault tolerance of the extra stage cube interconnection network,” 1984 International Conf. Parallel Processing, Aug. 1984, pp. 169–173.Google Scholar
  4. [BaB68]
    G. H. Barnes, R. Brown, M. Kato, D. J. Kuck, D. L. Slotnick, and R. A. Stokes “The Illiac IV computer,” IEEE Trans. Computers, Vol. C-17, Aug. 1968, pp. 746–757.Google Scholar
  5. [Bae80]
    J.-L. Baer “Computer Systems Architecture,” Computer Science Press, Potomac, MD, 1980.Google Scholar
  6. [BaL81]
    G. H. Barnes and S. F. Lundstrom “Design and validation of a connection network for many-processor multi-processor systems,” Computer, Vol. 14, Dec. 1981, pp. 31–41.Google Scholar
  7. [Bat76]
    K. E. Batcher “The flip network in STARAN,” 1976 International Conf. Parallel Processing, Aug. 1976, pp. 65–71.Google Scholar
  8. [Bat77]
    K. E. Batcher “STARAN series E,” 1977 International Conf. Parallel Processing, Aug. 1977, pp. 140–143.Google Scholar
  9. [Bat82]
    K. E. Batcher “Bit serial parallel processing systems,” IEEE Trans. Computers, Vol. C-31, May 1982, pp. 377–384.Google Scholar
  10. [BeD85]
    J. Beetem, M. Denneau, and D. Weingarten “The GF11 supercomputer,” 12th Ann. International Symp. Computer Architecture, June 1985, pp. 108–115.Google Scholar
  11. [BoD72]
    W. J. Bouknight, S. A. Denenberg, D. E. McIntryre, J. M. Randall, A. H. Sameh, and D. L. Slotnick “The Illiac IV system,” Proc. IEEE, Vol. 60, Apr. 1972, pp. 369–388.Google Scholar
  12. [BrH83]
    G. Broomell and J. R. Heath “Classification categories and historical development of circuit switching topologies,” ACM Computing Surveys, Vol. 15, June 1983, pp. 95–133.Google Scholar
  13. [CoG74]
    G. R. Couranz, M. S. Gerhardt, and C. J. Young “Programmable RADAR signal processing using the RAP,” 1974 Sagamore Computer Conf. Parallel Processing, Aug. 1974, pp. 37–52.Google Scholar
  14. [CrG85]
    W. Crowther, J. Goodhue, E. Starr, R. Thomas, W. Williken, and T. Blackadar “Performance measurements on a 128-switch Butterfly parallel processor,” 1985 International Conf. Parallel Processing, Aug. 1985, pp. 531–540.Google Scholar
  15. [DaH85]
    N. J. Davis IV, W. T.-Y. Hsu, and H. J. Siegel “Fault location techniques for distributed control interconnection networks,” IEEE Trans. Computers, Vol. C-34, Oct. 1985, pp. 902–910.Google Scholar
  16. [DaS86]
    N. J. Davis IV and H. J. Siegel “Performance analysis of multiple-packet multistage ‘cube’ networks and comparison to circuit switching,” 1986 International Conf. Parallel Processing, Aug. 1986, pp. 108–114.Google Scholar
  17. [DeB80]
    J. B. Dennis, G. A. Boughton, and C. K. C. Leung “Building blocks for data flow prototypes,” 7th Ann. Symp. Computer Architecture, May 1980, pp. 1–8.Google Scholar
  18. [Duf85]
    M.J.B. Duff “Real Applications on CLIP4,” in Integrated Technology for Parallel Image Processing, S. Levialdi, ed., Academic Press, Orlando, Florida, 1985, pp. 153–165.Google Scholar
  19. [Fen74]
    T. Y. Feng “Data manipulating functions in parallel processors and their implementations,” IEEE Trans. Computers, Vol. C-23, Mar. 1974, pp. 309–318.Google Scholar
  20. [Fen81]
    T.Y. Feng “A survey of interconnection networks,” Computer, Vol. 14, Dec. 1981, pp. 12–27.Google Scholar
  21. [FeW81]
    T. Y. Feng and C-L. Wu “Fault-diagnosis for a class of multistage interconnection networks,” IEEE Trans. Computers, Vol. C-30, Oct. 1981, pp. 743–758.Google Scholar
  22. [FeZ85]
    T. Y. Feng and Q. Zhang “Fault diagnosis of multistage interconnection networks with four valid states,” 5th International Conf. Distributed Computing Systems, May 1985, pp. 218–226.Google Scholar
  23. [Fil82]
    A. E. Filip “A distributed signal processing architecture,” 3rd International Conf. Distributed Computing Systems, Oct. 1982, pp. 49–55.Google Scholar
  24. [Fly66]
    M. J. Flynn “Very high-speed computing systems,” Proc. IEEE, Vol. 54, Dec. 1966, pp. 1901–1909.Google Scholar
  25. [GoG83]
    A. Gottlieb, R. Grishman, C. P. Kruskal, K. P. McAuliffe, L. Rudolph, and M. Snir “The NYU Ultracomputer — designing an MIMD shared-memory parallel computer,” IEEE Trans. Computers, Vol. C-32, Feb. 1983, pp. 175–189.Google Scholar
  26. [GoL73]
    G. R. Goke and G. J. Lipovski “Banyan networks for partitioning multiprocessor systems,” 1st Ann. Symp. Computer Architecture, Dec. 1973, pp. 21–28.Google Scholar
  27. [HaD85]
    M. T. Hagan, H. B. Demuth, and P. H. Singgih “Parallel signal processing on the HEP,” 1985 International Conf. on Parallel Processing, Aug. 1985, pp. 599–606.Google Scholar
  28. [Hig72]
    L. C. Higbie “The Omen computer: associative array processor,” IEEE Computer Society Compcon 72, Sept. 1972, pp. 287–290.Google Scholar
  29. [Hil85]
    W. D. Hillis “The Connection Machine,” The M.I.T. Press, Cambridge, MA, 1985.Google Scholar
  30. [HoJ81]
    R. W. Hockney and C. R. Jeshope “Parallel Computers,” Adam Hilger Ltd., Bristol, England, 1981.Google Scholar
  31. [Hun81]
    D. J. Hunt “The ICL DAP and its application to image processing,” in Languages and Architectures for Image Processing, M. J. B. Duff and S. Levialdi, eds., Academic Press, London, England, 1981, pp. 275–282.Google Scholar
  32. [HwB84]
    K. Hwang and F. Briggs “Computer Architecture and Parallel Processing,” McGraw-Hill, New York, NY, 1984.Google Scholar
  33. [Int85]
    Intel Corporation “A New Direction in Scientific Computing,” Order # 28009-001, Intel Corporation, 1985.Google Scholar
  34. [JeS86a]
    M. Jeng and H. J. Siegel “A fault-tolerant multistage interconnection network for multiprocessor systems using dynamic redundancy,” 6th International Conf. Distributed Computing Systems, June 1986, pp. 70–77.Google Scholar
  35. [JeS86b]
    M. Jeng and H. J. Siegel “Implementation approach and reliability estimation of dynamic redundancy networks,” 1986 Real-Time Systems Symp., Dec. 1986, pp. 79–81.Google Scholar
  36. [JeS87]
    M. Jeng and H. J. Siegel “Design and analysis of dynamic redundancy networks” IEEE Trans. Computers, to appearGoogle Scholar
  37. [Kuc78]
    D. J. Kuck “The Structure of Computers and Computations, Vol. 1,” John Wiley and Sons, New York, NY, 1978.Google Scholar
  38. [Law75]
    D. H. Lawrie “Access and alignment of data in an array processor,” IEEE Trans. Computers, Vol. C-24, Dec. 1975, pp. 1145–1155.Google Scholar
  39. [Lel85]
    M. D. P. Leland “On the power of the augmented data manipulator network,” 1985 International Conf. Parallel Processing, Aug. 1985, pp. 74–78.Google Scholar
  40. [MaG79]
    G. M. Masson, G. C. Gingher, and S. Nakamura “A sampler of circuit switching networks,” Computer, Vol. 12, June 1979, pp. 32–48.Google Scholar
  41. [MaM81]
    M. Malek and W. W. Myre “A description method of interconnection networks,” IEEE Technical Committee on Distributed Processing Quarterly, Feb. 1981, pp. 1–6.Google Scholar
  42. [McS82]
    R. J. McMillen and H. J. Siegel “Routing schemes for the augmented data manipulator network in an MIMD system,” IEEE Trans. Computers, Vol. C-31, Dec. 1982, pp. 1202–1214.Google Scholar
  43. [McS85]
    R. J. McMillen and H. J. Siegel “Evaluation of cube and data manipulator networks,” Journ. Parallel and Distributed Computing, Vol. 2, Feb. 1985, pp. 79–107.Google Scholar
  44. [Nut77]
    G. J. Nutt “Microprocessor implementation of a parallel processor,” 4th Ann. Symp. Computer Architecture, Mar. 1977, pp. 147–152.Google Scholar
  45. [OkT82]
    Y. Okada, H. Tajima, and R. Mori “A reconfigurable parallel processor with microprogram control,” IEEE Micro, Vol. 2, Nov. 1982, pp. 48–60.Google Scholar
  46. [PaR82]
    D. S. Parker and C. S. Raghavendra “The gamma network: a multiprocessor interconnection network with redundant paths,” 9th Ann. Symp. Computer Architecture, Apr. 1982, pp. 73–80.Google Scholar
  47. [Pat81]
    J. H. Patel “Performance of processor-memory interconnections for multiprocessors,” IEEE Trans. Computers, Vol. C-30, Oct. 1981, pp. 771–780.Google Scholar
  48. [Pea77]
    M. C. Pease III “The indirect binary n-cube microprocessor array,” IEEE Trans. Computers, Vol. C-26, May 1977, pp. 458–473.Google Scholar
  49. [PfB85]
    G. F. Pfister, W. C. Brantley, D. A. George, S. L. Harvey, W. J. Kleinfelder, K. P. McAuliffe, E. A. Melton, V. A. Norton, and J. Weiss “The IBM Research Parallel Processor Prototype (RP3): introduction and architecture,” 1985 International Conf. Parallel Processing, Aug. 1985, pp. 764–771.Google Scholar
  50. [Sei85]
    C. L. Seitz “The Cosmic Cube,” Comm. ACM, Jan. 1985, pp. 22–33.Google Scholar
  51. [SeU80]
    M. C. Sejnowski, E. T. Upchurch, R. N. Kapur, D. P. S. Charlu, and G. J. Lipovski “An overview of the Texas Reconfigurable Array Computer,” AFIPS 1980 Nat'l Computer Conf., June 1980, pp. 631–641.Google Scholar
  52. [Sie77]
    H. J. Siegel “Analysis techniques for SIMD machine interconnection networks and the effects of processor address masks,” IEEE Trans. Computers, Vol. C-26, Feb. 1977, pp. 153–161.Google Scholar
  53. [Sie79]
    H. J. Siegel “A model of SIMD machines and a comparison of various interconnection networks,” IEEE Trans. Computers, Vol. C-28, Dec. 1979, pp. 907–917.Google Scholar
  54. [Sie80]
    H. J. Siegel “The theory underlying the partitioning of permutation networks,” IEEE Trans. Computers, Vol. C-29, Sept. 1980, pp. 791–801.Google Scholar
  55. [Sie85]
    H. J. Siegel “Interconnection Networks for Large-Scale Parallel Processing: Theory and Case Studies,” Lexington Books, D.C. Heath and Company, Lexington, MA, 1985.Google Scholar
  56. [SiM79]
    H. J. Siegel, R. J. McMillen, and P. T. Mueller, Jr. “A survey of interconnection methods for reconfigurable parallel processing systems,” AFIPS 1979 Nat'l Computer Conf., June 1979, pp. 529–542.Google Scholar
  57. [SiM81a]
    H. J. Siegel and R. J. McMillen “Using the augmented data manipulator network in PASM,” Computer, Vol. 14, Feb. 1981, pp. 25–33.Google Scholar
  58. [SiM81b]
    H. J. Siegel and R. J. McMillen “The multistage cube: a versatile interconnection network,” Computer, Vol. 14, Dec. 1981, pp. 65–76.Google Scholar
  59. [SiS78]
    H. J. Siegel and S. D. Smith “Study of multistage SIMD interconnection networks,” 5th Ann. Symp. Computer Architecture, Apr. 1978, pp. 223–229.Google Scholar
  60. [SiS81]
    H. J. Siegel, L. J. Siegel, F. C. Kemmerer, P. T. Mueller, Jr., H. E. Smalley, Jr., and S. D. Smith “PASM: a partitionable SIMD/MIMD system for image processing and pattern recognition,” IEEE Trans. Computers, Vol. C-30, Dec. 1981, pp. 934–947.Google Scholar
  61. [SiS87]
    H. J. Siegel, T. Schwederski, J. T. Kuehn, and N. J. Davis IV “An overview of the PASM parallel processing system,” in Computer Architecture, D. D. Gajski, V. M. Milutinovic, H. J. Siegel, and B. P. Furht, eds., IEEE Computer Society Press, Washington, DC, 1987, pp. 387–407.Google Scholar
  62. [Sto71]
    H. S. Stone “Parallel processing with the perfect shuffle,” IEEE Trans. Computers, Vol. C-20, Feb. 1971, pp. 153–161.Google Scholar
  63. [SuB77]
    H. Sullivan, T. R. Bashkow, and K. Klappholz “A large-scale homogeneous, fully distributed parallel machine,” 4th Ann. Symp. Computer Architecture, Mar. 1977, pp. 105–124.Google Scholar
  64. [SwF77]
    R. J. Swan, S. Fuller, and D. P. Siewiorek “Cm⋆: a modular multimicroprocessor,” AFIPS 1977 Nat'l Computer Conf., June 1977, pp. 637–644.Google Scholar
  65. [ThM79]
    K. J. Thurber and G. M. Masson “Distributed-Processor Communication Architecture,” Lexington Books, D. C. Heath and Company, Lexington, MA, 1979.Google Scholar
  66. [ThN81]
    S. Thanawastien and V. P. Nelson “Interference analysis of shuffle/exchange networks,” IEEE Trans. Computers, Vol. C-30, August 1981, pp. 545–556.Google Scholar
  67. [Thu79]
    K. J. Thurber “Parallel processor architectures-part 1: general purpose systems,” Computer Design, Vol. 18, Jan. 1979, pp. 89–97.Google Scholar
  68. [Wes72]
    A. H. Wester “Special features in SIMDA,” 1972 Sagamore Computer Conf. Parallel Processing, Aug. 1972, pp. 29–40.Google Scholar
  69. [WuF80]
    C-L. Wu and T. Y. Feng “On a class of multistage interconnection networks,” IEEE Trans. Computers, Vol. C-29, Aug. 1980, pp. 694–702.Google Scholar
  70. [WuF84]
    C.-L. Wu and T.Y. Feng, eds. “Tutorial: Interconnection Networks for Parallel and Distributed Processing,” IEEE Computer Society Press, Silver Spring, MD, 1984.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1988

Authors and Affiliations

  • Howard Jay Siegel
    • 1
  • William Tsun-yuk Hsu
    • 2
  • Menkae Jeng
    • 3
  • Wayne G. Nation
    • 3
  1. 1.Supercomputing Research CenterLanhamUSA
  2. 2.Center for Supercomputing Research & DevelopmentUniversity of IllinoisUrbanaUSA
  3. 3.PASM Parallel Processing Laboratory School of Electrical EngineeringPurdue UniversityWest LafayetteUSA

Personalised recommendations