# Communication techniques in parallel processing

Part II - Parallel Computer Architectures

First Online:

## Abstract

Large-scale parallel processing is one basis for the design of the supercomputer systems needed for many scientific, industrial, and military applications. The interconnection network in a parallel processing system provides the vehicle for communications among the processors and memories. Eight interconnection techniques for supporting large-scale parallelism (e.g., 2^{6} to 2^{16} processors) are overviewed. These are the Cube, Shuffle-Exchange, PM21 (Plus-Minus 2^{i}), and FNN (Four Nearest Neighbor) single stage networks, and the Generalized Cube, Extra Stage Cube, Augmented Data Manipulator, and Dynamic Redundancy multistage networks.

## Keywords

Interconnection Network Parallel Processing System Primary Path Secondary Path Multistage Interconnection Network
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

## Preview

Unable to display preview. Download preview PDF.

## 8. References

- [AdA87]G. B. Adams III, D. P. Agrawal, and H. J. Siegel “A survey and comparison of fault-tolerant multistage interconnection networks,” Computer, Vol. 20, June 1987, pp. 14–27.Google Scholar
- [AdS82]G. B. Adams III and H. J. Siegel “The extra stage cube: a fault-tolerant interconnection network for super-systems,” IEEE Trans. Computers, Vol. C-31, May 1982, pp. 443–454.Google Scholar
- [AdS84]G. B. Adams III and H. J. Siegel “Modifications to improve the fault tolerance of the extra stage cube interconnection network,” 1984 International Conf. Parallel Processing, Aug. 1984, pp. 169–173.Google Scholar
- [BaB68]G. H. Barnes, R. Brown, M. Kato, D. J. Kuck, D. L. Slotnick, and R. A. Stokes “The Illiac IV computer,” IEEE Trans. Computers, Vol. C-17, Aug. 1968, pp. 746–757.Google Scholar
- [Bae80]J.-L. Baer “Computer Systems Architecture,” Computer Science Press, Potomac, MD, 1980.Google Scholar
- [BaL81]G. H. Barnes and S. F. Lundstrom “Design and validation of a connection network for many-processor multi-processor systems,” Computer, Vol. 14, Dec. 1981, pp. 31–41.Google Scholar
- [Bat76]K. E. Batcher “The flip network in STARAN,” 1976 International Conf. Parallel Processing, Aug. 1976, pp. 65–71.Google Scholar
- [Bat77]K. E. Batcher “STARAN series E,” 1977 International Conf. Parallel Processing, Aug. 1977, pp. 140–143.Google Scholar
- [Bat82]K. E. Batcher “Bit serial parallel processing systems,” IEEE Trans. Computers, Vol. C-31, May 1982, pp. 377–384.Google Scholar
- [BeD85]J. Beetem, M. Denneau, and D. Weingarten “The GF11 supercomputer,” 12th Ann. International Symp. Computer Architecture, June 1985, pp. 108–115.Google Scholar
- [BoD72]W. J. Bouknight, S. A. Denenberg, D. E. McIntryre, J. M. Randall, A. H. Sameh, and D. L. Slotnick “The Illiac IV system,” Proc. IEEE, Vol. 60, Apr. 1972, pp. 369–388.Google Scholar
- [BrH83]G. Broomell and J. R. Heath “Classification categories and historical development of circuit switching topologies,” ACM Computing Surveys, Vol. 15, June 1983, pp. 95–133.Google Scholar
- [CoG74]G. R. Couranz, M. S. Gerhardt, and C. J. Young “Programmable RADAR signal processing using the RAP,” 1974 Sagamore Computer Conf. Parallel Processing, Aug. 1974, pp. 37–52.Google Scholar
- [CrG85]W. Crowther, J. Goodhue, E. Starr, R. Thomas, W. Williken, and T. Blackadar “Performance measurements on a 128-switch Butterfly parallel processor,” 1985 International Conf. Parallel Processing, Aug. 1985, pp. 531–540.Google Scholar
- [DaH85]N. J. Davis IV, W. T.-Y. Hsu, and H. J. Siegel “Fault location techniques for distributed control interconnection networks,” IEEE Trans. Computers, Vol. C-34, Oct. 1985, pp. 902–910.Google Scholar
- [DaS86]N. J. Davis IV and H. J. Siegel “Performance analysis of multiple-packet multistage ‘cube’ networks and comparison to circuit switching,” 1986 International Conf. Parallel Processing, Aug. 1986, pp. 108–114.Google Scholar
- [DeB80]J. B. Dennis, G. A. Boughton, and C. K. C. Leung “Building blocks for data flow prototypes,” 7th Ann. Symp. Computer Architecture, May 1980, pp. 1–8.Google Scholar
- [Duf85]M.J.B. Duff “Real Applications on CLIP4,” in Integrated Technology for Parallel Image Processing, S. Levialdi, ed., Academic Press, Orlando, Florida, 1985, pp. 153–165.Google Scholar
- [Fen74]T. Y. Feng “Data manipulating functions in parallel processors and their implementations,” IEEE Trans. Computers, Vol. C-23, Mar. 1974, pp. 309–318.Google Scholar
- [Fen81]T.Y. Feng “A survey of interconnection networks,” Computer, Vol. 14, Dec. 1981, pp. 12–27.Google Scholar
- [FeW81]T. Y. Feng and C-L. Wu “Fault-diagnosis for a class of multistage interconnection networks,” IEEE Trans. Computers, Vol. C-30, Oct. 1981, pp. 743–758.Google Scholar
- [FeZ85]T. Y. Feng and Q. Zhang “Fault diagnosis of multistage interconnection networks with four valid states,” 5th International Conf. Distributed Computing Systems, May 1985, pp. 218–226.Google Scholar
- [Fil82]A. E. Filip “A distributed signal processing architecture,” 3rd International Conf. Distributed Computing Systems, Oct. 1982, pp. 49–55.Google Scholar
- [Fly66]M. J. Flynn “Very high-speed computing systems,” Proc. IEEE, Vol. 54, Dec. 1966, pp. 1901–1909.Google Scholar
- [GoG83]A. Gottlieb, R. Grishman, C. P. Kruskal, K. P. McAuliffe, L. Rudolph, and M. Snir “The NYU Ultracomputer — designing an MIMD shared-memory parallel computer,” IEEE Trans. Computers, Vol. C-32, Feb. 1983, pp. 175–189.Google Scholar
- [GoL73]G. R. Goke and G. J. Lipovski “Banyan networks for partitioning multiprocessor systems,” 1st Ann. Symp. Computer Architecture, Dec. 1973, pp. 21–28.Google Scholar
- [HaD85]M. T. Hagan, H. B. Demuth, and P. H. Singgih “Parallel signal processing on the HEP,” 1985 International Conf. on Parallel Processing, Aug. 1985, pp. 599–606.Google Scholar
- [Hig72]L. C. Higbie “The Omen computer: associative array processor,” IEEE Computer Society Compcon 72, Sept. 1972, pp. 287–290.Google Scholar
- [Hil85]W. D. Hillis “The Connection Machine,” The M.I.T. Press, Cambridge, MA, 1985.Google Scholar
- [HoJ81]R. W. Hockney and C. R. Jeshope “Parallel Computers,” Adam Hilger Ltd., Bristol, England, 1981.Google Scholar
- [Hun81]D. J. Hunt “The ICL DAP and its application to image processing,” in Languages and Architectures for Image Processing, M. J. B. Duff and S. Levialdi, eds., Academic Press, London, England, 1981, pp. 275–282.Google Scholar
- [HwB84]K. Hwang and F. Briggs “Computer Architecture and Parallel Processing,” McGraw-Hill, New York, NY, 1984.Google Scholar
- [Int85]Intel Corporation “A New Direction in Scientific Computing,” Order # 28009-001, Intel Corporation, 1985.Google Scholar
- [JeS86a]M. Jeng and H. J. Siegel “A fault-tolerant multistage interconnection network for multiprocessor systems using dynamic redundancy,” 6th International Conf. Distributed Computing Systems, June 1986, pp. 70–77.Google Scholar
- [JeS86b]M. Jeng and H. J. Siegel “Implementation approach and reliability estimation of dynamic redundancy networks,” 1986 Real-Time Systems Symp., Dec. 1986, pp. 79–81.Google Scholar
- [JeS87]M. Jeng and H. J. Siegel “Design and analysis of dynamic redundancy networks” IEEE Trans. Computers, to appearGoogle Scholar
- [Kuc78]D. J. Kuck “The Structure of Computers and Computations, Vol. 1,” John Wiley and Sons, New York, NY, 1978.Google Scholar
- [Law75]D. H. Lawrie “Access and alignment of data in an array processor,” IEEE Trans. Computers, Vol. C-24, Dec. 1975, pp. 1145–1155.Google Scholar
- [Lel85]M. D. P. Leland “On the power of the augmented data manipulator network,” 1985 International Conf. Parallel Processing, Aug. 1985, pp. 74–78.Google Scholar
- [MaG79]G. M. Masson, G. C. Gingher, and S. Nakamura “A sampler of circuit switching networks,” Computer, Vol. 12, June 1979, pp. 32–48.Google Scholar
- [MaM81]M. Malek and W. W. Myre “A description method of interconnection networks,” IEEE Technical Committee on Distributed Processing Quarterly, Feb. 1981, pp. 1–6.Google Scholar
- [McS82]R. J. McMillen and H. J. Siegel “Routing schemes for the augmented data manipulator network in an MIMD system,” IEEE Trans. Computers, Vol. C-31, Dec. 1982, pp. 1202–1214.Google Scholar
- [McS85]R. J. McMillen and H. J. Siegel “Evaluation of cube and data manipulator networks,” Journ. Parallel and Distributed Computing, Vol. 2, Feb. 1985, pp. 79–107.Google Scholar
- [Nut77]G. J. Nutt “Microprocessor implementation of a parallel processor,” 4th Ann. Symp. Computer Architecture, Mar. 1977, pp. 147–152.Google Scholar
- [OkT82]Y. Okada, H. Tajima, and R. Mori “A reconfigurable parallel processor with microprogram control,” IEEE Micro, Vol. 2, Nov. 1982, pp. 48–60.Google Scholar
- [PaR82]D. S. Parker and C. S. Raghavendra “The gamma network: a multiprocessor interconnection network with redundant paths,” 9th Ann. Symp. Computer Architecture, Apr. 1982, pp. 73–80.Google Scholar
- [Pat81]J. H. Patel “Performance of processor-memory interconnections for multiprocessors,” IEEE Trans. Computers, Vol. C-30, Oct. 1981, pp. 771–780.Google Scholar
- [Pea77]M. C. Pease III “The indirect binary n-cube microprocessor array,” IEEE Trans. Computers, Vol. C-26, May 1977, pp. 458–473.Google Scholar
- [PfB85]G. F. Pfister, W. C. Brantley, D. A. George, S. L. Harvey, W. J. Kleinfelder, K. P. McAuliffe, E. A. Melton, V. A. Norton, and J. Weiss “The IBM Research Parallel Processor Prototype (RP3): introduction and architecture,” 1985 International Conf. Parallel Processing, Aug. 1985, pp. 764–771.Google Scholar
- [Sei85]C. L. Seitz “The Cosmic Cube,” Comm. ACM, Jan. 1985, pp. 22–33.Google Scholar
- [SeU80]M. C. Sejnowski, E. T. Upchurch, R. N. Kapur, D. P. S. Charlu, and G. J. Lipovski “An overview of the Texas Reconfigurable Array Computer,” AFIPS 1980 Nat'l Computer Conf., June 1980, pp. 631–641.Google Scholar
- [Sie77]H. J. Siegel “Analysis techniques for SIMD machine interconnection networks and the effects of processor address masks,” IEEE Trans. Computers, Vol. C-26, Feb. 1977, pp. 153–161.Google Scholar
- [Sie79]H. J. Siegel “A model of SIMD machines and a comparison of various interconnection networks,” IEEE Trans. Computers, Vol. C-28, Dec. 1979, pp. 907–917.Google Scholar
- [Sie80]H. J. Siegel “The theory underlying the partitioning of permutation networks,” IEEE Trans. Computers, Vol. C-29, Sept. 1980, pp. 791–801.Google Scholar
- [Sie85]H. J. Siegel “Interconnection Networks for Large-Scale Parallel Processing: Theory and Case Studies,” Lexington Books, D.C. Heath and Company, Lexington, MA, 1985.Google Scholar
- [SiM79]H. J. Siegel, R. J. McMillen, and P. T. Mueller, Jr. “A survey of interconnection methods for reconfigurable parallel processing systems,” AFIPS 1979 Nat'l Computer Conf., June 1979, pp. 529–542.Google Scholar
- [SiM81a]H. J. Siegel and R. J. McMillen “Using the augmented data manipulator network in PASM,” Computer, Vol. 14, Feb. 1981, pp. 25–33.Google Scholar
- [SiM81b]H. J. Siegel and R. J. McMillen “The multistage cube: a versatile interconnection network,” Computer, Vol. 14, Dec. 1981, pp. 65–76.Google Scholar
- [SiS78]H. J. Siegel and S. D. Smith “Study of multistage SIMD interconnection networks,” 5th Ann. Symp. Computer Architecture, Apr. 1978, pp. 223–229.Google Scholar
- [SiS81]H. J. Siegel, L. J. Siegel, F. C. Kemmerer, P. T. Mueller, Jr., H. E. Smalley, Jr., and S. D. Smith “PASM: a partitionable SIMD/MIMD system for image processing and pattern recognition,” IEEE Trans. Computers, Vol. C-30, Dec. 1981, pp. 934–947.Google Scholar
- [SiS87]H. J. Siegel, T. Schwederski, J. T. Kuehn, and N. J. Davis IV “An overview of the PASM parallel processing system,” in Computer Architecture, D. D. Gajski, V. M. Milutinovic, H. J. Siegel, and B. P. Furht, eds., IEEE Computer Society Press, Washington, DC, 1987, pp. 387–407.Google Scholar
- [Sto71]H. S. Stone “Parallel processing with the perfect shuffle,” IEEE Trans. Computers, Vol. C-20, Feb. 1971, pp. 153–161.Google Scholar
- [SuB77]H. Sullivan, T. R. Bashkow, and K. Klappholz “A large-scale homogeneous, fully distributed parallel machine,” 4th Ann. Symp. Computer Architecture, Mar. 1977, pp. 105–124.Google Scholar
- [SwF77]R. J. Swan, S. Fuller, and D. P. Siewiorek “Cm⋆: a modular multimicroprocessor,” AFIPS 1977 Nat'l Computer Conf., June 1977, pp. 637–644.Google Scholar
- [ThM79]K. J. Thurber and G. M. Masson “Distributed-Processor Communication Architecture,” Lexington Books, D. C. Heath and Company, Lexington, MA, 1979.Google Scholar
- [ThN81]S. Thanawastien and V. P. Nelson “Interference analysis of shuffle/exchange networks,” IEEE Trans. Computers, Vol. C-30, August 1981, pp. 545–556.Google Scholar
- [Thu79]K. J. Thurber “Parallel processor architectures-part 1: general purpose systems,” Computer Design, Vol. 18, Jan. 1979, pp. 89–97.Google Scholar
- [Wes72]A. H. Wester “Special features in SIMDA,” 1972 Sagamore Computer Conf. Parallel Processing, Aug. 1972, pp. 29–40.Google Scholar
- [WuF80]C-L. Wu and T. Y. Feng “On a class of multistage interconnection networks,” IEEE Trans. Computers, Vol. C-29, Aug. 1980, pp. 694–702.Google Scholar
- [WuF84]C.-L. Wu and T.Y. Feng, eds. “Tutorial: Interconnection Networks for Parallel and Distributed Processing,” IEEE Computer Society Press, Silver Spring, MD, 1984.Google Scholar

## Copyright information

© Springer-Verlag Berlin Heidelberg 1988