Static Program Partitioning
In this chapter we attempt to define more precisely the problems of program partitioning and interprocessor communication, to model them, identify the variables involved and quantify them. We mentioned earlier that program partitioning, interprocessor communication, parallelism, and data dependences are all closely related. In shared memory parallel machines, interprocessor communication is not explicit but it happens through sharing of specific memory locations. Processors read from and write to shared memory under the control of synchronization which enforces a predetermined order. However, in message passing systems (e.g., hypercube architectures), communication between processors takes place through the use of explicit send/receive of packets of data. Since there is no shared memory in such systems, data is requested by a given processor from another specific processor in the distributed system. The latter processor must then assemble a packet with the requested data and forward it to the requesting processor. The message may be routed through other intermediate processors before it reaches its destination. Thus communication overhead is far more critical on distributed systems than it is on shared memory systems. As discussed in Chapter 3, a consequence of this is that parallel tasks in distributed systems are of coarse granularity.
KeywordsExecution Time Shared Memory Communication Overhead Parallel Execution Task Graph
Unable to display preview. Download preview PDF.