Improvement of Load Balancing in Shared-Memory Multiprocessor Systems

Abstract

Parallel programming is one of the most effective approaches to handle complex problems regarding time complexity by reducing computation time, by getting the most of the capacity of the processors and shared-memory or distributed systems. One of the main ingredients of parallel programming is ‘Loops’ and especially DOALL loops. All loop scheduling algorithms try to achieve load balancing by using chunk resizing techniques (decreasing, increasing…). In this paper, prior loop scheduling algorithms will be evaluated on Mandelbrot. A new algorithm is obtained by merging the increasing and decreasing chunk size techniques in order to acquire the advantages of both approaches. The experimental results show that in the decreasing approach, a large number of small chunks in the last stages will increase the scheduling overhead because of the increasing of inter-processor communication. Also, the large chunk size of the initial stages will increase load imbalance. On the other hand, for the increasing approach, the large chunks assigned to processors at the last stages might increase load imbalance especially when iterations of the last stages are more time consuming than others. This work will introduce a new approach. This approach will try to minimize the load imbalance and communication overhead that are caused by using a decreasing chunk size approach and to minimize the load imbalance and scheduling overhead caused by the use of increase chunk size approach, which is going to provide better performance for many workload patterns.