Abstract
The optimization of the execution time of a parallel algorithm can be achieved through the use of an analytical cost model function representing the running time. Typically the cost function includes a set of parameters that model the behavior of the system and the algorithm. In order to reach an optimal execution, some of these parameters must be fitted according to the input problem and to the target architecture. An optimization problem can be stated where the modeled execution time for the algorithm is used to estimate the parameters. Due to the large number of variable parameters in the model, analytical minimization techniques are discarded. Exhaustive search techniques can be used to solve the optimization problem, but when the number of parameters or the size of the computational system increases, the method is impracticable due to time restrictions. The use of approximation methods to guide the search is also an alternative. However, the dependence on the algorithm modeled and the bad quality of the solutions as a result of the presence of many local optima values in the objective functions are also drawbacks to these techniques. The problem becomes particularly difficult in complex systems hosting a large number of heterogeneous processors solving non-trivial scientific applications. The use of metaheuristics allows for the development of valid approaches to solve general problems with a large number of parameters. A well-known advantage of metaheuristic methods is the ability to obtain high-quality solutions at low running times while maintaining generality. We propose combining the parameterized analytical cost model function and metaheuristic minimization methods, which contributes to a novel real alternative to minimize the parallel execution time in complex systems. The success of the proposed approach is shown with two different algorithmic schemes on parallel heterogeneous systems. Furthermore, the development of a general framework allows us to easily develop and experiment with different metaheuristics to adjust them to particular problems.
Similar content being viewed by others
References
Almeida F, Andonov R, González D, Moreno L, Poirriez V, Rodríguez C (2002) Optimal tiling for the RNA base pairing problem. In: Proc 14th ACM symposium on parallel algorithms and architectures (SPAA), August 2002, Winnipeg, Manitoba, Canada
Almeida F, González D, Moreno LM (2006) The master–slave paradigm on heterogeneous systems: A dynamic programming approach for the optimal mapping. J Syst Archit 52(2):105–116
Andonov R, Rajopadhye S (1997) Optimal orthogonal tiling of 2-D iterations. J Parallel Distrib Comput (45):159–165
Andonov R, Balev S, Rajopadhye S, Yanev N (2001) Optimal semi-oblique tiling and its application to sequence comparison. In: Proc 13th ACM symposium on parallel algorithms and architectures (SPAA), July 2001, Crete Islands, Greece
Bajaj R, Agrawal DP (2004) Improving scheduling of tasks in a heterogeneous environment. IEEE Trans Parallel Distrib Syst 15(2):107–118
Banino C, Beaumont O, Carter L, Ferrante J, Legrand A, Robert Y (2004) Scheduling strategies for master–slave tasking on heterogeneous processor platforms. IEEE Trans Parallel Distrib Syst 15(4):319–330
Banino C, Beaumont O, Legrand A, Robert Y (2002) Scheduling strategies for master–slave tasking on heterogeneous processor grids. Technical Report 2002-12, INRIA
Beaumont O, Boudet V, Petitet A, Rastello F, Robert Y (2001) A proposal for heterogeneous cluster scalapack (dense linear solver). IEEE Trans Comput 50(10):1052–1070
Blackford LS, Choi J, Cleary A, D’Azevedo E, Demmel J, Dhillon I, Dongarra JJ, Hammarling S, Henry G, Petitet A, Stanley K, Walker D, Whaley RC (1997) ScaLAPACK user’s guide. Society for Industrial and Applied Mathematics
Cuenca J, García LP, Giménez D, Dongarra J (2005) Processes distribution of homogeneous parallel linear algebra routines on heterogeneous clusters. In: Proc IEEE international conference on cluster computing. IEEE Computer Society, September 2005, Boston, Massachusetts, USA
Cuenca J, Giménez D, González J (2004) Architecture of an automatic tuned linear algebra library. Parallel Comput 30(2):187–220
Cuenca J, Giménez D, González J, Dongarra J, Roche K (2003) Automatic optimisation of parallel linear algebra routines in systems with variable load. In: Proc of parallel and distributed processing. IEEE Computer Society, February 2003, Genova, Italy
Cuenca J, Giménez D, Martínez-Gallar JP (2005) Heuristics for work distribution of a homogeneous parallel dynamic programming scheme on heterogeneous systems. Parallel Comput 31:717–735
Dréo J, Pétrowski A, Siarry P, Taillard E (2005) Metaheuristics for hard optimization. Springer, Berlin
Epstein L, Sgall J (2004) Approximation schemes for scheduling on uniformly related and identical parallel machines. Algorithmica 39:43–57
Fujita S, Masukawa M, Tagashira S (2003) A fast branch-and-bound scheme for the multiprocessor scheduling problem with communication time. In: Proc IEEE international conference on parallel processing workshop. IEEE Computer Society, October 2003, Kaohsiung, Taiwan
Gamma E, Helm R, Johnson R, Vlissides J (2002) Design patterns: elements of reusable object-oriented software. Addison-Wesley, Reading
Giménez D, Calvo A-L, Cortés A, Pozuelo C (2008) Using metaheuristics in a parallel computing course. In: Proc international conference on computational science, LNCS, June 2008, Krakóv, Poland
Giménez D, Martínez JP (2004) Automatic optimization in parallel dynamic programming schemes. In: Proc of the 6th international meeting VECPAR04, June 2004, Valencia, Spain
Glover F (1986) Future paths for integer programming and links to artificial intelligence. Comput & Ops Res 13(5):533–549
Glover F, Kochenberger GA (2003) Handbook of metaheuristics. Kluwer, Dordrecht
González D, Almeida F, Moreno L, Rodríguez C (2003) Toward the automatic optimization mapping of pipeline algorithms. Parallel Comput 29(2):241–254
Grama A, Gupta A, Karypis G, Kumar V (2003) Introduction to parallel computing, 2nd edn. Addison-Wesley, Reading
Hromkovič J (2003) Algorithmics for hard problems, 2nd edn. Springer, Berlin
Jansen K (2004) Scheduling malleable parallel tasks: an asymptotic fully polynomial time approximation scheme. Algorithmica 39:59–81
Kalinov A, Lastovetsky A (2001) Heterogeneous distribution of computations while solving linear algebra problems on network of heterogeneous computers. J Parallel Distrib Comput 61(44):520–535
Martínez JP, Almeida F, Giménez D (2006) Mapping in heterogeneous systems with heuristic methods. In: Proc applied parallel computing, state of the art in scientific computing, LNCS, June 2006, Umea, Sweden
Raidl GR (2006) A unified view on hybrid metaheuristics. In: Proc hybrid metaheuristics, third international workshop, LNCS, October 2006, Gran Canaria, Spain
Sabin G, Kettimuthu R, Rajan A, Sadayappan P (2003) Scheduling of parallel jobs in a heterogeneous multi-site environment. In: Job scheduling strategies for parallel processing, 9th international workshop, LNCS, June 2003, Seattle, WA, USA
Seymour K, You H, Dongarra J (2008) A comparison of search heuristics for empirical code optimization. In: The third international workshop on automatic performance tuning, October 2008, Tokyo, Japan
Vaessens R, Aarts E, Lenstra J (1992) A local search template. In: Parallel problem solving from nature. Elsevier, Amsterdam, pp 67–76
Vidyarthi DP, Tripathi AK, Sarker BK, Dhawan A, Yang LT (2004) Cluster-based multiple task allocation in distributed computing system. In: Proc 18th international parallel and distributed processing symposium, April 2004, Santa Fe, New Mexico
Voss S, Martello S, Osman IH, Roucairol C (1999) Meta-heuristics: advances and trends in local search paradigms for optimization. Kluwer, Dordrecht
Wu X (1999) Performance evaluation, prediction and visualization of parallel systems. Kluwer, Dordrecht
Workshop on scheduling for large-scale systems, May 2009, Knoxville, Tennessee
Zhao W, Ramamritham K (1987) Simple and integrated heuristic algorithms for scheduling tasks with time and resource constrains. J Syst Softw 7:195–205
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Almeida, F., Cuenca, J., Giménez, D. et al. A framework for the application of metaheuristics to tasks-to-processors assignation problems. J Supercomput 62, 698–723 (2012). https://doi.org/10.1007/s11227-009-0309-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11227-009-0309-2