Lobachevskii Journal of Mathematics

, Volume 39, Issue 4, pp 571–575 | Cite as

Analytical Estimation of the Scalability of Iterative Numerical Algorithms on Distributed Memory Multiprocessors

  • L. B. Sokolinsky


This article presents a new high-level parallel computational model named BSF "— Bulk Synchronous Farm. The BSF model extends the BSP model to deal with the computeintensive iterative numericalmethods executed on distributed-memory multiprocessor systems. The BSF model is based on the master-worker paradigm and the SPMD programming model. The BSF model makes it possible to predict the upper scalability bound of a BSF-program with great accuracy. The BSF model also provides equations for estimating the speedup and parallel efficiency of a BSF-program.

Keywords and phrases

Parallel computation model bulk synchronous farm BSF model iterative algorithms distributed memory scalability bound 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    G. Bilardi and A. Pietracaprina, “Models of computation, Theoretical,” in Encyclopedia of Parallel Computing (Springer US, Boston, MA, 2011), pp. 1150–1158. doi 10.1007/978-0-387-09766-4_218Google Scholar
  2. 2.
    L. G. Valiant, “A bridging model for parallel computation,” Commun. ACM 33 (8), 103–111 (1990). doi 10.1145/79173.79181CrossRefGoogle Scholar
  3. 3.
    F. M. Auf der Heide and R. Wanka, “Parallel bridging models and their impact on algorithm design,” in Proceedings of the International Conference on Computational Science—ICCS’01, Part II, Lect. Notes Comput. Sci. 2074, 628–637 (2001). doi 10.1007/3-540-45718-6_68zbMATHGoogle Scholar
  4. 4.
    L. G. Valiant, “A bridging model for multi-core computing,” J. Comput. Syst. Sci. 77, 154–166 (2011). doi 10.1016/j.jcss.2010.06.012MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    V. Blanco, J. A. Gonzalez, C. Leon, C. Rodriguez, G. Rodriguez, and M. Printista, “Predicting the performance of parallel programs,” Parallel Comput. 30, 337–356 (2004). doi 10.1016/j.parco.2003.11.004CrossRefGoogle Scholar
  6. 6.
    A. V. Gerbessiotis, “Extending the BSP model for multi-core and out-of-core computing: MBSP,” Parallel Comput. 41, 90–102 (2015). doi 10.1016/j.parco.2014.12.002CrossRefGoogle Scholar
  7. 7.
    H. Cha and D. Lee, “H-BSP: a hierarchical BSP computationmodel,” J. Supercomput. 18, 179–200 (2001). doi 10.1023/A:1008113017444CrossRefzbMATHGoogle Scholar
  8. 8.
    D. Culler, R. Karp, D. Patterson, A. Sahay, K. E. Schauser, E. Santos, R. Subramonian, and T. von Eicken, “LogP: towards a realistic model of parallel computation,” in Proceedings of the 4th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming—PPOPP’93 (ACM Press, New York, 1993), pp. 1–12. doi 10.1145/155332.155333Google Scholar
  9. 9.
    A. Alexandrov, M. F. Ionescu, K. E. Schauser, and C. Scheiman, “LogGP: incorporating long messages into the LogP model for parallel computation,” J. Parallel Distrib. Comput. 44, 71–79 (1997). doi 10.1006/jpdc.1997.1346CrossRefGoogle Scholar
  10. 10.
    G. Liu, Y. Wang, T. Zhao, J. Gu, and D. Li, “mHLogGP: a parallel computation model for CPU/GPU,” in Network and Parallel Computing, Proceedings of the 9th IFIP International Conference NPC 2012, Gwangju, Korea, Sept. 6–8, 2012, Ed. by J. J. Park, A. Zomaya, and S. S. Yeo (Springer, Berlin, Heidelberg, 2012), pp. 217–224. doi 10.1007/978-3-642-35606-3_25Google Scholar
  11. 11.
    F. Lu, J. Song, and Y. Pang, “HLognGP: a parallel computation model forGPUclusters,” Concurr. Comput.: Practice Experience 27, 4880–4896 (2015). doi 10.1002/cpe.3475CrossRefGoogle Scholar
  12. 12.
    F. Ino, N. Fujimoto, and K. Hagihara, “LogGPS: a parallel computational model for synchronization analysis,” ACM SIGPLAN Not. 36 (7), 133–142 (2001). doi 10.1145/568014.379592CrossRefGoogle Scholar
  13. 13.
    K. W. Cameron, R. Ge, and X. Sun, “logNP and log3P: accurate analytical models of point-to-point communication in distributed systems,” IEEE Trans. Comput. 56, 314–327 (2007). doi 10.1109/TC.2007.38MathSciNetCrossRefGoogle Scholar
  14. 14.
    L. Yuan, Y. Zhang, Y. Tang, L. Rao, and X. Sun, “LogGPH: a parallel computational model with hierarchical communication awareness,” in Proceedings of the 2010 13th IEEE International Conference on Computational Science and Engineering CSE’10 (IEEE Comput. Soc., Washington, DC, 2010), pp. 268–274. doi 10.1109/CSE.2010.40Google Scholar
  15. 15.
    A. Tiskin, “BSP (Bulk Synchronous Parallelism),” in Encyclopedia of Parallel Computing (Springer US, Boston, MA, 2011), pp. 192–199. doi 10.1007/978-0-387-09766-4_311Google Scholar
  16. 16.
    L. A. Hageman and D. M. Young, Applied Iterative Methods (Academic, New York, London, Toronto, Sydney, San Francisco, 1981).zbMATHGoogle Scholar
  17. 17.
    C. T. Kelley, IterativeMethods for Linear and Nonlinear Equations (Soc. Ind. Appl. Math., Philadelphia, 1995). doi 10.1137/1.9781611970944CrossRefGoogle Scholar
  18. 18.
    A. Hadjidimos, “A survey of the iterativemethods for the solution of linear systems by extrapolation, relaxation and other techniques,” J. Comput. Appl.Math. 20, 37–51 (1987). doi 10.1016/0377-0427(87)90124-5MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    S. Ma and A. Chronopoulos, “Implementation of iterative methods for large sparse nonsymmetric linear systems on a parallel vector machine,” Int. J. High Perform. Comput. Appl. 4 (4), 9–24 (1990). doi 10.1177/109434209000400402Google Scholar
  20. 20.
    S. Sahni and G. Vairaktarakis, “The master-slave paradigm in parallel computer and industrial settings,” J. Global Optimiz. 9, 357–377 (1996). doi 10.1007/BF00121679MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    F. Darema, D. A. George, V. A. Norton, and G. F. Pfister, “A single-program-multiple-data computational model for EPEX/FORTRAN,” Parallel Comput. 7, 11–24 (1988). doi 10.1016/0167-8191(88)90094-4CrossRefzbMATHGoogle Scholar
  22. 22.
    F. Darema, “SPMD computational model,” in Encyclopedia of Parallel Computing (SpringerUS, Boston, MA, 2011), pp. 1933–1943. doi 10.1007/978-0-387-09766-4_26Google Scholar
  23. 23.
    I. Sokolinskaya and L. B. Sokolinsky, “On the solution of linear programming problems in the age of big data,” in Parallel Computational Technologies. PCT 2017, Commun. Comput. Inform. Sci. 753, 86–100 (2017). doi 10.1007/978-3-319-67035-5_7Google Scholar
  24. 24.
    I. Sokolinskaya and L. B. Sokolinsky, “Scalability evaluation of NSLP algorithm for solving non-stationary linear programming problems on cluster computing systems,” in Supercomputing, RuSCDays 2017, Commun. Comput. Inform. Sci. 793, 40–53 (2017). doi 10.1007/978-3-319-71255-0_4Google Scholar

Copyright information

© Pleiades Publishing, Ltd. 2018

Authors and Affiliations

  1. 1.South Ural State University (National Research University)ChelyabinskRussia

Personalised recommendations