High Performance Inverse Preconditioning

  • George A. GravvanisEmail author
Original Paper


The derivation of parallel numerical algorithms for solving sparse linear systems on modern computer systems and software platforms has attracted the attention of many researchers over the years. In this paper we present an overview on the design issues of parallel approximate inverse matrix algorithms, based on an anti-diagonal “wave pattern” approach and a “fish-bone” computational procedure, for computing explicitly various families of exact and approximate inverses for solving sparse linear systems. Parallel preconditioned conjugate gradient-type schemes in conjunction with parallel approximate inverses are presented for the efficient solution of sparse linear systems. Applications of the proposed parallel methods by solving characteristic sparse linear systems on symmetric multiprocessor systems and distributed systems are discussed and the parallel performance of the proposed schemes is given, using MPI, OpenMP and Java multithreading.


Conjugate Gradient Method Relative Speedup Sparse Linear System Approximate Inverse Sparse Approximate Inverse 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Akl SG (1997) Parallel computation: models and methods. Prentice-Hall, Englewood Cliffs Google Scholar
  2. 2.
    Baker M, Carpenter B, Fox G, Ko SH, Lim S (1999) mpiJava: an object-oriented Java interface to MPI. In: Proceedings of the 11th IPPS/SPDP’99 workshops held in conjunction with the 13th international parallel processing symposium and 10th symposium on parallel and distributed processing, pp 748–762 Google Scholar
  3. 3.
    Benzi M (2002) Preconditioning techniques for large linear systems: a survey. J Comput Phys 182:418–477 zbMATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Benzi M, Meyer CD, Tuma M (1996) A sparse approximate inverse preconditioner for the conjugate gradient method. SIAM J Sci Comput 17:1135–1149 zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Boisvert RF (1991) Algorithms for special tridiagonal systems. SIAM J Sci Stat Comput 12(2):423–442 zbMATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Bull M, Kambites ME (2000) JOMP—an OpenMP-like interface for Java. In: Proceedings of the 2000 ACM Java grande conference, pp 44–53 Google Scholar
  7. 7.
    Bull M, Telford S (2000) Programming models for parallel Java applications. UKHEC Newslett 2:9 Google Scholar
  8. 8.
    Bull J, Smith L, Pottage L, Freeman R (2001) Benchmarking Java against C and Fortran for scientific applications. In: ACM 2001 Java grande/ISCOPE conference, pp 97–105 Google Scholar
  9. 9.
    Christofer TW, Thiruvathukal GK (2000) High-performance Java platform computing. Sun Microsystems Press, Santa Clara Google Scholar
  10. 10.
    Chronopoulos AT, Gear CW (1989) s-Step iterative methods for symmetric linear systems. J Comput Appl Math 25:153–168 zbMATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Cosgrove JDF, Dias JC, Griewank A (1992) Approximate inverse preconditioning for sparse linear systems. Int J Comput Math 44:91–110 zbMATHCrossRefGoogle Scholar
  12. 12.
    D’Azevedo E, Eijkhout V, Romine C (1993) A matrix framework for conjugate gradient methods and some variants of CG with less synchronization overhead. In: Proceedings of the sixth SIAM conference on parallel processing for scientific computing. SIAM, Philadelphia, pp 644–646 Google Scholar
  13. 13.
    de Brozolo CR, Robert Y (1988) Parallel and vector conjugate gradient-like algorithms for sparse nonsymmetric linear systems. In: Proceedings of the 2nd international conference on supercomputing, pp 478–487 Google Scholar
  14. 14.
    Demmel J, Heath M, van der Vorst H (1993) Parallel numerical linear algebra. In: Acta numerica 1993. Cambridge University Press, Cambridge Google Scholar
  15. 15.
    Dongarra J, van der Vorst H (1992) Performance of various computers using standard sparse linear equations solving techniques. Supercomputer 9(5):17–29 Google Scholar
  16. 16.
    Dongarra J, Duff I, Sorensen D, van der Vorst H (1998) Numerical linear algebra for high-performance computers. SIAM, Philadelphia zbMATHGoogle Scholar
  17. 17.
    Dubois P, Greenbaum A, Rodrigue G (1979) Approximating the inverse of a matrix for use in iterative algorithms on vector processors. Computing 22:257–268 zbMATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    Duff I (2000) The impact of high performance computing in the solution of linear systems: trends and problems. J Comput Appl Math 123:515–530 zbMATHCrossRefMathSciNetGoogle Scholar
  19. 19.
    Duff I, Erisman M, Reid J (1986) Direct methods for sparse matrices. Oxford University Press, Oxford zbMATHGoogle Scholar
  20. 20.
    Epitropou VN, Giannoutakis KM, Gravvanis GA (2007) Java multithreading based parallel preconditioned generalized conjugate gradient type methods. In: Kranzlmuller D, Schreiner W, Volkert J (eds) Proc. of the sixth intern symposium on parallel and distributed computing 2007. IEEE Computer Society Press, Los Alamitos, pp 311–318 Google Scholar
  21. 21.
    Evans DJ (1967) The use of preconditioning in iterative methods for solving linear equations with symmetric positive definite matrices. J Inst Math Appl 4:295–314 CrossRefGoogle Scholar
  22. 22.
    Evans DJ (1983) Preconditioning methods: theory and applications. Gordon and Breach, New York Google Scholar
  23. 23.
    Giannoutakis KM, Gravvanis GA (2007) On the design of parallel finite element approximate inverses. In: Arabnia HR (ed) Proceedings of the 2007 international conference on parallel and distributed processing techniques and applications (PDPTA ’2007), vol I. CSREA Press, Las Vegas, pp 292–298 Google Scholar
  24. 24.
    Giannoutakis KM, Gravvanis GA (2008) Parallel approximate finite element inverses on symmetric multiprocessor systems. In: Bubak M, van Albada GD, Dongarra J, Sloot PMA (eds) Computational science—ICCS 2008: 8th international conference proceedings. Lecture notes of computer science, vol 5101, Part I. Springer, New York, pp 925–934 Google Scholar
  25. 25.
    Giannoutakis KM, Gravvanis GA (2008) Design and implementation of parallel approximate inverse classes using OpenMP. Concurr Comput: Pract Exp, to appear Google Scholar
  26. 26.
    Giannoutakis KM, Gravvanis GA (2008) High performance finite element approximate inverse preconditioning. Appl Math Comput 201:293–304 zbMATHCrossRefMathSciNetGoogle Scholar
  27. 27.
    Giannoutakis KM, Gravvanis GA, Clayton B, Patil A, Enright T, Morrison JP (2007) Matching high performance approximate inverse preconditioning to architectural platforms. J Supercomput 42(2):145–163 CrossRefGoogle Scholar
  28. 28.
    Grama A, Gupta A, Karypis G, Kumar V (2003) Introduction to parallel computing. Addison-Wesley, Reading Google Scholar
  29. 29.
    Gravvanis GA (1996) The rate of convergence of explicit approximate inverse preconditioning. Int J Comput Math 60:77–89 zbMATHCrossRefGoogle Scholar
  30. 30.
    Gravvanis GA (1998) Parallel matrix techniques. In: Papailiou KD, Tsahalis D, Periaux J, Hirsch C, Pandolfi M (eds) Computational fluid dynamics 98, vol I, Part 1. Wiley, New York, pp 472–477 Google Scholar
  31. 31.
    Gravvanis GA (1998) An approximate inverse matrix technique for arrowhead matrices. Int J Comput Math 70:35–45 zbMATHCrossRefMathSciNetGoogle Scholar
  32. 32.
    Gravvanis GA (1999) Parallel preconditioned algorithms for solving special tridiagonal systems. In: Proceedings of dynamical systems and applications, vol III. Dynamic Publishers, Atlanta, pp 241–248 Google Scholar
  33. 33.
    Gravvanis GA (2000) Explicit isomorphic iterative methods for solving arrow-type linear systems. Int J Comput Math 74(2):195–206 zbMATHCrossRefMathSciNetGoogle Scholar
  34. 34.
    Gravvanis GA (2000) Solving parabolic and nonlinear 1D problems with periodic boundary conditions. In: CD-ROM proceedings of the European congress on computational methods in applied sciences and engineering 2000 Google Scholar
  35. 35.
    Gravvanis GA (2002) Explicit approximate inverse preconditioning techniques. Arch Comput Methods Eng 9(4):371–402 zbMATHCrossRefGoogle Scholar
  36. 36.
    Gravvanis GA (2009) OpenMP based parallel normalized direct methods for sparse finite element linear systems. J Supercomput 47(1):44–52 CrossRefMathSciNetGoogle Scholar
  37. 37.
    Gravvanis GA, Epitropou VN (2008) Java multithreading based parallel approximate arrow-type inverses. Concurr Comput: Pract Exp 20(10):1151–1172 CrossRefGoogle Scholar
  38. 38.
    Gravvanis GA, Giannoutakis KM (2003) On the rate of convergence and complexity of finite element normalized explicit approximate inverse preconditioning. In: Bathe KJ (ed) Computational fluid and solid mechanics 2003, vol 2. Elsevier, Amsterdam, pp 1963–1967 Google Scholar
  39. 39.
    Gravvanis GA, Giannoutakis KM (2003) Normalized explicit finite element approximate inverses. Int J Differ Equ Appl 6(3):253–267 MathSciNetGoogle Scholar
  40. 40.
    Gravvanis GA, Giannoutakis KM (2004) Parallel approximate finite element inverse preconditioning on distributed systems. In: Proceedings of the third international symposium on parallel and distributed computing 2004 (ISPDC 2004) and the third international workshop on algorithms, models and tools for parallel computing on heterogeneous networks (HeteroPar 2004). IEEE Computer Society Press, Los Alamitos, pp 277–283 CrossRefGoogle Scholar
  41. 41.
    Gravvanis GA, Giannoutakis KM (2004) Normalized explicit finite element approximate inverse preconditioning. Int J Comput Struct 82(28):2377–2388 CrossRefMathSciNetGoogle Scholar
  42. 42.
    Gravvanis GA, Giannoutakis KM (2006) On the performance of parallel normalized explicit preconditioned conjugate gradient—type methods. In: CD-ROM proceedings of the 20th international parallel and distributed processing symposium 2006 (IPDPS 2006), General Co-Chair: Paul Spirakis and H.J. Siegel, IEEE Computer Society Google Scholar
  43. 43.
    Gravvanis GA, Giannoutakis KM (2006) Parallel exact and approximate arrow-type inverses on symmetric multiprocessor systems. In: ICCS 2006, Part I. LNCS, vol 3991. Springer, New York, pp 506–513 CrossRefGoogle Scholar
  44. 44.
    Gravvanis GA, Giannoutakis KM (2006) Normalized explicit preconditioned methods for solving 3D boundary value problems on uniprocessor and distributed systems. Int J Numer Methods Eng 65(1):84–110 zbMATHCrossRefMathSciNetGoogle Scholar
  45. 45.
    Gravvanis GA, Giannoutakis KM (2008) An improved parallel algorithm for computing approximate inverses by reducing synchronizations. In: Proc of the seventh intern symposium on parallel and distributed computing 2008. IEE Computer Society Press, to appear Google Scholar
  46. 46.
    Gravvanis GA, Giannoutakis KM (2008) Fast parallel finite element approximate inverses. Comput Model Eng Sci 32(1):35–44 MathSciNetGoogle Scholar
  47. 47.
    Gravvanis GA, Lipitakis EA (1996) An explicit sparse unsymmetric finite element solver. Commun Numer Methods Eng 12:21–29 zbMATHCrossRefGoogle Scholar
  48. 48.
    Gravvanis GA, Platis AN, Giannoutakis KM, Violentis JB, Lipitakis EA (2003) Performability evaluation of multitasking and multiprocessor systems by explicit approximate inverses. In: Arabnia HR, Mun Y (eds) Proceedings of the international conference on parallel and distributed processing techniques and applications (PDPTA 2003), vol III. CSREA Press, Las Vegas, pp 1324–1331 Google Scholar
  49. 49.
    Gravvanis GA, Epitropou VN, Giannoutakis KM (2007) On the performance of parallel approximate inverse preconditioning using Java multithreading techniques. Appl Math Comput 190:255–270 zbMATHCrossRefMathSciNetGoogle Scholar
  50. 50.
    Grote MJ, Huckle T (1997) Parallel preconditioning with sparse approximate inverses. SIAM J Sci Comput 18:838–853 zbMATHCrossRefMathSciNetGoogle Scholar
  51. 51.
    Grunwald D, Vajracharya S (1994) Efficient barriers for distributed shared memory computers. In: Proceedings of the 8th international parallel processing symposium. IEEE Computer Society Press, Los Alamitos Google Scholar
  52. 52.
    Hackbusch W (1994) Iterative solution of large linear systems of equations. Springer, New York Google Scholar
  53. 53.
    Hestenes MR, Stiefel E (1954) Methods of conjugate gradients for solving linear systems. J Res Natl Bur Stand 49:409–436 MathSciNetGoogle Scholar
  54. 54.
    Huckle T (1998) Efficient computations of sparse approximate inverses. Numer Linear Algorithms Appl 5:57–71 zbMATHCrossRefMathSciNetGoogle Scholar
  55. 55.
    Huckle T (1999) Approximate sparsity patterns for the inverse of a matrix and preconditioning. Appl Numer Math 30:291–303 zbMATHCrossRefMathSciNetGoogle Scholar
  56. 56.
    Kolotilina LY, Yeremin AY (1993) Factorized sparse approximate inverse preconditioning. SIAM J Matrix Anal Appl 14:45–58 zbMATHCrossRefMathSciNetGoogle Scholar
  57. 57.
    Li W, Li E, Meng R, Wang T, Dulong C (2006) Performance analysis of Java concurrent programming: a case study of video mining system. In: CD-ROM proceedings of the 20th international parallel and distributed processing symposium 2006 (IPDPS 2006). IEEE Computer Society, Los Alamitos Google Scholar
  58. 58.
    Lipitakis EA (1983) A normalized sparse linear equation solver. J Comput Appl Math 9:287–298 zbMATHCrossRefGoogle Scholar
  59. 59.
    Lipitakis EA, Evans DJ (1984) Solving linear finite element systems by normalized approximate matrix factorization semi-direct methods. Comput Methods Appl Mech Eng 43:1–19 zbMATHCrossRefMathSciNetGoogle Scholar
  60. 60.
    Lipitakis EA, Evans DJ (1987) Explicit semi-direct methods based on approximate inverse matrix techniques for solving boundary-value problems on parallel processors. Math Comput Simul 29:1–17 zbMATHCrossRefGoogle Scholar
  61. 61.
    Lipitakis EA, Gravvanis GA (1995) Explicit preconditioned iterative methods for solving large unsymmetric finite element systems. Computing 54:167–183 zbMATHCrossRefMathSciNetGoogle Scholar
  62. 62.
    Marr DT, Binns F, Hill D, Hinton G, Koufaty D, Miller A, Upton M (2002) Hyper-threading technology architecture and microarchitecture, hyper-threading technology. Intel Technol J 6(1):4–13 Google Scholar
  63. 63.
    O’Leary D, Stewart G (1990) Computing the eigenproblem and eigenvectors of arrowhead matrices. J Comput Phys 90:497–505 zbMATHCrossRefMathSciNetGoogle Scholar
  64. 64.
    Parlett B, Nour-Omid B (1985) The use of refined error bound when updating eigenvalues of tridiagonals. Linear Algebra Appl 68:179–219 zbMATHCrossRefMathSciNetGoogle Scholar
  65. 65.
    Quinn MJ (2003) Parallel programming in C with MPI and OpenMP. Mc-Graw Hill, New York Google Scholar
  66. 66.
    Saad Y (1996) Iterative methods for sparse linear systems. PWS, Boston zbMATHGoogle Scholar
  67. 67.
    Saad Y, van der Vorst HA (2000) Iterative solution of linear systems in the 20th century. J Comput Appl Math 123:1–33 zbMATHCrossRefMathSciNetGoogle Scholar
  68. 68.
    Sonneveld P (1989) CGS: a fast Lanczos-type solver for nonsymmetric linear systems. SIAM J Sci Stat Comput 10:36–52 zbMATHCrossRefMathSciNetGoogle Scholar
  69. 69.
    Tian X, Bik A, Girkar M, Grey P, Saito H, Su E (2002) Intel OpenMP C++/Fortran compiler for hyper-threading technology: implementation and performance, hyper-threading technology. Intel Technol J 6(1):36–45 Google Scholar
  70. 70.
    van der Vorst HA (1989) High performance preconditioning. SIAM J Sci Stat Comput 10:1174–1185 zbMATHCrossRefGoogle Scholar
  71. 71.
    van der Vorst HA (1992) Bi-CGSTAB: a fast and smoothly converging variant of Bi-CG for the solution of non-symmetric linear systems. SIAM J Sci Stat Comput 13:631–644 zbMATHCrossRefGoogle Scholar

Copyright information

© CIMNE, Barcelona, Spain 2008

Authors and Affiliations

  1. 1.Department of Electrical and Computer Engineering, School of EngineeringDemocritus University of ThraceXanthiGreece

Personalised recommendations