Skip to main content
SpringerLink
Log in

Application of Alternating Decision Trees in Selecting Sparse Linear Solvers

  • Chapter
  • First Online:
Software Automatic Tuning

Abstract

The solution of sparse linear systems, a fundamental and resource-intensive task in scientific computing, can be approached through multiple algorithms. Using an algorithm well adapted to characteristics of the task can significantly enhance the performance, such as reducing the time required for the operation, without compromising the quality of the result. However, the “best” solution method can vary even across linear systems generated in course of the same PDE-based simulation, thereby making solver selection a very challenging problem. In this paper, we use a machine learning technique, Alternating Decision Trees (ADT), to select efficient solvers based on the properties of sparse linear systems and runtime-dependent features, such as the stages of simulation. We demonstrate the effectiveness of this method through empirical results over linear systems drawn from computational fluid dynamics and magnetohydrodynamics applications. The results also demonstrate that using ADT can resolve the problem of “over-fitting”, which occurs when limited amount of data is available.

This work was sponsored in part by the U.S. National Science Foundation under award 04-06403 to the University of Tennessee, with subcontracts to Columbia University, the University of California at San Diego, and the College of Information Science and Technology at the University of Nebraska, Omaha.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    This work was sponsored in part by the U.S. National Science Foundation under award 04-06403 to the University of Tennessee, with subcontracts to Columbia University, the University of California at San Diego, and the College of Information Science and Technology at the University of Nebraska, Omaha.

  2. 2.

    We say that a set of examples is drawn Independently and Identically Distributed (IID) according to \(\mathcal{D}\) if they can be seen as independent draws from the fixed distribution \(\mathcal{D}\). In other words, if they are independent random variables all having distribution \(\mathcal{D}\).

References

  1. Axelsson O (1987) A survey of preconditioned iterative methods for linear systems of equations. BIT

    Google Scholar 

  2. Balay S, Buschelman K, Gropp W, Kaushik D, Knepley M, McInnes L, Smith BF, Zhang H (2004) PETSc users manual. Technical Report ANL-95/11 - Revision 2.2.1, Argonne National Laboratory, http://www.mcs.anl.gov/petsc

  3. Barrett R, Berry M, Dongarra J, Eijkhout V, Romine C (1996) Algorithmic bombardment for the iterative solution of linear systems: a polyiterative approach. J Comput Appl Math 74:91–110

    Article  MathSciNet  MATH  Google Scholar 

  4. Bennett BAV, Smooke MD (1999) Local rectangular refinement with application to nonreacting and reacting fluid flow problems. J Comput Phys 151:648–727

    Article  MathSciNet  Google Scholar 

  5. Bhowmick S, McInnes LC, Norris B, Raghavan P (2003) The role of multi-method linear solvers in pde-based simulations. In: Sloot PMA, Tan CJK, Dongarra JJ, Hoekstra AG (eds) Lecture Notes in computer science, computational science and its applications-ICCSA 2003, vol 2667. Springer, pp 828–839

    Google Scholar 

  6. Bhowmick S, Raghavan P, McInnes L, Norris B (2004) Faster PDE-based simulations using robust composite linear solvers. Future Generation Comput Syst 20:373–386

    Article  Google Scholar 

  7. Bhowmick S, Raghavan P, Teranishi K (2002) A combinatorial scheme for developing efficient composite solvers. In: Sloot PMA, Tan CJK, Dongarra JJ, Hoekstra AG (eds) Lecture notes in computer science, computational science-ICCS 2002, vol 2330. Springer, pp 325–334

    Google Scholar 

  8. Bhowmick S, Toth B, Raghavan P (2009) Towards low-cost, high-accuracy classifiers for linear solver selection. In: ICCS (1), pp 463–472

    Google Scholar 

  9. Breiman L (1998) Arcing classifiers. Ann Stat 26(3):801–849

    Article  MathSciNet  MATH  Google Scholar 

  10. Davis T (1997) University of Florida Sparse Matrix Collection. NA Digest, 97(23). http://www.cise.ufl.edu/research/sparse/matrices

  11. Demmel J, Dongarra J, Eijkhout V, Fuentes E, Petitet A, Vuduc R, Whaley RC, Yelick K (2004) Self adapting linear algebra algorithms and software. IEEE Proceedings

    Google Scholar 

  12. Dongarra J, Eijkhout V (2003) Self adapting numerical algorithm for next generation applications. Int J High Perform Comput Appl 17(2):125–132

    Article  Google Scholar 

  13. Dongarra J, Eijkhout V (2003) Self-adapting numerical software and automatic tuning of heuristics. In: Proceedings of the International Conference on Computational Science, June 2–4, 2003, St. Petersburg (Russia) and Melbourne (Australia), Lecture Notes in Computer Science 2660, Springer, pp 759–770

    Google Scholar 

  14. Driven-Cavity. Nonlinear Driven Cavity and Pseudotransient Timestepping in 2D. http://www-unix.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/src/snes/examples/tutorials/ex27.c.html.

  15. Drucker H, Cortes C (1996) Boosting decision trees. In: NIPS8, pp 479–485

    Google Scholar 

  16. Duff IS, Erisman AM, Rei JK (1986) Direct methods for sparse matrices. Clarendon, Oxford

    MATH  Google Scholar 

  17. Eijkhout V, Fuentes E Anamod online documentation. http://www.tacc.utexas.edu/~eijkhout/doc/anamod/html/

  18. Eijkhout V, Fuentes E A proposed standard for numerical metadata. submitted to ACM Trans Math Software

    Google Scholar 

  19. Ern A, Giovangigli V, Keyes DE, Smooke MD (1994) Towards polyalgorithmic linear system solvers for nonlinear elliptic problems. SIAM J Sci Comput 15(3):681–703

    Article  MathSciNet  MATH  Google Scholar 

  20. Falgout RD, Yang UM (2002) hypre: A library of high performance preconditioners. In: International Conference on Computational Science, vol 3. pp 632–641

    Google Scholar 

  21. Freund Y, Mason L (1999) The alternating decision tree learning algorithm. In: Proceedings of the 16th International Conference on Machine Learning. pp 124–133

    Google Scholar 

  22. Freund Y, Schapire RE (1997) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55(1):119–139

    Article  MathSciNet  MATH  Google Scholar 

  23. Freund Y, Schapire RE (1999) A short introduction to boosting. J Jpn Society Artif Intell 14(5):771–780

    Google Scholar 

  24. Fuentes E (2007) Statistical and machine learning techniques applied to algorithm selection for solving sparse linear systems. Doctoral Dissertation, University of Tennessee

    Google Scholar 

  25. Gannon D, Bramley R, Stuckey T, Balasubramanian J, Villacis J, Akman E, Berg F, Diwan S, Govindaraju M (2000) The linear system analyzer. In: Houstis EN, Rice JR, Gallopoulos E, Bramley R (eds) Enabling technologies for computational science. Kluwer, Dordrecht

    Google Scholar 

  26. Gropp WD, Keyes DE, McInnes LC, Tidriri MD (2000) Globalized Newton-Krylov-Schwarz algorithms and software for parallel implicit CFD. Int J High Perform Comput Appl 14: 102–136

    Article  Google Scholar 

  27. Hastie T, Tibshirani R, Friedman JH (2001) The elements of statistical learning. Springer

    Google Scholar 

  28. Holloway A, Chen T-Y (2007) Neural networks for predicting the behavior of preconditioned iterative solvers. To appear in the International Conference on Computational Science

    Google Scholar 

  29. Holloway A, Chen T-Y (2007) Neural networks for predicting the behavior of preconditioned iterative solvers. In: ICCS ’07: Proceedings of the 7th international conference on Computational Science, Part I, Springer, Berlin, Heidelberg, pp 302–309

    Google Scholar 

  30. Houstis EN, Catlin AC, Rice JR, Verykios VS, Ramakrishnan N, Houstis CE (2000) PYTHIA-II: a knowledge/database system for managing performance data and recommending scientific software. Trans Math Softw 26(2):227–253

    Article  MATH  Google Scholar 

  31. Kelley CT, Keyes DE (1998) Convergence analysis of pseudo-transient continuation. SIAM J Numer Anal 35:508–523

    Article  MathSciNet  MATH  Google Scholar 

  32. Kuefler E, Chen T-Y (2008) On using reinforcement learning to solve sparse linear systems. In: ICCS ’08: Proceedings of the 8th international conference on Computational Science, Part I, Springer, Berlin, Heidelberg, pp 955–964

    Google Scholar 

  33. LCRC. Argonne National Laboratory Computing Project. http://www.lcrc.anl.gov/jazz/index.php

  34. M3D-Home. http://w3.pppl.gov/~jchen/index.html

  35. McCormick SF, Copper Mountain Conference on Multigrid Methods (1988) In: McCormick SF, Dekker M (eds) Multigrid methods: theory, applications, and supercomputing. New York

    Google Scholar 

  36. McInnes L, Norris B, Bhowmick S, Raghavan P (2003) Adaptive sparse linear solvers for implicit cfd using Newton-Krylov algorithms. In: Proceedings of the Second MIT Conference on Computational Fluid and Solid Mechanics, June 17–20

    Google Scholar 

  37. MLJava. http://seed.ucsd.edu/twiki/bin/view/Softtools/MLJavaPage

  38. Nocedal J, Wright SJ (1999) Numerical optimization. Springer, New York

    Book  MATH  Google Scholar 

  39. Park W, Belova EV, Fu GY, Tang XZ, Strauss HR, Sugiyama LE (1999) Plasma simulation studies using multilevel physics models. Phys Plasmas 6(5):1796–1803

    Article  Google Scholar 

  40. Quinlan JR (1996) Bagging, boosting, and C4.5. In: Proceedings of the Thirteenth National Conference on Artificial Intelligence, pp 725–730

    Google Scholar 

  41. Saad Y (1995) Iterative methods for sparse linear systems. PWS Publishing

    Google Scholar 

  42. Schapire RE (1990) The strength of weak learnability. Mach Learn 5(2):197–227

    Google Scholar 

  43. SPOOLES. Sparse direct solver. http://www.netlib.org/linalg/spooles/spooles.2.2.html

  44. SuperLU. Sparse direct solver. http://crd.lbl.gov/~xiaoye/SuperLU

  45. Wikipedia. Receiver operating characteristic. http://en.wikipedia.org/wiki/Receiver_operating_characteristic

  46. Witten IH, Frank E (2005) Data mining:practical machine learning tools and techniques, 2nd edn. Morgan Kaufmann

    Google Scholar 

  47. Witten IH, Frank E (2005) Data mining: practical machine learning tools and techniques. Morgan Kaufmann, San Francisco

    MATH  Google Scholar 

  48. Xu S, Zhang J. A data mining approach to matrix preconditioning problem. In: Proceedings of the Eighth Workshop on Mining Scientific and Engineering Datasets (MSD05)

    Google Scholar 

Download references

Acknowledgments

We would like to thank Jin Chen of the Princeton Plasma Physics Lab for providing us with the M3D matrices. We are also grateful to Raphael Pelossof of Columbia University for his package to render ROC curves from the MLJava output files.

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Nebraska, Omaha, NE, USA

    Sanjukta Bhowmick

Authors
  1. Sanjukta Bhowmick
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Victor Eijkhout
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yoav Freund
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Erika Fuentes
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. David Keyes
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sanjukta Bhowmick .

Editor information

Editors and Affiliations

  1. Central Research Laboratory, Hitachi Ltd., Higashi-Koigakubo 1-280, Kokubunji-shi, Tokyo, 185-8601, Japan

    Ken Naono

  2. Cray, Inc., Jackson St. 380, St Paul, 55101, Minnesota, USA

    Keita Teranishi

  3. Dept. Computer & Information Sciences, University of Delaware, Smith Hall 101, Newark, 19716, Delaware, USA

    John Cavazos

  4. Dept. Computer Science, University of Tokyo, Hongo 7-3-1, Tokyo, 113-0033, Japan

    Reiji Suda

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer New York

About this chapter

Cite this chapter

Bhowmick, S., Eijkhout, V., Freund, Y., Fuentes, E., Keyes, D. (2011). Application of Alternating Decision Trees in Selecting Sparse Linear Solvers. In: Naono, K., Teranishi, K., Cavazos, J., Suda, R. (eds) Software Automatic Tuning. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-6935-4_10

Download citation

  • DOI: https://doi.org/10.1007/978-1-4419-6935-4_10

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4419-6934-7

  • Online ISBN: 978-1-4419-6935-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation