Numerical Algorithms

, Volume 76, Issue 1, pp 163–189 | Cite as

A robust approach for finding all well-separated solutions of sparse systems of nonlinear equations

  • Ali BaharevEmail author
  • Ferenc Domes
  • Arnold Neumaier
Open Access
Original Paper


Tearing is a long-established decomposition technique, widely adapted across many engineering fields. It reduces the task of solving a large and sparse nonlinear system of equations to that of solving a sequence of low-dimensional ones. The most serious weakness of this approach is well-known: It may suffer from severe numerical instability. The present paper resolves this flaw for the first time. The new approach requires reasonable bound constraints on the variables. The worst-case time complexity of the algorithm is exponential in the size of the largest subproblem of the decomposed system. Although there is no theoretical guarantee that all solutions will be found in the general case, increasing the so-called sample size parameter of the method improves robustness. This is demonstrated on two particularly challenging problems. Our first example is the steady-state simulation a challenging distillation column, belonging to an infamous class of problems where tearing often fails due to numerical instability. This column has three solutions, one of which is missed using tearing, but even with problem-specific methods that are not based on tearing. The other example is the Stewart–Gough platform with 40 real solutions, an extensively studied benchmark in the field of numerical algebraic geometry. For both examples, all solutions are found with a fairly small amount of sampling.


Decomposition methods Diakoptics Large-scale systems of equations Numerical instability Sparse matrices Tearing 


  1. 1.
    Aspen Technology, Inc (2009) Aspen Simulation Workbook, Version Number: V7.1. Burlington, MA, USA. EO and SM Variables and Synchronization, p. 110Google Scholar
  2. 2.
    Auger, A., Hansen, N.: A restart CMA evolution strategy with increasing population size. In: 2005. The 2005 IEEE Congress on Evolutionary Computation, vol. 2, pp 1769–1776. IEEE (2005)Google Scholar
  3. 3.
    Bachmann, B., Aronßon, P, Fritzson, P.: Robust initialization of differential algebraic equations. In: 1st International Workshop on Equation-Based Object-Oriented Modeling Languages and Tools, vol. 2007, pp 151–163, Linköping University Electronic Press; Linköpings universitet, Linköping Electronic Conference Proceedings (2007)Google Scholar
  4. 4.
    Baharev, A.:, Exact and heuristic methods for tearing (2016)
  5. 5.
    Baharev, A., Neumaier, A.: A globally convergent method for finding all steady-state solutions of distillation columns. AIChE J. 60, 410–414 (2014)CrossRefGoogle Scholar
  6. 6.
    Baharev, A., Kolev, L., Rév, E: Computing multiple steady states in homogeneous azeotropic and ideal two-product distillation. AIChE J. 57, 1485–1495 (2011)CrossRefGoogle Scholar
  7. 7.
    Baharev, A., Domes, F., Neumaier, A.: Online supplementary material of the present manuscript. (2016a)
  8. 8.
    Baharev, A., Schichl, H., Neumaier, A.: Decomposition methods for solving nonlinear systems of equations,, submitted (2016b)
  9. 9.
    Baharev, A., Schichl, H., Neumaier, A.: Ordering matrices to bordered lower triangular form with minimal border width,, submitted (2016c)
  10. 10.
    Bates, D.J., Hauenstein, J.D., Sommese, A.J.: Efficient path tracking methods. Numer. Algorithm. 58(4), 451–459 (2011)MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    Bates, D.J., Hauenstein, J.D., Sommese, A.J., Wampler, C.W.: Numerically Solving Polynomial Systems with Bertini, Software, Environments and Tools, vol 25. SIAM, Philadelphia, PA (2013)Google Scholar
  12. 12.
    Bates, D.J., Newell, A.J., Niemerg, M.: BertiniLab: A MATLAB interface for solving systems of polynomial equations. Numer. Algorithm. 71(1), 229–244 (2016)MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    Beelitz, T., Frommer, A., Lang, B., Willems, P.: Symbolic–numeric techniques for solving nonlinear systems. PAMM 5(1), 705–708 (2005)CrossRefGoogle Scholar
  14. 14.
    Bekiaris, N., Meski, G.A., Radu, C.M., Morari, M.: Multiple steady states in homogeneous azeotropic distillation. Ind. Eng. Chem. Res. 32, 2023–2038 (1993)CrossRefGoogle Scholar
  15. 15.
    Boston, J.F., Sullivan, S.L.: A new class of solution methods for multicomponent, multistage separation processes. Can. J. Chem. Eng. 52, 52–63 (1974)CrossRefGoogle Scholar
  16. 16.
    Christensen, J.H.: The structuring of process optimization. AIChE J. 16(2), 177–184 (1970)CrossRefGoogle Scholar
  17. 17.
    Dassault Systèmes, AB: Dymola—Dynamic Modeling Laboratory. User Manual. Vol. 2., Ch. 8. Advanced Modelica Support (2014)Google Scholar
  18. 18.
    Davis, T.A.: Direct methods for sparse linear systems. In: Higham, N.J. (ed.) Fundamentals of Algorithms. SIAM, Philadelphia, USA (2006)Google Scholar
  19. 19.
    Dietmaier, P.: The Stewart-Gough platform of general geometry can have 40 real postures, pp 7–16. Springer, Netherlands, Dordrecht (1998)Google Scholar
  20. 20.
    Doedel, E.J., Wang, X.J., Fairgrieve, T.F.: AUTO94: Software for continuation and bifurcation problems in ordinary differential equations. Technical Report CRPC-95-1, Center for Research on Parallel Computing, California Institute of Technology, Pasadena CA 91125 (1995)Google Scholar
  21. 21.
    Doherty, M.F., Fidkowski, Z.T., Malone, M.F., Taylor, R.: Perry’s Chemical Engineers’ Handbook, 8th edn., p 33. McGraw-Hill Professional (2008). chapter 13Google Scholar
  22. 22.
    Dorigo, M., Birattari, M., Stützle, T: Ant colony optimization. IEEE Comput. Intell. Mag. 1(4), 28–39 (2006)CrossRefGoogle Scholar
  23. 23.
    Dorn, C., Güttinger, T E, Wells, G.J., Morari, M.: Stabilization of an unstable distillation column. Ind. Eng. Chem. Res. 37, 506–515 (1998)CrossRefGoogle Scholar
  24. 24.
    Duff, I.S., Erisman, A.M., Reid, J.K.: Direct methods for sparse matrices. Clarendon Press, Oxford (1986)Google Scholar
  25. 25.
    Dulmage, A.L., Mendelsohn, N.S.: Coverings of bipartite graphs. Can. J. Math. 10, 517–534 (1958)MathSciNetzbMATHCrossRefGoogle Scholar
  26. 26.
    Dulmage, A.L., Mendelsohn, N.S.: A structure theory of bipartite graphs of finite exterior dimension. Trans. Royal Soc. Can. Sec. 3(53), 1–13 (1959)zbMATHGoogle Scholar
  27. 27.
    Dulmage, A.L., Mendelsohn, N.S.: Two Algorithms for Bipartite Graphs. J. Soc. Ind. Appl. Math. 11, 183–194 (1963)MathSciNetzbMATHCrossRefGoogle Scholar
  28. 28.
    Eberhart, R., Kennedy, J.: A new optimizer using particle swarm theory. In: Proceedings of the Sixth International Symposium on Micro Machine and Human Science, 1995. MHS’95, pp 39–43. IEEE (2002)Google Scholar
  29. 29.
    Erisman, A.M., Grimes, R.G., Lewis, J.G., Poole, W.G.J.: A structurally stable modification of Hellerman-Rarick’s P 4 algorithm for reordering unsymmetric sparse matrices. SIAM J. Numer. Anal. 22, 369–385 (1985)MathSciNetzbMATHCrossRefGoogle Scholar
  30. 30.
    Faugère, J C, Lazard, D.: Combinatorial classes of parallel manipulators. Mech Mach. Theory 30(6), 765–776 (1995)CrossRefGoogle Scholar
  31. 31.
    Fletcher, R., Hall, J.A.J.: Ordering algorithms for irreducible sparse linear systems. Ann. Oper. Res. 43, 15–32 (1993)MathSciNetzbMATHCrossRefGoogle Scholar
  32. 32.
    Fourer, R.: Staircase matrices and systems. SIAM Rev. 26(1), 1–70 (1984)MathSciNetzbMATHCrossRefGoogle Scholar
  33. 33.
    Fourer, R., Gay, D.M., Kernighan, B.W.: AMPL: A Modeling Language for Mathematical Programming, Brooks/Cole, USA (2003)Google Scholar
  34. 34.
    Fritzson, P.: Principles of Object-Oriented Modeling and Simulation with Modelica 2.1. Wiley-IEEE Press (2004)Google Scholar
  35. 35.
    Golub, G.H., Van Loan, C.F.: Matrix computations, 3rd edn. The Johns Hopkins University Press, Baltimore, USA (1996)Google Scholar
  36. 36.
    gPROMS. Process Systems Enterprise Limited, gPROMS., [Online; accessed 17-November-2015] (2015)
  37. 37.
    Gupta, P.K., Westerberg, A.W., Hendry, J.E., Hughes, R.R.: Assigning output variables to equations using linear programming. AIChE J.ournal 20(2), 397–399 (1974)CrossRefGoogle Scholar
  38. 38.
    Güttinger, T E, Morari, M.: Comments on multiple steady states in homogeneous azeotropic distillation. Ind. Eng. Chem. Res. 35, 2816–2816 (1996)CrossRefGoogle Scholar
  39. 39.
    Güttinger, T E, Dorn, C., Morari, M.: Experimental study of multiple steady states in homogeneous azeotropic distillation. Ind. Eng. Chem. Res. 36, 794–802 (1997)CrossRefGoogle Scholar
  40. 40.
    Guzman, Y.A., Hasan, M.M.F., Floudas, C.A.: Computational comparison of convex underestimators for use in a branch-and-bound global optimization framework. In: Rassias, T.M., Floudas, C.A., Butenko, S. (eds.) Optimization in Science and Engineering, pp 229–246. Springer, New York, USA (2014)Google Scholar
  41. 41.
    Gwaltney, C.R., Lin, Y., Simoni, L.D., Stadtherr, M.A.: Interval methods for nonlinear equation solving applications. Wiley, Chichester, UK (2008)Google Scholar
  42. 42.
    Hellerman, E., Rarick, D.C.: Reinversion with preassigned pivot procedure. Math Programm. 1, 195–216 (1971)MathSciNetzbMATHCrossRefGoogle Scholar
  43. 43.
    Hellerman, E., Rarick, D.C.: The partitioned preassigned pivot procedure (P 4). In: Rose, D.J., Willoughby, R.A. (eds.) Sparse Matrices and their Applications, The IBM Research Symposia Series, pp 67–76. Springer, US (1972)Google Scholar
  44. 44.
    HSL: A collection of Fortran codes for large scale scientific computation. (2016)
  45. 45.
    Johnson, D.M., Dulmage, A.L., Mendelsohn, N.S.: Connectivity and reducibility of graphs. Can. J. Math. 14, 529–539 (1962)MathSciNetzbMATHCrossRefGoogle Scholar
  46. 46.
    Kannan, A., Joshi, M.R., Reddy, G.R., Shah, D.M.: Multiple-steady-states identification in homogeneous azeotropic distillation using a process simulator. Ind. Eng. Chem. Res. 44, 4386–4399 (2005)CrossRefGoogle Scholar
  47. 47.
    Kearfott, R.B.: Decomposition of arithmetic expressions to improve the behavior of interval iteration for nonlinear systems. Computing 47(2), 169–191 (1991)MathSciNetzbMATHCrossRefGoogle Scholar
  48. 48.
    Kröner, A, Marquardt, W., Gilles, E.: Getting around consistent initialization of DAE systems? Comput. Chem. Eng. 21(2), 145–158 (1997)CrossRefGoogle Scholar
  49. 49.
    Lazard, D.: On the representation of rigid-body motions and its application to generalized platform manipulators, pp 175–181. Springer, Netherlands, Dordrecht (1993)Google Scholar
  50. 50.
    Lewis, W.K., Matheson, G.L.: Studies in distillation. Ind. Eng. Chem. 24, 494–498 (1932)CrossRefGoogle Scholar
  51. 51.
    Malinen, I., Tanskanen, J.: Homotopy parameter bounding in increasing the robustness of homotopy continuation methods in multiplicity studies. Comput. Chem. Eng. 34(11), 1761–1774 (2010)CrossRefGoogle Scholar
  52. 52.
    Mattsson, S., Elmqvist, H., Otter, M.: Physical system modeling with Modelica. Control Eng. Pract. 6, 501—-510 (1998)CrossRefGoogle Scholar
  53. 53.
    Mitsos, A., Chachuat, B., Barton, P.I.: McCormick-based relaxations of algorithms. SIAM J. Optim. 20(2), 573–601 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  54. 54.
    Modelica: Modelica and the modelica association., [Online; accessed 10-October-2016] (2016)
  55. 55.
    Modelon, A.B.: User Guide, verison 1.17., [Online; accessed 10-October-2016] (2016)
  56. 56.
    Mourrain, B.: The 40 g̈eneric positions of a parallel robot. In: Proceedings of the 1993 International Symposium on Symbolic and Algebraic Computation, ACM, NY, USA, ISSAC ’93, pp. 173–182, doi: 10.1145/164081.164120 (1993)
  57. 57.
    Naphthali, L.M., Sandholm, D.P.: Multicomponent separation calculations by linearization. AIChE J. 17, 148–153 (1971)CrossRefGoogle Scholar
  58. 58.
    Neumaier, A.: Interval methods for systems of equations. Cambridge University Press, Cambridge (1990)zbMATHGoogle Scholar
  59. 59.
    Neumaier, A., Azmi, B.: LMBOPT – A limited memory method for bound-constrained optimization,, in preparation (2017)
  60. 60.
    Ochel, L.A., Bachmann, B.: Initialization of equation-based hybrid models within OpenModelica. In: 5th International Workshop on Equation-Based Object-Oriented Modeling Languages and Tools (University of Nottingham), pp 97–103. Linköping University Electronic Press; Linköpings universitet, Linköping Electronic Conference Proceedings, Nottingham, Uk (2013)Google Scholar
  61. 61.
    OpenModelica: Openmodelica user’s guide., [Online; accessed 10-October-2016] (2016)
  62. 62.
    De P Soares, R., Secchi, A.R.: EMSO: A new environment for modelling, simulation and optimisation. In: Computer Aided Chemical Engineering, vol. 14, pp 947–952. Elsevier (2003)Google Scholar
  63. 63.
    Pantelides, C.C.: The consistent initialization of differential-algebraic systems. SIAM J. Sci. Stat. Comput. 9(2), 213–231 (1988)MathSciNetzbMATHCrossRefGoogle Scholar
  64. 64.
    Petlyuk, F.B.: Distillation theory and its application to optimal design of separation units. Cambridge University Press, Cambridge, UK (2004)Google Scholar
  65. 65.
    Piela, P.C., Epperly, T.G., Westerberg, K.M., Westerberg, A.W.: ASCEND: An object-oriented computer environment for modeling and analysis: the modeling language. Comput. Chem. Eng. 15(1), 53–72 (1991)CrossRefGoogle Scholar
  66. 66.
    Pothen, A., Fan, C.J.: Computing the block triangular form of a sparse matrix. ACM Trans. Math. Softw. 16, 303–324 (1990)MathSciNetzbMATHCrossRefGoogle Scholar
  67. 67.
    Schichl, H., Neumaier, A.: Interval analysis on directed acyclic graphs for global optimization. J. Global Optim. 33, 541–562 (2005)MathSciNetzbMATHCrossRefGoogle Scholar
  68. 68.
    Scott, J.K., Stuber, M.D., Barton, P.I.: Generalized McCormick relaxations. J. Glob. Optim. 51(4), 569–606 (2011)MathSciNetzbMATHCrossRefGoogle Scholar
  69. 69.
    Shcherbina, O., Neumaier, A., Sam-Haroud, D., Vu, X.H., Nguyen, T.V.: Benchmarking global optimization and constraint satisfaction codes. In: Bliek, C., Jermann, C., Neumaier, A. (eds.) Global Optimization and Constraint Satisfaction, Lecture Notes in Computer Science., vol. 2861, pp 211–222. Springer, Berlin Heidelberg (2003)
  70. 70.
    Sielemann, M.: Device-oriented modeling and simulation in aircraft energy systems design. Dissertation, TU Hamburg, Hamburg (2012).  10.15480/882.1111
  71. 71.
    Sielemann, M., Schmitz, G.: A quantitative metric for robustness of nonlinear algebraic equation solvers. Math. Comput. Simul. 81(12), 2673–2687 (2011)MathSciNetzbMATHCrossRefGoogle Scholar
  72. 72.
    Sielemann, M., Casella, F., Otter, M.: Robustness of declarative modeling languages: improvements via probability-one homotopy. Simul. Modell. Pract. Theory 38, 38–57 (2013)CrossRefGoogle Scholar
  73. 73.
    Smith, L.: Improved placement of local solver launch points for large-scale global optimization. PhD thesis, Ottawa-Carleton Institute for Electrical and Computer Engineering (OCIECE). Carleton University, Ontario, Canada (2011)Google Scholar
  74. 74.
    Smith, L., Chinneck, J., Aitken, V.: Constraint consensus concentration for identifying disjoint feasible regions in nonlinear programmes. Optim. Methods Softw. 28(2), 339–363 (2013a)MathSciNetzbMATHCrossRefGoogle Scholar
  75. 75.
    Smith, L., Chinneck, J., Aitken, V.: Improved constraint consensus methods for seeking feasibility in nonlinear programs. Comput. Optim. Appl. 54(3), 555–578 (2013b)MathSciNetzbMATHCrossRefGoogle Scholar
  76. 76.
    Soares, R.P.: Finding all real solutions of nonlinear systems of equations with discontinuities by a modified affine arithmetic. Comput. Chem. Eng. 48, 48–57 (2013)CrossRefGoogle Scholar
  77. 77.
    Sommese, A.J., Wampler II, C.W.: The numerical solution of systems of polynomials arising in engineering and science. World Scientific (2005)Google Scholar
  78. 78.
    Stadtherr, M.A., Wood, E.S.: Sparse matrix methods for equation-based chemical process flowsheeting–I: Reordering phase. Comput. Chem. Eng. 8(1), 9–18 (1984a)CrossRefGoogle Scholar
  79. 79.
    Stadtherr, M.A., Wood, E.S.: Sparse matrix methods for equation-based chemical process flowsheeting–II: Numerical Phase. Comput. Chem. Eng. 8(1), 19–33 (1984b)CrossRefGoogle Scholar
  80. 80.
    Steward, D.V.: Partitioning and tearing systems of equations. J. Soc. Indust. Appl. Math. Ser. B Numer. Anal. 2(2), 345–365 (1965)MathSciNetzbMATHCrossRefGoogle Scholar
  81. 81.
    Thiele, E., Geddes, R.: Computation of distillation apparatus for hydrocarbon mixtures. Ind. Eng. Chem. 25, 289–295 (1933)CrossRefGoogle Scholar
  82. 82.
    Tiller, M.: Introduction to physical modeling with Modelica. Springer Science & Business Media (2001)Google Scholar
  83. 83.
    Ugray, Z., Lasdon, L., Plummer, J., Glover, F., Kelly, J., Martí, R: Scatter Search and Local NLP Solvers: A Multistart Framework for Global Optimization. INFORMS J. Comput. 19(3), 328–340 (2007). doi: 10.1287/ijoc.1060.0175
  84. 84.
    Unger, J., Kröner, A, Marquardt, W.: Structural analysis of differential-algebraic equation systems —– theory and applications. Comput. Chem. Eng. 19(8), 867–882 (1995)CrossRefGoogle Scholar
  85. 85.
    Vadapalli, A., Seader, J.D.: A generalized framework for computing bifurcation diagrams using process simulation programs. Comput. Chem. Eng. 25, 445–464 (2001)CrossRefGoogle Scholar
  86. 86.
    Verschelde, J.: Algorithm 795: PHCpack: A general-purpose solver for polynomial systems by homotopy continuation. ACM Trans. Math. Softw. 25(2), 251–276 (1999)zbMATHCrossRefGoogle Scholar
  87. 87.
    Verschelde, J.: Polynomial homotopy continuation with phcpack. ACM Commun. Comput. Algebra 44(3/4), 217–220 (2011)zbMATHGoogle Scholar
  88. 88.
    Verschelde, J.: The database of polynomial systems. (2016)
  89. 89.
    Vieira, R. Jr, E. B.: Direct methods for consistent initialization of DAE systems. Comput. Chem. Eng. 25(9–10), 1299–1311 (2001)Google Scholar
  90. 90.
    Vu, X.H., Schichl, H., Sam-Haroud, D.: Interval propagation and search on directed acyclic graphs for numerical constraint solving. J. Glob. Optim. 45(4), 499 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  91. 91.
    Wächter, A, Biegler, L.T.: On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Programm. 106, 25–57 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  92. 92.
    Wampler, C.W.: Forward displacement analysis of general six-in-parallel sps (Stewart) platform manipulators using soma coordinates. Mech. Mach. Theory 31 (3), 331–337 (1996)CrossRefGoogle Scholar
  93. 93.
    Westerberg, A.W., Edie, F.C.: Computer-aided design, Part 1 enhancing convergence properties by the choice of output variable assignments in the solution of sparse equation sets. Chem. Eng. J. 2, 9–16 (1971a)CrossRefGoogle Scholar
  94. 94.
    Westerberg AW, Edie FC: Computer-aided design, part 2 an approach to convergence and tearing in the solution of sparse equation sets. Chem. Eng. J. 2(1), 17–25 (1971b)Google Scholar
  95. 95.
    Wu, W., Reid, G.: Finding points on real solution components and applications to differential polynomial systems. In: Proceedings of the 38th International Symposium on Symbolic and Algebraic Computation, ACM, NY, USA, ISSAC ’13, pp 339–346 (2013)Google Scholar

Copyright information

© The Author(s) 2016

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  1. 1.Faculty of MathematicsUniversity of ViennaViennaAustria

Personalised recommendations