Mathematical Programming

, Volume 47, Issue 1–3, pp 425–439

Active set algorithms for isotonic regression; A unifying framework

  • Michael J. Best
  • Nilotpal Chakravarti


In this and subsequent papers we will show that several algorithms for the isotonic regression problem may be viewed as active set methods. The active set approach provides a unifying framework for studying algorithms for isotonic regression, simplifies the exposition of existing algorithms and leads to several new efficient algorithms. We also investigate the computational complexity of several algorithms.

In this paper we consider the isotonic regression problem with respect to a complete order\(\begin{gathered} minimize\sum\limits_{i = 1}^n {w_i } (y_i - x_i )^2 \hfill \\ subject tox_1 \leqslant x_2 \leqslant \cdot \cdot \cdot \leqslant x_n \hfill \\ \end{gathered} \) where eachwi is strictly positive and eachyi is an arbitrary real number. We show that the Pool Adjacent Violators algorithm (due to Ayer et al., 1955; Miles, 1959; Kruskal, 1964), is a dual feasible active set method and that the Minimum Lower Set algorithm (due to Brunk et al., 1957) is a primal feasible active set method of computational complexity O(n2). We present a new O(n) primal feasible active set algorithm. Finally we discuss Van Eeden's method and show that it is of worst-case exponential time complexity.

Key words

Isotonic regression active sets 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    M. Ayer, H.D. Brunk, G.M. Ewing, W.T. Reid and E. Silverman, “An empirical distribution function for sampling with incomplete information,”Annals of Mathematical Statistics 26 (1955) 641–647.Google Scholar
  2. [2]
    M.L. Balinski, “A competitive (dual) simplex method for the assignment problem,”Mathematical Programming 34 (1986) 125–141.Google Scholar
  3. [3]
    R.E. Barlow, D.J. Bartholomew, D.J. Bremner and H.D. Brunk,Statistical Inference Under Order Restrictions (Wiley, New York, 1972).Google Scholar
  4. [4]
    M.J. Best, “Equivalence of some quadratic programming algorithms,”Mathematical Programming 30 (1984) 71–87.Google Scholar
  5. [5]
    M.J. Best and N. Chakravarti, “An O(n 2) algorithm for a certain parametric quadratic programming problem,” Research Report CORR 87-23, University of Waterloo (Waterloo, Ont., 1987).Google Scholar
  6. [6]
    M.J. Best and K. Ritter, “A quadratic programming algorithm,”Zeitschrift für Operations Research 32 (1988) 271–297.Google Scholar
  7. [7]
    M.J. Best and K. Ritter,Quadratic Programming: Active Set Analysis and Computer Programs (Prentice-Hall, Englewood Cliffs, NJ, 1990), to appear.Google Scholar
  8. [8]
    H.D. Brunk, G.M. Ewing and W.R. Utz, “Minimizing integrals in certain classes of monotone functions,”Pacific Journal of Mathematics 7 (1957) 833–847.Google Scholar
  9. [9]
    R.L. Dykstra, “An isotonic regression algorithm,”Journal of Statistical Planning and Inference 5 (1981) 355–363.Google Scholar
  10. [10]
    C. Van Eeden, “Maximum likelihood estimation of ordered probabilities,”Proceedings of the Koninklijke Nederlandse Akademie van Wetenschappen (A) 59/Indagtiones Mathematicae 18 (1956) 444–455.Google Scholar
  11. [11]
    R. Fletcher,Practical Methods for Optimization Vol. II (Wiley, New York, 1980).Google Scholar
  12. [12]
    F. Gebhardt, “An algorithm for monotone regression with one or more independent variables,”Biometrika 57 (1970) 263–271.Google Scholar
  13. [13]
    P.E. Gill, W. Murray and M.H. Wright,Practical Optimization (Academic Press, London, 1981).Google Scholar
  14. [14]
    S.J. Grotzinger and C. Witzgall, “Projection onto order simplexes,”Applications of Mathematics and Optimization 12 (1984) 247–270.Google Scholar
  15. [15]
    M.S. Hung, “A polynomial simplex method for the assignment problem,”Operations Research 31 (1983) 595–600.Google Scholar
  16. [16]
    J.B. Kruskal, “Nonmetric multidimensional scaling,”Psychometrica 29 (1964) 115–129.Google Scholar
  17. [17]
    C.I.C. Lee, “The min-max algorithm and isotonic regression,”Annals of Statistics 11 (1983) 467–477.Google Scholar
  18. [18]
    M.L. Lenard, “A computational study of active set strategies in nonlinear programming with linear constraint,”Mathematical Programming 16 (1979) 81–97.Google Scholar
  19. [19]
    O.L. Mangasarian,Nonlinear Programming (McGraw-Hill, New York, 1969).Google Scholar
  20. [20]
    W.L. Maxwell and J.A. Muckstadt, “Establishing consistent and realistic reorder intervals in production-distribution systems,”Operations Research 33 (1985) 1316–1341.Google Scholar
  21. [21]
    R.E. Miles, “The complete amalgamation into blocks, by weighted means, of a finite set of real numbers,”Biometrika 46 (1959) 317–327.Google Scholar
  22. [22]
    J.B. Orlin, “On the simplex algorithm for networks and generalized networks,”Mathematical Programming Study 24 (1985) 166–178.Google Scholar
  23. [23]
    C.H. Papadimitrou and K. Steiglitz,Combinatorial Optimization: Algorithms and Complexity (Prentice-Hall, Englewood Cliffs, NJ, 1982).Google Scholar
  24. [24]
    R. Roundy, “A 98% effective lot-sizing rule for a multi-product multistage production/inventory system,”Mathematics of Operations Research 11 (1986) 699–727.Google Scholar
  25. [25]
    F.T. Wright, “Estimating strictly increasing functions,”Journal of the American Statistical Association 73 (1978) 636–639.Google Scholar

Copyright information

© The Mathematical Programming Society, Inc. 1990

Authors and Affiliations

  • Michael J. Best
    • 1
  • Nilotpal Chakravarti
    • 2
  1. 1.Department of Combinatorics and OptimizationUniversity of WaterlooWaterlooCanada
  2. 2.Department of Mathematical SciencesNorthern Illinois UniversityDeKalbUSA

Personalised recommendations