Abstract
In this and subsequent papers we will show that several algorithms for the isotonic regression problem may be viewed as active set methods. The active set approach provides a unifying framework for studying algorithms for isotonic regression, simplifies the exposition of existing algorithms and leads to several new efficient algorithms. We also investigate the computational complexity of several algorithms.
In this paper we consider the isotonic regression problem with respect to a complete order\(\begin{gathered} minimize\sum\limits_{i = 1}^n {w_i } (y_i - x_i )^2 \hfill \\ subject tox_1 \leqslant x_2 \leqslant \cdot \cdot \cdot \leqslant x_n \hfill \\ \end{gathered} \) where eachw i is strictly positive and eachy i is an arbitrary real number. We show that the Pool Adjacent Violators algorithm (due to Ayer et al., 1955; Miles, 1959; Kruskal, 1964), is a dual feasible active set method and that the Minimum Lower Set algorithm (due to Brunk et al., 1957) is a primal feasible active set method of computational complexity O(n 2). We present a new O(n) primal feasible active set algorithm. Finally we discuss Van Eeden's method and show that it is of worst-case exponential time complexity.
Similar content being viewed by others
References
M. Ayer, H.D. Brunk, G.M. Ewing, W.T. Reid and E. Silverman, “An empirical distribution function for sampling with incomplete information,”Annals of Mathematical Statistics 26 (1955) 641–647.
M.L. Balinski, “A competitive (dual) simplex method for the assignment problem,”Mathematical Programming 34 (1986) 125–141.
R.E. Barlow, D.J. Bartholomew, D.J. Bremner and H.D. Brunk,Statistical Inference Under Order Restrictions (Wiley, New York, 1972).
M.J. Best, “Equivalence of some quadratic programming algorithms,”Mathematical Programming 30 (1984) 71–87.
M.J. Best and N. Chakravarti, “An O(n 2) algorithm for a certain parametric quadratic programming problem,” Research Report CORR 87-23, University of Waterloo (Waterloo, Ont., 1987).
M.J. Best and K. Ritter, “A quadratic programming algorithm,”Zeitschrift für Operations Research 32 (1988) 271–297.
M.J. Best and K. Ritter,Quadratic Programming: Active Set Analysis and Computer Programs (Prentice-Hall, Englewood Cliffs, NJ, 1990), to appear.
H.D. Brunk, G.M. Ewing and W.R. Utz, “Minimizing integrals in certain classes of monotone functions,”Pacific Journal of Mathematics 7 (1957) 833–847.
R.L. Dykstra, “An isotonic regression algorithm,”Journal of Statistical Planning and Inference 5 (1981) 355–363.
C. Van Eeden, “Maximum likelihood estimation of ordered probabilities,”Proceedings of the Koninklijke Nederlandse Akademie van Wetenschappen (A) 59/Indagtiones Mathematicae 18 (1956) 444–455.
R. Fletcher,Practical Methods for Optimization Vol. II (Wiley, New York, 1980).
F. Gebhardt, “An algorithm for monotone regression with one or more independent variables,”Biometrika 57 (1970) 263–271.
P.E. Gill, W. Murray and M.H. Wright,Practical Optimization (Academic Press, London, 1981).
S.J. Grotzinger and C. Witzgall, “Projection onto order simplexes,”Applications of Mathematics and Optimization 12 (1984) 247–270.
M.S. Hung, “A polynomial simplex method for the assignment problem,”Operations Research 31 (1983) 595–600.
J.B. Kruskal, “Nonmetric multidimensional scaling,”Psychometrica 29 (1964) 115–129.
C.I.C. Lee, “The min-max algorithm and isotonic regression,”Annals of Statistics 11 (1983) 467–477.
M.L. Lenard, “A computational study of active set strategies in nonlinear programming with linear constraint,”Mathematical Programming 16 (1979) 81–97.
O.L. Mangasarian,Nonlinear Programming (McGraw-Hill, New York, 1969).
W.L. Maxwell and J.A. Muckstadt, “Establishing consistent and realistic reorder intervals in production-distribution systems,”Operations Research 33 (1985) 1316–1341.
R.E. Miles, “The complete amalgamation into blocks, by weighted means, of a finite set of real numbers,”Biometrika 46 (1959) 317–327.
J.B. Orlin, “On the simplex algorithm for networks and generalized networks,”Mathematical Programming Study 24 (1985) 166–178.
C.H. Papadimitrou and K. Steiglitz,Combinatorial Optimization: Algorithms and Complexity (Prentice-Hall, Englewood Cliffs, NJ, 1982).
R. Roundy, “A 98% effective lot-sizing rule for a multi-product multistage production/inventory system,”Mathematics of Operations Research 11 (1986) 699–727.
F.T. Wright, “Estimating strictly increasing functions,”Journal of the American Statistical Association 73 (1978) 636–639.
Author information
Authors and Affiliations
Additional information
This work was supported by the National Science and Engineering Research Council of Canada under Research Grant A8189 and an Ontario Graduate Scholarship.
Rights and permissions
About this article
Cite this article
Best, M.J., Chakravarti, N. Active set algorithms for isotonic regression; A unifying framework. Mathematical Programming 47, 425–439 (1990). https://doi.org/10.1007/BF01580873
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01580873