Active set algorithms for isotonic regression; A unifying framework
 Michael J. Best,
 Nilotpal Chakravarti
 … show all 2 hide
Rent the article at a discount
Rent now* Final gross prices may vary according to local VAT.
Get AccessAbstract
In this and subsequent papers we will show that several algorithms for the isotonic regression problem may be viewed as active set methods. The active set approach provides a unifying framework for studying algorithms for isotonic regression, simplifies the exposition of existing algorithms and leads to several new efficient algorithms. We also investigate the computational complexity of several algorithms.
In this paper we consider the isotonic regression problem with respect to a complete order \(\begin{gathered} minimize\sum\limits_{i = 1}^n {w_i } (y_i  x_i )^2 \\ subject tox_1 \leqslant x_2 \leqslant \cdot \cdot \cdot \leqslant x_n \\ \end{gathered} \) where eachw _{ i } is strictly positive and eachy _{ i } is an arbitrary real number. We show that the Pool Adjacent Violators algorithm (due to Ayer et al., 1955; Miles, 1959; Kruskal, 1964), is a dual feasible active set method and that the Minimum Lower Set algorithm (due to Brunk et al., 1957) is a primal feasible active set method of computational complexity O(n ^{2}). We present a new O(n) primal feasible active set algorithm. Finally we discuss Van Eeden's method and show that it is of worstcase exponential time complexity.
 M. Ayer, H.D. Brunk, G.M. Ewing, W.T. Reid and E. Silverman, “An empirical distribution function for sampling with incomplete information,”Annals of Mathematical Statistics 26 (1955) 641–647.
 M.L. Balinski, “A competitive (dual) simplex method for the assignment problem,”Mathematical Programming 34 (1986) 125–141.
 R.E. Barlow, D.J. Bartholomew, D.J. Bremner and H.D. Brunk,Statistical Inference Under Order Restrictions (Wiley, New York, 1972).
 M.J. Best, “Equivalence of some quadratic programming algorithms,”Mathematical Programming 30 (1984) 71–87.
 M.J. Best and N. Chakravarti, “An O(n ^{2}) algorithm for a certain parametric quadratic programming problem,” Research Report CORR 8723, University of Waterloo (Waterloo, Ont., 1987).
 M.J. Best and K. Ritter, “A quadratic programming algorithm,”Zeitschrift für Operations Research 32 (1988) 271–297.
 M.J. Best and K. Ritter,Quadratic Programming: Active Set Analysis and Computer Programs (PrenticeHall, Englewood Cliffs, NJ, 1990), to appear.
 H.D. Brunk, G.M. Ewing and W.R. Utz, “Minimizing integrals in certain classes of monotone functions,”Pacific Journal of Mathematics 7 (1957) 833–847.
 R.L. Dykstra, “An isotonic regression algorithm,”Journal of Statistical Planning and Inference 5 (1981) 355–363.
 C. Van Eeden, “Maximum likelihood estimation of ordered probabilities,”Proceedings of the Koninklijke Nederlandse Akademie van Wetenschappen (A) 59/Indagtiones Mathematicae 18 (1956) 444–455.
 R. Fletcher,Practical Methods for Optimization Vol. II (Wiley, New York, 1980).
 F. Gebhardt, “An algorithm for monotone regression with one or more independent variables,”Biometrika 57 (1970) 263–271.
 P.E. Gill, W. Murray and M.H. Wright,Practical Optimization (Academic Press, London, 1981).
 S.J. Grotzinger and C. Witzgall, “Projection onto order simplexes,”Applications of Mathematics and Optimization 12 (1984) 247–270.
 M.S. Hung, “A polynomial simplex method for the assignment problem,”Operations Research 31 (1983) 595–600.
 J.B. Kruskal, “Nonmetric multidimensional scaling,”Psychometrica 29 (1964) 115–129.
 C.I.C. Lee, “The minmax algorithm and isotonic regression,”Annals of Statistics 11 (1983) 467–477.
 M.L. Lenard, “A computational study of active set strategies in nonlinear programming with linear constraint,”Mathematical Programming 16 (1979) 81–97.
 O.L. Mangasarian,Nonlinear Programming (McGrawHill, New York, 1969).
 W.L. Maxwell and J.A. Muckstadt, “Establishing consistent and realistic reorder intervals in productiondistribution systems,”Operations Research 33 (1985) 1316–1341.
 R.E. Miles, “The complete amalgamation into blocks, by weighted means, of a finite set of real numbers,”Biometrika 46 (1959) 317–327.
 J.B. Orlin, “On the simplex algorithm for networks and generalized networks,”Mathematical Programming Study 24 (1985) 166–178.
 C.H. Papadimitrou and K. Steiglitz,Combinatorial Optimization: Algorithms and Complexity (PrenticeHall, Englewood Cliffs, NJ, 1982).
 R. Roundy, “A 98% effective lotsizing rule for a multiproduct multistage production/inventory system,”Mathematics of Operations Research 11 (1986) 699–727.
 F.T. Wright, “Estimating strictly increasing functions,”Journal of the American Statistical Association 73 (1978) 636–639.
 Title
 Active set algorithms for isotonic regression; A unifying framework
 Journal

Mathematical Programming
Volume 47, Issue 13 , pp 425439
 Cover Date
 19900501
 DOI
 10.1007/BF01580873
 Print ISSN
 00255610
 Online ISSN
 14364646
 Publisher
 SpringerVerlag
 Additional Links
 Topics
 Keywords

 Isotonic regression
 active sets
 Industry Sectors
 Authors

 Michael J. Best ^{(1)}
 Nilotpal Chakravarti ^{(2)}
 Author Affiliations

 1. Department of Combinatorics and Optimization, University of Waterloo, N2L 3G1, Waterloo, Ont., Canada
 2. Department of Mathematical Sciences, Northern Illinois University, 60115, DeKalb, IL, USA