Summary
We consider the problem of minimizing the distance from a given n-dimensional vector to a set defined by constraints of the form x i ≤ x j. Such constraints induce a partial order of the components x i, which can be illustrated by an acyclic directed graph. This problem is also known as the isotonic regression (IR) problem. IR has important applications in statistics, operations research and signal processing, with most of them characterized by a very large value of n. For such large-scale problems, it is of great practical importance to develop algorithms whose complexity does not rise with n too rapidly. The existing optimization-based algorithms and statistical IR algorithms have either too high computational complexity or too low accuracy of the approximation to the optimal solution they generate. We introduce a new IR algorithm, which can be viewed as a generalization of the Pool-Adjacent-Violator (PAV) algorithm from completely to partially ordered data. Our algorithm combines both low computational complexity O(n 2) and high accuracy. This allows us to obtain sufficiently accurate solutions to IR problems with thousands of observations.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Acton, S.A., Bovik, A.C.: Nonlinear image estimation using piecewise and local image models. IEEE Transactions on Image Processing, 7, 979–991 (1998).
Ayer, M., Brunk, H.D., Ewing, G.M., Reid, W.T., Silverman, E.: An empirical distribution function for sampling with incomplete information. The Annals of Mathematical Statistics, 26, 641–647 (1955).
Barlow, R.E., Bartholomew, D.J., Bremner, J.M., Brunk, H.D.: Statistical inference under order restrictions. Wiley, New York (1972).
Best, M.J., Chakravarti, N.: Active set algorithms for isotonic regression: a unifying framework. Mathematical Programming, 47, 425–439 (1990).
Bril, G., Dykstra, R., Pillers, C., Robertson, T.: Algorithm AS 206, isotonic regression in two independent variables. Applied Statistics, 33, 352–357 (1984).
Brunk, H.D.: Maximumlikelihood estimates of monotone parameters. Annals of Mathematical Statistics, 26, 607–616 (1955).
Burdakov, O., Grimvall, A., Hussian, M.: A generalised PAV algorithm for monotonic regression in several variables. In: Antoch, J. (ed.) COMPSTAT, Proceedings in Computational Statistics, 16th Symposium Held in Prague, Czech Republic. Physica-Verlag, A Springer Company, Heidelberg New York, 761–767 (2004).
Cormen, T. H., Leiserson, C.E., Rivest, R. L., Stein, C.: Introduction to Algorithms. The MIT Press, Cambridge MA (2001).
De Simone, V., Marino, M., Toraldo, G.: Isotonic regression problems. In: Floudas, C.A., Pardalos, P.M. (eds) Encyclopedia of Optimization. Kluwer Academic Publishers, Dordrecht London Boston (2001).
Dykstra, R., Robertson, T.: An algorithm for isotonic regression for two or more independent variables. he Annals of Statistics, 10, 708–716 (1982).
Hanson, D.L., Pledger, G., Wright, F.T.: On consistency in monotonic regression. The Annals of Statistics, 1, 401–421 (1973).
Hussian, M., Grimvall, A., Burdakov, O., Sysoev, O.: Monotonic regression for the detection of temporal trends in environmental quality data. MATCH Commun. Math. Comput. Chem., 54, 535–550 (2005).
Lee, C.I.C.: The min-max algorithm and isotonic regression. The Annals of Statistics, 11, 467–477 (1983).
Maxwell, W.L., Muchstadt, J.A.: Establishing consistent and realistic reorder intervals in production-distribution systems. Operations Research, 33, 1316–1341 (1985).
Mukarjee, H.: Monotone nonparametric regression. The Annals of Statistics, 16, 741–750 (1988).
Mukarjee, H., Stern, H.: Feasible nonparametric estimation of multiargument monotone functions. Journal of the American Statistical Association, 425, 77–80 (1994).
Pardalos, P.M., Xue, G.: Algorithms for a class of isotonic regression problems. Algorithmica, 23, 211–222 (1999).
Restrepo, A., Bovik, A.C.: Locally monotonic regression. IEEE Transactions on Signal Processing, 41, 2796–2810 (1993).
Roundy, R.: A 98% effective lot-sizing rule for a multiproduct multistage production/inventory system. Mathematics of Operations Research, 11, 699–727 (1986).
Schell, M.J., Singh, B.: The reduced monotonic regression method. Journal of the American Statistical Association, 92, 128–135 (1997).
Strand, M.: Comparison of methods for monotone nonparametric multiple regression. Communications in Statistics-Simulation and Computation, 32, 165–178 (2003).
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer Science+Business Media, Inc.
About this chapter
Cite this chapter
Burdakov, O., Sysoev, O., Grimvall, A., Hussian, M. (2006). An O(n 2) Algorithm for Isotonic Regression. In: Di Pillo, G., Roma, M. (eds) Large-Scale Nonlinear Optimization. Nonconvex Optimization and Its Applications, vol 83. Springer, Boston, MA. https://doi.org/10.1007/0-387-30065-1_3
Download citation
DOI: https://doi.org/10.1007/0-387-30065-1_3
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-387-30063-4
Online ISBN: 978-0-387-30065-8
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)