Abstract
Rank estimators for linear regression models have been designed as robust estimators insensitive to outliers. The estimator is defined as a minimizer of Jaeckel’s dispersion function. We study algorithms for minimization of the function. Based on P-completeness arguments, we show that the minimization is computationally as hard as general linear programming. We also show that approximate algorithms with controlled error cannot be conceptually simpler since they can be converted into exact algorithms solving the same P-complete problem. Thus, approximate algorithms from literature, which do not use linear programming, cannot be guaranteed to approach the minimizer with a controlled error. Finally, we design two-stage methods combining advantages of both approaches: approximate algorithms with a simple and fast iteration step allow us to get close to the minimizer and exact algorithms, requiring LP techniques, then guarantee convergence and an exact result. We also present computational experiments illustrating the practical behavior of two-stage methods.
Supported by the Czech Science Foundation under project 19-02773S. Also the discussions with Jaromír Antoch are acknowledged.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Antoch, J., Ekblom, H.: Selected algorithms for robust M- and L-regression estimators. In: Dutter, R., Filzmoser, P., Gather, U., Rousseeuw, P.J. (eds.) Developments in Robust Statistics, pp. 32–49. Physica-Verlag, Heidelberg (2003). https://doi.org/10.1007/978-3-642-57338-5_3
Antoch, J., Ekblom, H.: Recursive robust regression computational aspects and comparison. Comput. Stat. Data Anal. 19(2), 115–128 (1995). https://doi.org/10.1016/0167-9473(93)E0050-E
Bindele, H.F., Abebe, A., Meyer, K.N.: General rank-based estimation for regression single index models. Ann. Inst. Stat. Math. 70(5), 1115–1146 (2017). https://doi.org/10.1007/s10463-017-0618-9
Černý, M., Hladík, M., Rada, M.: Walks on hyperplane arrangements and optimization of piecewise linear functions. arXiv:1912.12750 [math], December 2019
Černý, M., Rada, M., Antoch, J., Hladík, M.: A class of optimization problems motivated by rank estimators in robust regression. arXiv:1910.05826 [math], October 2019
Dutta, S., Datta, S.: Rank-based inference for covariate and group effects in clustered data in presence of informative intra-cluster group size. Stat. Med. 37(30), 4807–4822 (2018)
Greenlaw, R., Hoover, J.H., Ruzzo, W.L.: Limits to Parallel Computation: P-Completeness Theory. Oxford University Press, New York (1995)
Grötschel, M., Lovász, L., Schrijver, A.: Geometric Algorithms and Combinatorial Optimization. AC, vol. 2. Springer, Heidelberg (1993). https://doi.org/10.1007/978-3-642-78240-4
Hettmansperger, T., McKean, J.: Robust Nonparametric Statistical Methods, 2nd edn. CRC Press (2010). https://doi.org/10.1201/b10451
Hettmansperger, T.P., McKean, J.W.: A robust alternative based on ranks to least squares in analysing linear models. Technometrics 19(3), 275–284 (1977)
Jaeckel, L.A.: Estimating regression coefficients by minimizing the dispersion of the residuals. Ann. Math. Stat. 43, 1449–1458 (1972)
Jurečková, J.: Nonparametric estimate of regression coefficients. Ann. Math. Stat. 42, 1328–1338 (1971)
Kloke, J.D., McKean, J.W.: Rfit: rank-based estimation for linear models. R J. 4(2), 57 (2012). https://doi.org/10.32614/RJ-2012-014
O’Leary, D.P.: Robust regression computation using iteratively reweighted least squares. SIAM J. Matrix Anal. Appl. 11(3), 466–480 (1990). https://doi.org/10.1137/0611032
Osborne, M.R.: A finite algorithm for the rank regression problem. J. Appl. Probab. 19(A), 241–252 (1982). https://doi.org/10.2307/3213564
Roos, C., Terlaky, T., Vial, J.P.: Interior Point Methods for Linear Optimization. Springer, Boston (2005). https://doi.org/10.1007/b100325
Schrijver, A.: Theory of Linear and Integer Programming. Wiley, New York (2000)
Wolke, R., Schwetlick, H.: Iteratively reweighted least squares: algorithms, convergence analysis, and numerical comparisons. SIAM J. Sci. Stat. Comput. 9(5), 907–921 (1988). https://doi.org/10.1137/0909062
Wolke, R.: Iteratively reweighted least squares: a comparison of several single step algorithms for linear models. BIT Numer. Math. 32(3), 506–524 (1992). https://doi.org/10.1007/BF02074884
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Černý, M., Rada, M., Sokol, O. (2020). Rank Estimators for Robust Regression: Approximate Algorithms, Exact Algorithms and Two-Stage Methods. In: Huynh, VN., Entani, T., Jeenanunta, C., Inuiguchi, M., Yenradee, P. (eds) Integrated Uncertainty in Knowledge Modelling and Decision Making. IUKM 2020. Lecture Notes in Computer Science(), vol 12482. Springer, Cham. https://doi.org/10.1007/978-3-030-62509-2_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-62509-2_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-62508-5
Online ISBN: 978-3-030-62509-2
eBook Packages: Computer ScienceComputer Science (R0)