Advertisement

Computing the halfspace depth with multiple try algorithm and simulated annealing algorithm

  • Wei ShaoEmail author
  • Yijun Zuo
Original paper
  • 22 Downloads

Abstract

The halfspace depth is a powerful tool for the nonparametric multivariate analysis. However, its computation is very challenging for it involves the infimum over infinitely many directional vectors. The exact computation of the halfspace depth is a NP-hard problem if both sample size n and dimension d are parts of input. The approximate algorithms often can not get accurate (exact) results in high dimensional cases within limited time. In this paper, we propose a new general stochastic optimization algorithm, which is the combination of simulated annealing and the multiple try Metropolis algorithm. As a by product of the new algorithm, it is then successfully applied to the computation of the halfspace depth of data sets which are not necessarily in general position. The simulation and real data examples indicate that the new algorithm is highly competitive to, especially in the high dimension and large sample cases, other (exact and approximate) algorithms, including the simulated annealing and the quasi-Newton method and so on, both in accuracy and efficiency.

Keywords

Half-space depth computation Multiple try Metropolis Simulated annealing Markov Chain Monte Carlo (MCMC) 

Notes

Acknowledgements

The authors would like to thank the Editor, Associate Editor and reviewers for their insightful comments and constructive suggestions, which led to distinctive improvements in this paper. The first author’s research was partially supported by the National Natural Science Foundation of China (11501320, 71471101, 11426143), the Natural Science Foundation of Shandong Province (ZR2014AP008) and the Natural Science Foundation of Qufu Normal University (bsqd20130114).

References

  1. Byrd RH, Liu P, Nocedal J, Zhu C (1995) A limited memory algorithm for bound constrained optimization. SIAM J Sci Comput 16:1190–1208MathSciNetCrossRefzbMATHGoogle Scholar
  2. Casarin R, Craiu R, Leisen R (2013) Interacting multiple try algorithms with different proposal distributions. Stat Comput 23:185–200MathSciNetCrossRefzbMATHGoogle Scholar
  3. Chen D, Morin P, Wagner U (2013) Absolute approximation of Tukey depth: theory and experiments. Comput Geom 46:566–573MathSciNetCrossRefzbMATHGoogle Scholar
  4. Cuesta-Albertos JA, Nieto-Reyes A (2008) The random Tukey depth. Comput Stat Data Anal 52:4979–4988MathSciNetCrossRefzbMATHGoogle Scholar
  5. Dutta S, Ghosh AK (2012) On robust classification using projection depth. Ann Inst Stat Math 64:657–676MathSciNetCrossRefzbMATHGoogle Scholar
  6. Dyckerhoff R, Mozharovskyi P (2016) Exact computation of the halfspace depth. Comput Stat Data Anal 98:19–30MathSciNetCrossRefzbMATHGoogle Scholar
  7. Geman S, Geman D (1984) Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE Trans Pattern Anal Mach Intell 6:721–741CrossRefzbMATHGoogle Scholar
  8. Hastings WK (1970) Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57:97–109MathSciNetCrossRefzbMATHGoogle Scholar
  9. Hestenes MR, Stiefel E (1952) Methods of conjugate gradients for solving linear systems. J Res Natl Bur Stand 49:409–436MathSciNetCrossRefzbMATHGoogle Scholar
  10. Kirkpatrick S, Gelatt CD, Vecchi MP (1983) Optimization by simulated annealing. Science 220:671–680MathSciNetCrossRefzbMATHGoogle Scholar
  11. Lange T, Mosler K, Mozharovskyi P (2014) Fast nonparametric classification based on data depth. Stat Pap 55:49–69MathSciNetCrossRefzbMATHGoogle Scholar
  12. Liang F, Liu C, Carroll RJ (2011) Advanced markov chain Monte Carlo methods: learning from past samples. Wiley, West SussexzbMATHGoogle Scholar
  13. Liang F, Cheng Y, Lin G (2014) Simulated stochastic approximation annealing for global optimization with a square-root cooling schedule. J Am Stat Assoc 109:847–863MathSciNetCrossRefzbMATHGoogle Scholar
  14. Liu JS (2001) Monte Carlo strategies in scientific computing. Springer, New YorkzbMATHGoogle Scholar
  15. Liu X (2017) Fast implementation of the Tukey depth. Comput Stat 32:1395–1410MathSciNetCrossRefzbMATHGoogle Scholar
  16. Liu JS, Liang F, Wong WH (2000) The multiple-try and local optimization in Metropolis sampling. J Am Stat Assoc 95:121–134MathSciNetCrossRefzbMATHGoogle Scholar
  17. Liu X, Zuo Y (2014) Computing halfspace depth and regression depth. Commun Stat Simul Comput 43:969–985MathSciNetCrossRefzbMATHGoogle Scholar
  18. Liu X, Zuo Y, Wang Q (2017) Finite sample breakdown point of Tukey’s halfspace median. Sci China Math 60(5):861–874MathSciNetCrossRefzbMATHGoogle Scholar
  19. Martino L, Olmo VPD, Read J (2012) A multi-point Metropolis scheme with generic weight functions. Stat Probab Lett 82:1445–1453MathSciNetCrossRefzbMATHGoogle Scholar
  20. Metropolis N, Rosenbluth AW, Rosenbluth MN, Teller AH, Teller E (1953) Equations of state calculations by fast computing machines. J Chem Phys 21:1087–1092CrossRefGoogle Scholar
  21. Nelder JA, Mead R (1965) A simplex method for function minimization. Comput J 7:308–313MathSciNetCrossRefzbMATHGoogle Scholar
  22. Pandolfi S, Bartolucci F, Friel N (2010) A generalization of the multiple-try Metropolis algorithm for Bayesian estimation and model selection. J Mach Learn Res 9:581–588Google Scholar
  23. R Core Team (2016) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna. https://www.R-project.org/
  24. Rousseeuw PJ, Hubert M (1999) Regression depth. J Am Stat Assoc 94:388–402MathSciNetCrossRefzbMATHGoogle Scholar
  25. Rousseeuw PJ, Struyf A (1998) Computing location depth and regression depth in higher dimensions. Stat Comput 8:193–203CrossRefGoogle Scholar
  26. Ruszczynski A (2006) Nonlinear optimization. Princeton University Press, PrincetonzbMATHGoogle Scholar
  27. Shao W, Guo G (2018) Multiple-try simulated annealing algorithm for global optimization. Math Probl Eng 2018:1–11MathSciNetGoogle Scholar
  28. Shao W, Zuo Y (2012) Simulated annealing for higher dimensional projection depth. Comput Stat Data Anal 56:4026–4036MathSciNetCrossRefzbMATHGoogle Scholar
  29. Shao W, Guo G, Zhao G, Meng F (2014) Simulated annealing for the bounds of Kendall’s tau and Spearman’s rho. J Stat Comput Simul 84:2688–2699MathSciNetCrossRefGoogle Scholar
  30. Tukey JW (1975) Mathematrics and the picturing of data. Proc Int Cong Math 2:523–531Google Scholar
  31. Zuo Y, Serfling R (2000) General notions of statistical depth function. Ann Stat 28:461–482MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of ManagementQufu Normal UniversityRizhaoChina
  2. 2.School of StatisticsQufu Normal UniversityQufuChina
  3. 3.Department of Statistics and ProbabilityMichigan State UniversityEast LansingUSA

Personalised recommendations