Statistics and Computing

, Volume 21, Issue 4, pp 613–632 | Cite as

Segmentation of the mean of heteroscedastic data via cross-validation

  • Sylvain Arlot
  • Alain CelisseEmail author


This paper tackles the problem of detecting abrupt changes in the mean of a heteroscedastic signal by model selection, without knowledge on the variations of the noise. A new family of change-point detection procedures is proposed, showing that cross-validation methods can be successful in the heteroscedastic framework, whereas most existing procedures are not robust to heteroscedasticity. The robustness to heteroscedasticity of the proposed procedures is supported by an extensive simulation study, together with recent partial theoretical results. An application to Comparative Genomic Hybridization (CGH) data is provided, showing that robustness to heteroscedasticity can indeed be required for their analysis.


Change-point detection Resampling Cross-validation Model selection Heteroscedastic data CGH profile segmentation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

11222_2010_9196_MOESM1_ESM.pdf (1.2 mb)
(PDF 1.20 MB)


  1. Abramovich, F., Benjamini, Y., Donoho, D.L., Johnstone, I.M.: Adapting to unknown sparsity by controlling the false discovery rate. Ann. Stat. 34(2), 584–653 (2006) zbMATHCrossRefMathSciNetGoogle Scholar
  2. Akaike, H.: Statistical predictor identification. Ann. Inst. Stat. Math. 22, 203–217 (1970) zbMATHCrossRefMathSciNetGoogle Scholar
  3. Akaike, H.: Information theory and an extension of the maximum likelihood principle. In: Second International Symposium on Information Theory, Tsahkadsor, 1971, pp. 267–281. Akadémiai Kiadó, Budapest (1973) Google Scholar
  4. Allen, D.M.: The relationship between variable selection and data augmentation and a method for prediction. Technometrics 16, 125–127 (1974) zbMATHCrossRefMathSciNetGoogle Scholar
  5. Arlot, S.: V-fold cross-validation improved: V-fold penalization. arXiv:0802.0566v2 (2008)
  6. Arlot, S.: Model selection by resampling penalization. Electron. J. Stat. 3, 557–624 (2009) (electronic) CrossRefMathSciNetGoogle Scholar
  7. Arlot, S.: Choosing a penalty for model selection in heteroscedastic regression. arXiv:0812.3141 (2010)
  8. Arlot, S., Celisse, A.: A survey of cross-validation procedures for model selection. Stat. Surv. 4, 40–79 (2010). doi: 10.1214/09-SS054 zbMATHCrossRefMathSciNetGoogle Scholar
  9. Arlot, S., Massart, P.: Data-driven calibration of penalties for least-squares regression. J. Mach. Learn. Res. 10, 245–279 (2009) (electronic) Google Scholar
  10. Baraud, Y.: Model selection for regression on a fixed design. Probab. Theory Relat. Fields 117(4), 467–493 (2000) zbMATHCrossRefMathSciNetGoogle Scholar
  11. Baraud, Y.: Model selection for regression on a random design. ESAIM Probab. Stat. 6, 127–146 (2002) (electronic) zbMATHCrossRefMathSciNetGoogle Scholar
  12. Baraud, Y., Giraud, C., Huet, S.: Gaussian model selection with an unknown variance. Ann. Stat. 37(2), 630–672 (2009) zbMATHCrossRefMathSciNetGoogle Scholar
  13. Barron, A., Birgé, L., Massart, P.: Risk bounds for model selection via penalization. Probab. Theory Relat. Fields 113, 301–413 (1999) zbMATHCrossRefGoogle Scholar
  14. Basseville, M., Nikiforov, I.V.: Detection of Abrupt Changes: Theory and Application. Prentice Hall Information and System Sciences Series. Englewood Cliffs, Prentice Hall (1993) Google Scholar
  15. Bellman, R.E., Dreyfus, S.E.: Applied Dynamic Programming. Princeton University Press, Princeton (1962) zbMATHGoogle Scholar
  16. Birgé, L., Massart, P.: From model selection to adaptive estimation. In: Pollard, D., Torgensen, E., Yang, G. (eds.) Festschrift for Lucien Le Cam: Research Papers in Probability and Statistics, pp. 55–87. Springer, New York (1997) Google Scholar
  17. Birgé, L., Massart, P.: Gaussian model selection. J. Eur. Math. Soc. 3(3), 203–268 (2001) zbMATHCrossRefMathSciNetGoogle Scholar
  18. Birgé, L., Massart, P.: Minimal penalties for Gaussian model selection. Probab. Theory Relat. Fields 138(1–2), 33–73 (2007) zbMATHCrossRefGoogle Scholar
  19. Brodsky, B.E., Darkhovsky, B.S.: Methods in Change-Point Problems. Kluwer Academic, Dordrecht (1993) Google Scholar
  20. Burman, P.: A comparative study of ordinary cross-validation, v-fold cross-validation and the repeated learning-testing methods. Biometrika 76(3), 503–514 (1989) zbMATHMathSciNetGoogle Scholar
  21. Burman, P., Nolan, D.: Data-dependent estimation of prediction functions. J. Time Ser. Anal. 13(3), 189–207 (1992) zbMATHCrossRefMathSciNetGoogle Scholar
  22. Celisse, A.: Model selection in density estimation via cross-validation. Technical Report (2008a). arXiv:0811.0802v2
  23. Celisse, A.: Model selection via cross-validation in density estimation, regression and change-points detection. PhD thesis, University Paris-Sud 11 (2008b).
  24. Celisse, A., Robin, S.: Nonparametric density estimation by exact leave-p-out cross-validation. Comput. Stat. Data Anal. 52(5), 2350–2368 (2008) zbMATHCrossRefMathSciNetGoogle Scholar
  25. Celisse, A., Robin, S.: A cross-validation based estimation of the proportion of true null hypotheses. J. Stat. Plan. Inference (2010). doi: 10.1016/j.jspi.2010.04.014 MathSciNetGoogle Scholar
  26. Chu, C.-K., Marron, J.S.: Comparison of two bandwidth selectors with dependent errors. Ann. Stat. 19(4), 1906–1918 (1991) zbMATHCrossRefMathSciNetGoogle Scholar
  27. Comte, F., Rozenholc, Y.: Adaptive estimation of mean and volatility functions in (auto-)regressive models. Stoch. Process. Appl. 97(1), 111–145 (2002) zbMATHCrossRefMathSciNetGoogle Scholar
  28. Dudoit, S., van der Laan, M.J.: Asymptotics of cross-validated risk estimation in estimator selection and performance assessment. Stat. Methodol. 2(2), 131–154 (2005) CrossRefMathSciNetGoogle Scholar
  29. Geisser, S.: A predictive approach to the random effect model. Biometrika 61(1), 101–107 (1974) zbMATHCrossRefMathSciNetGoogle Scholar
  30. Geisser, S.: The predictive sample reuse method with applications. J. Am. Stat. Assoc. 70, 320–328 (1975) zbMATHCrossRefGoogle Scholar
  31. Gendre, X.: Simultaneous estimation of the mean and the variance in heteroscedastic Gaussian regression. Electron. J. Stat. 2, 1345–1372 (2008) CrossRefMathSciNetGoogle Scholar
  32. Harchaoui, Z., Vallet, F., Lung-Yut-Fong, A., Cappé, O.: A regularized kernel-based approach to unsupervised audio segmentation. In: Proc. International Conference on Acoustics, Speech and Signal Processing, ICASSP, 2009 Google Scholar
  33. Kearns, M., Mansour, Y., Ng, A.Y., Ron, D.: An experimental and theoretical comparison of model selection methods. Mach. Learn. 7, 7–50 (1997) CrossRefGoogle Scholar
  34. Lachenbruch, P.A., Mickey, M.R.: Estimation of error rates in discriminant analysis. Technometrics 10, 1–11 (1968) CrossRefMathSciNetGoogle Scholar
  35. Lavielle, M.: Using penalized contrasts for the change-point problem. Signal Process. 85, 1501–1510 (2005) zbMATHCrossRefGoogle Scholar
  36. Lavielle, M., Teyssière, G.: Detection of multiple change-points in multivariate time series. Lith. Math. J. 46, 287–306 (2006) zbMATHCrossRefGoogle Scholar
  37. Lebarbier, É.: Detecting multiple change-points in the mean of a Gaussian process by model selection. Signal Process. 85, 717–736 (2005) zbMATHCrossRefGoogle Scholar
  38. Li, K.-C.: Asymptotic optimality for C p, C L, cross-validation and generalized cross-validation: discrete index set. Ann. Stat. 15(3), 958–975 (1987) zbMATHCrossRefGoogle Scholar
  39. Mallows, C.L.: Some comments on C p. Technometrics 15, 661–675 (1973) zbMATHCrossRefGoogle Scholar
  40. Massart, P.: Concentration Inequalities and Model Selection. Lecture Notes in Mathematics. Springer, Berlin (2007) zbMATHGoogle Scholar
  41. Miao, B.Q., Zhao, L.C.: On detection of change points when the number is unknown. Chin. J. Appl. Probab. Stat. 9(2), 138–145 (1993) zbMATHMathSciNetGoogle Scholar
  42. Opsomer, J., Wang, Y., Yang, Y.: Nonparametric regression with correlated errors. Stat. Sci. 16(2), 134–153 (2001) zbMATHCrossRefMathSciNetGoogle Scholar
  43. Picard, D.: Testing and estimating change-points in time series. Adv. Appl. Probab. 17(4), 841–867 (1985) zbMATHCrossRefMathSciNetGoogle Scholar
  44. Picard, F.: Process segmentation/clustering application to the analysis of array CGH data. PhD thesis, Université Paris-Sud 11, 2005.
  45. Picard, F., Robin, S., Lavielle, M., Vaisse, C., Daudin, J.-J.: A statistical approach for array CGH data analysis. BMC Bioinform. 27(6) (2005) (electronic access) Google Scholar
  46. Picard, F., Robin, S., Lebarbier, É., Daudin, J.-J.: A segmentation/clustering model for the analysis of array CGH data. Biometrics 63(3), 758–766 (2007) zbMATHCrossRefMathSciNetGoogle Scholar
  47. Rissanen, J.: A universal prior for integers and estimation by minimum description length. Ann. Stat. 11(2), 416–431 (1983) zbMATHCrossRefMathSciNetGoogle Scholar
  48. Schwarz, G.: Estimating the dimension of a model. Ann. Stat. 6(2), 461–464 (1978) zbMATHCrossRefGoogle Scholar
  49. Shao, J.: An asymptotic theory for linear model selection. Stat. Sinica 7, 221–264 (1997) zbMATHGoogle Scholar
  50. Shibata, R.: An optimal selection of regression variables. Biometrika 68(1), 45–54 (1981) zbMATHCrossRefMathSciNetGoogle Scholar
  51. Stone, C.J.: An asymptotically optimal window selection rule for kernel density estimates. Ann. Stat. 12(4), 1285–1297 (1984) zbMATHCrossRefGoogle Scholar
  52. Stone, M.: Cross-validatory choice and assessment of statistical predictions. J. R. Stat. Soc., Ser. B 36, 111–147 (1974) zbMATHGoogle Scholar
  53. Stone, M.: An asymptotic equivalence of choice of model by cross-validation and Akaike’s criterion. J. R. Stat. Soc. B 39(1), 44–47 (1977) zbMATHGoogle Scholar
  54. Tibshirani, R., Knight, K.: The covariance inflation criterion for adaptive model selection. J. R. Stat. Soc., Ser. B Stat. Methodol. 61(3), 529–546 (1999) zbMATHCrossRefMathSciNetGoogle Scholar
  55. Yang, Y.: Regression with multiple candidate model: selection or mixing? Stat. Sinica 13, 783–809 (2003) zbMATHGoogle Scholar
  56. Yang, Y.: Comparing learning methods for classification. Stat. Sinica 16, 635–657 (2006) zbMATHGoogle Scholar
  57. Yang, Y.: Consistency of cross-validation for comparing regression procedures. Ann. Stat. 35(6), 2450–2473 (2007) zbMATHCrossRefGoogle Scholar
  58. Yao, Y.-C.: Estimating the number of change-points via Schwarz’ criterion. Stat. Probab. Lett. 6(3), 181–189 (1988) zbMATHCrossRefGoogle Scholar
  59. Zhang, N.R., Siegmund, D.O.: Modified Bayes information criterion with application to the analysis of comparative genomic hybridization data. Biometrics 63, 22–32 (2007) zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.Willow Project-Team, Laboratoire d’Informatique de l’Ecole Normale Superieure, CNRS/ENS/INRIA UMR 8548Paris Cedex 13France
  2. 2.Laboratoire de Mathématique Paul Painlevé UMR 8524 CNRS, Université Lille 1Villeneuve d’Ascq CedexFrance

Personalised recommendations