A weighted \(\chi ^2\) test to detect the presence of a major change point in non-stationary Markov chains

Abstract

The problem of detecting a major change point in a stochastic process is often of interest in applications, in particular when the effects of modifications of some external variables, on the process itself, must be identified. We here propose a modification of the classical Pearson \(\chi ^2\) test to detect the presence of such major change point in the transition probabilities of an inhomogeneous discrete time Markov Chain, taking values in a finite space. The test can be applied also in presence of big identically distributed samples of the Markov Chain under study, which might not be necessarily independent. The test is based on the maximum likelihood estimate of the size of the ’right’ experimental unit, i.e. the units that must be aggregated to filter out the small scale variability of the transition probabilities. We here apply our test both to simulated data and to a real dataset, to study the impact, on farmland uses, of the new Common Agricultural Policy, which entered into force in EU in 2015.

This is a preview of subscription content, access via your institution.

Fig. 1

References

  1. Anderson TW, Goodman LA (1957) Statistical Inference about Markov Chains. Ann Math Stat 28(1):89–110. https://doi.org/10.1214/aoms/1177707039

    MathSciNet  Article  MATH  Google Scholar 

  2. Bertoni D et al (2018) Farmland use transitions after the CAP greening: a preliminary analysis using Markov chains approach. Land Use Policy 79:789–800. https://doi.org/10.1016/j.landusepol.2018.09.012

    Article  Google Scholar 

  3. Cortignani R, Severini S, Dono G (2017) Complying with greening practices in the new CAP direct payments: an application on Italian specialized arable farms. Land Use Policy 61:265–275. https://doi.org/10.1016/j.landusepol.2016.11.026

    Article  Google Scholar 

  4. Eisinger RD, Chen Y (2017) Sampling for conditional inference on contingency tables. J Comput Graph Stat 26(1):79–87. https://doi.org/10.1080/10618600.2016.1153478

    MathSciNet  Article  Google Scholar 

  5. Fisz M (1963) Probability theory and mathematical statistics. Wiley, New York

    Google Scholar 

  6. Fu X, Wang X, Yang YJ (2018) Deriving suitability factors for CA–Markov land use simulation model based on local historical data. J Environ Manag 206:10–19. https://doi.org/10.1016/j.jenvman.2017.10.012

    Article  Google Scholar 

  7. Gregorio AD, Iacus SM (2008) Least squares volatility change point estimation for partially observed diffusion processes. Commun Stat Theory Methods 37(15):2342–2357. https://doi.org/10.1080/03610920801919692

    MathSciNet  Article  MATH  Google Scholar 

  8. Iacus SM, Yoshida N (2012) Estimation for the change point of volatility in a stochastic differential equation. Stoch Process Their Appl 122(3):1068–1092. https://doi.org/10.1016/j.spa.2011.11.005

    MathSciNet  Article  MATH  Google Scholar 

  9. Knoke D, Bohrnstedt GW, Mee AP (2002) Statistics for social data analysis. F.E. Peacock Publishers, Itasca. ISBN: 0-87581-448-4

    Google Scholar 

  10. Micheletti A, Morale D, Rapati D, Nolli P (2010) A stochastic model for simulation and forecasting of emergencies in the area of Milano. In: 2010 IEEE Workshop on Health Care Management (WHCM). IEEE, Venice, Italy. https://doi.org/10.1109/WHCM.2010.5441259

  11. Micheletti A, Nakagawa J, Alessi AA, Morale D, Villa E (2016) A germ-grain model applied to the morphological study of dual phase steel. J Math Ind 6:12. https://doi.org/10.1186/s13362-016-0033-5

    MathSciNet  Article  MATH  Google Scholar 

  12. Moreno E, Casella G, Garcia-Ferrer A (2005) An objective Bayesian analysis of the change point problem. Stoch Environ Res Risk Assess 19:191–204. https://doi.org/10.1007/s00477-004-0224-2

    Article  Google Scholar 

  13. Olsen LR, Chaudhuri P, Godtliebsen F (2008) Multiscale spectral analysis for detecting short and long range change points in time series. Comput Stat Data Anal 52:3310–3330. https://doi.org/10.1016/j.csda.2007.10.027

    MathSciNet  Article  MATH  Google Scholar 

  14. Rancoita PMV, Giusti A, Micheletti A (2011) Intensity estimation of stationary fibre processes from digital images with a learned detector. Image Anal Stereology 30(3):167

    MathSciNet  Article  Google Scholar 

  15. Sang L et al (2011) Simulation of land use spatial pattern of towns and villages based on CA–Markov model. Math Comput Model 54(3):938–943. https://doi.org/10.1016/j.mcm.2010.11.019

    Article  Google Scholar 

  16. Schütz N, Holschneider M (2011) Detection of trend changes in time series using Bayesian inference. Phys Rev E 84(2):021120. https://doi.org/10.1103/PhysRevE.84.021120

    Article  Google Scholar 

  17. Solazzo R, Donati M, Arfini F (2015) Impact assessment of greening and the issue of nitrogen-fixing crops: evidence from northern Italy. Outlook Agric 44(3):215–222. https://doi.org/10.5367/oa.2015.0215

    Article  Google Scholar 

Download references

Acknowledgements

This study has been supported by Fondazione Cariplo, within the research project “Evaluation of CAP 2015–2020 and taking action—CAPTION” (Project Id: 2017-2513). Funding has also been received from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie project BIGMATH, Grant Agreement No 812912.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Alessandra Micheletti.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Micheletti, A., Aletti, G., Ferrandi, G. et al. A weighted \(\chi ^2\) test to detect the presence of a major change point in non-stationary Markov chains. Stat Methods Appl 29, 899–912 (2020). https://doi.org/10.1007/s10260-020-00510-0

Download citation

Keywords

  • Weighted \(\chi ^2\) test
  • Inhomogeneous discrete time Markov chains
  • Nonparametric inference