Penalized estimation of flexible hidden Markov models for time series of counts
We propose an effectively nonparametric approach to fitting hidden Markov models to time series of counts, where the state-dependent distributions are estimated in a completely data-driven way without the need to specify a parametric family of distributions. To avoid overfitting, a roughness penalty based on higher-order differences between adjacent count probabilities is added to the likelihood, which is demonstrated to produce smooth state-dependent probability mass functions. The feasibility of the suggested approach is assessed in simulation experiments, and further illustrated in two real-data applications, where we model the distributions of (i) major earthquake counts and (ii) acceleration counts of an oceanic whitetip shark (Carcharhinus longimanus) over time. The proposed methodology is implemented in the accompanying R package countHMM, which is available on CRAN.
KeywordsCount data Nonparametric statistics Penalized likelihood Smoothing parameter selection State-space model Time series modeling
The authors are grateful to the reviewer for carefully reading the article and for the comments, which greatly improved the article. The authors also wish to thank Yannis Papastamatiou and Yuuki Watanabe for providing the oceanic whitetip shark data.
- 1.Adam, T.: countHMM: Penalized estimation of flexible hidden Markov models for time series of counts. R package, version 0.1.0. (2019). https://cran.r-project.org/package=countHMM
- 5.Baum, J., Medina, E., Musick, J.A., Smale, M.: Carcharhinus longimanus. The IUCN Red List of threatened species 2015, 2019 (2015). https://doi.org/10.2305/IUCN.UK.2015.RLTS.T39374A85699641.en. Downloaded on May 23
- 12.Howey-Jordan, L.A., Brooks, E.J., Abercrombie, D.L., Jordan, L.K.B., Brooks, A., Williams, S., Gospodarczyk, E., Chapman, D.D.: Complex movements, philopatry and expanded depth range of a severely threatened pelagic shark, the oceanic whitetip (Carcharhinus longimanus) in the western North Atlantic. PLoS One 8(2), e56588 (2013)CrossRefGoogle Scholar
- 29.R Core Team. R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna (2017). https://www.r-project.org
- 34.Turner, R.: hmm.discnp: Hidden Markov models with discrete non-parametric observation distributions. R package, version 2.1-5 (2018). https://cran.r-project.org/package=hmm.discnp
- 35.Visser, I., Raijmakers, M.E.J., Molenaar, P.: Fitting hidden Markov models to psychological data. Sci. Program. 10(3), 185–199 (2002)Google Scholar