Hierarchical Continuous Time Hidden Markov Model, with Application in Zero-Inflated Accelerometer Data

  • Zekun XuEmail author
  • Eric B. Laber
  • Ana-Maria Staicu
Part of the Emerging Topics in Statistics and Biostatistics book series (ETSB)


Wearable devices including accelerometers are increasingly being used to collect high-frequency human activity data in situ. There is tremendous potential to use such data to inform medical decision making and public health policies. However, modeling such data is challenging as they are high-dimensional, heterogeneous, and subject to informative missingness, e.g., zero readings when the device is removed by the participant. We propose a flexible and extensible continuous-time hidden Markov model to extract meaningful activity patterns from human accelerometer data. To facilitate estimation with massive data we derive an efficient learning algorithm that exploits the hierarchical structure of the parameters indexing the proposed model. We also propose a bootstrap procedure for interval estimation. The proposed methods are illustrated using data from the 2003–2004 and 2005–2006 National Health and Nutrition Examination Survey.


Continuous-time hidden Markov model Consensus optimization Accelerometer data 


  1. 1.
    Albert, A. (1962). Estimating the infinitesimal generator of a continuous time, finite state Markov process. The Annals of Mathematical Statistics, 33(2), 727–753.MathSciNetCrossRefGoogle Scholar
  2. 2.
    Altman, R. M. (2007). Mixed hidden Markov models: An extension of the hidden Markov model to the longitudinal data setting. Journal of the American Statistical Association, 102(477), 201–210.MathSciNetCrossRefGoogle Scholar
  3. 3.
    Bartalesi, R., Lorussi, F., Tesconi, M., Tognetti, A., Zupone, G., & De Rossi, D. (2005). Wearable kinesthetic system for capturing and classifying upper limb gesture. In Eurohaptics Conference, 2005 and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2005. World Haptics 2005. First Joint (pp. 535–536). New York: IEEE.Google Scholar
  4. 4.
    Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011). Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends® in Machine Learning, 3(1), 1–122.CrossRefGoogle Scholar
  5. 5.
    Cappé, O., Moulines, E., & Rydén, T. (2005). Inference in hidden Markov models. Springer series in statistics. Basel: Springer Nature Switzerland AGGoogle Scholar
  6. 6.
    Catellier, D. J., Hannan, P. J., Murray, D. M., Addy, C. L., Conway, T. L., Yang, S., et al. (2005). Imputation of missing data when measuring physical activity by accelerometry. Medicine and Science in Sports and Exercise, 37(11 suppl.), S555.CrossRefGoogle Scholar
  7. 7.
    Cradock, A. L., Wiecha, J. L., Peterson, K. E., Sobol, A. M., Colditz, G. A., & Gortmaker, S. L. (2004). Youth recall and tritrac accelerometer estimates of physical activity levels. Medicine and Science in Sports and Exercise, 36(3), 525–532.CrossRefGoogle Scholar
  8. 8.
    Douc, R., Moulines, E., & Rydén, T. (2004). Asymptotic properties of the maximum likelihood estimator in autoregressive models with Markov regime. The Annals of Statistics, 32(5), 2254–2304.MathSciNetCrossRefGoogle Scholar
  9. 9.
    Efron, B. (1992). Bootstrap methods: another look at the jackknife. In Breakthroughs in statistics (pp. 569–593). New York: Springer.CrossRefGoogle Scholar
  10. 10.
    Evenson, K. R. (2011). Towards an understanding of change in physical activity from pregnancy through postpartum. Psychology of Sport and Exercise, 12(1), 36–45.CrossRefGoogle Scholar
  11. 11.
    Gruen, M. E., Alfaro-Córdoba, M., Thomson, A. E., Worth, A. C. , Staicu, A.-M. , & Lascelles, B. D. X. (2017). The use of functional data analysis to evaluate activity in a spontaneous model of degenerative joint disease associated pain in cats. PLoS One, 12(1), e0169576.CrossRefGoogle Scholar
  12. 12.
    Hansen, B. H., Kolle, E., Dyrstad, S. M., Holme, I., & Anderssen, S. A. (2012). Accelerometer-determined physical activity in adults and older people. Medicine and Science in Sports and Exercise, 44(2), 266–272.CrossRefGoogle Scholar
  13. 13.
    He, J., Li, H., & Tan, J. (2007). Real-time daily activity classification with wireless sensor networks using hidden Markov model. In Engineering in Medicine and Biology Society, 2007. EMBS 2007. 29th Annual International Conference of the IEEE (pp. 3192–3195). New York: IEEE.Google Scholar
  14. 14.
    Hong, M., & Luo, Z.-Q. (2017). On the linear convergence of the alternating direction method of multipliers. Mathematical Programming, 162(1–2), 165–199.MathSciNetCrossRefGoogle Scholar
  15. 15.
    Kanai, M., Izawa, K. P., Kobayashi, M., Onishi, A., Kubo, H., Nozoe, M., et al. (2018). Effect of accelerometer-based feedback on physical activity in hospitalized patients with ischemic stroke: A randomized controlled trial. Clinical Rehabilitation.
  16. 16.
    Lee, J. A., & Gill, J. (2016). Missing value imputation for physical activity data measured by accelerometer. Statistical Methods in Medical Research.
  17. 17.
    Liu, Y.-Y., Li, S., Li, F., Song, L., & Rehg, J. M. (2015). Efficient learning of continuous-time hidden Markov models for disease progression. In Advances in Neural Information Processing Systems (pp. 3600–3608).Google Scholar
  18. 18.
    Marshall, A., Medvedev, O., & Markarian, G. (2007). Self management of chronic disease using mobile devices and bluetooth monitors. In Proceedings of the ICST 2nd International Conference on Body Area Networks (pp. 22). ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering).Google Scholar
  19. 19.
    Metzger, J. S., Catellier, D. J., Evenson, K. R., Treuth, M. S., Rosamond, W. D., & Siega-Riz, A. M. (2008). Patterns of objectively measured physical activity in the United States. Medicine and Science in Sports and Exercise, 40(4), 630–638.CrossRefGoogle Scholar
  20. 20.
    Morris, J. S., Arroyo, C., Coull, B. A., Ryan, L. M., Herrick, R., & Gortmaker, S. L. (2006). Using wavelet-based functional mixed models to characterize population heterogeneity in accelerometer profiles: a case study. Journal of the American Statistical Association, 101(476), 1352–1364.MathSciNetCrossRefGoogle Scholar
  21. 21.
    Napolitano, M. A., Borradaile, K. E., Lewis, B. A., Whiteley, J. A., Longval, J. L., Parisi, A. F., et al. (2010). Accelerometer use in a physical activity intervention trial. Contemporary Clinical Trials, 31(6), 514–523.CrossRefGoogle Scholar
  22. 22.
    Nickel, C., Busch, C., Rangarajan, S., & Möbius, M. (2011). Using hidden Markov models for accelerometer-based biometric gait recognition. In 2011 IEEE 7th International Colloquium on Signal Processing and its Applications (CSPA) (pp. 58–63). New York: IEEE.CrossRefGoogle Scholar
  23. 23.
    Nodelman, U., Shelton, C. R., & Koller, D. (2012). Expectation maximization and complex duration distributions for continuous time Bayesian networks. Preprint. arXiv:1207.1402.Google Scholar
  24. 24.
    Pyke, R. (1961a). Markov renewal processes: Definitions and preliminary properties. The Annals of Mathematical Statistics, 32, 1231–1242.MathSciNetCrossRefGoogle Scholar
  25. 25.
    Pyke, R. (1961b). Markov renewal processes with finitely many states. The Annals of Mathematical Statistics, 32, 1243–1259.MathSciNetCrossRefGoogle Scholar
  26. 26.
    Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2), 257–286.CrossRefGoogle Scholar
  27. 27.
    Robertson, W., Stewart-Brown, S., Wilcock, E., Oldfield, M., & Thorogood, M. (2010). Utility of accelerometers to measure physical activity in children attending an obesity treatment intervention. Journal of Obesity2011.
  28. 28.
    Ronao, C. A., & Cho, S.-B. (2014). Human activity recognition using smartphone sensors with two-stage continuous hidden Markov models. In 2014 10th International Conference on Natural Computation (ICNC) (pp. 681–686). New York: IEEE.CrossRefGoogle Scholar
  29. 29.
    Schmid, D., Ricci, C., & Leitzmann, M. F. (2015). Associations of objectively assessed physical activity and sedentary time with all-cause mortality in us adults: The NHANES study. PLoS One, 10(3), e0119591.CrossRefGoogle Scholar
  30. 30.
    Scott, S. L., James, G. M., & Sugar, C. A. (2005). Hidden Markov models for longitudinal comparisons. Journal of the American Statistical Association, 100(470), 359–369.MathSciNetCrossRefGoogle Scholar
  31. 31.
    Shi, W., Ling, Q., Yuan, K., Wu, G., & Yin, W. (2014). On the linear convergence of the admm in decentralized consensus optimization. IEEE Transactions on Signal Processing62(7), 1750–1761.Google Scholar
  32. 32.
    Shirley, K. E., Small, D. S., Lynch, K. G., Maisto, S. A., & Oslin, D. W. (2010). Hidden Markov models for alcoholism treatment trial data. The Annals of Applied Statistics, 4, 366–395.MathSciNetCrossRefGoogle Scholar
  33. 33.
    Troiano, R. P., Berrigan, D., Dodd, K. W., Mâsse, L. C., Tilert, T., McDowell, M., et al. (2008). Physical activity in the United States measured by accelerometer. Medicine and Science in Sports and Exercise, 40(1), 181.CrossRefGoogle Scholar
  34. 34.
    Wang, X., Sontag, D., & Wang, F. (2014). Unsupervised learning of disease progression models. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 85–94). New York: ACM.Google Scholar
  35. 35.
    Witowski, V., Foraita, R., Pitsiladis, Y., Pigeot, I., & N. Wirsik (2014). Using hidden Markov models to improve quantifying physical activity in accelerometer data—A simulation study. PLoS One, 9(12), e114089.CrossRefGoogle Scholar
  36. 36.
    Xiao, L., Huang, L., Schrack, J. A., Ferrucci, L., Zipunnikov, V., & Crainiceanu, C. M. (2014). Quantifying the lifetime circadian rhythm of physical activity: A covariate-dependent functional approach. Biostatistics, 16(2), 352–367.MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Department of StatisticsNorth Carolina State UniversityRaleighUSA

Personalised recommendations