Modeling Binary Time Series Using Gaussian Processes with Application to Predicting Sleep States


Motivated by the problem of predicting sleep states, we develop a mixed effects model for binary time series with a stochastic component represented by a Gaussian process. The fixed component captures the effects of covariates on the binary-valued response. The Gaussian process captures the residual variations in the binary response that are not explained by covariates and past realizations. We develop a frequentist modeling framework that provides efficient inference and more accurate predictions. Results demonstrate the advantages of improved prediction rates over existing approaches such as logistic regression, generalized additive mixed model, models for ordinal data, gradient boosting, decision tree and random forest. Using our proposed model, we show that previous sleep state and heart rates are significant predictors for future sleep states. Simulation studies also show that our proposed method is promising and robust. To handle computational complexity, we utilize Laplace approximation, golden section search and successive parabolic interpolation. With this paper, we also submit an R-package (HIBITS) that implements the proposed procedure.

This is a preview of subscription content, access via your institution.


  1. BANERJEE, S., CARLIN, B.P., and GELFAND, A.E. (2014), Hierarchical Modeling and Analysis for Spatial Data, CRC Press.

  2. BANERJEE, S., GELFAND, A.E., FINLEY, A.O., and SANG, H. (2008), “Gaussian Predictive Process Models for Large Spatial Data Sets”, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 70(4), 825–848.

    MathSciNet  Article  Google Scholar 

  3. BENBADIS, S.R. (2006), “Introduction to Sleep Electroencephalography”, in Sleep: A Comprehensive Handbook, USA: John Wiley and Sons, pp. 989–1024.

    Google Scholar 

  4. BONNEY, G.E. (1987), “Logistic Regression for Dependent Binary Observations”, Biometrics, 45, 951–973.

    Article  Google Scholar 

  5. BRILLINGER, D.R. (1983), “A Generalized Linear Model with Gaussian Regressor Variables”, in A Festschrift for Erich L. Lehmann, Pacific Grove, CA:Wadsworth, pp. 97–114.

  6. CAIADO, J., CRATO, N., and PEÑA, D. (2006), “A Periodogram-Based Metric for Time Series Classification”, Computational Statistics and Data Analysis 50(10), 2668–2684.

    MathSciNet  Article  Google Scholar 

  7. CORNFORD, D. (1998), “Non-Zero Mean Gaussian Process Prior Wind Field Models”, Technical Report, Aston University, Birmingham.

  8. FOKIANOS, K., and KEDEM, B. (1998), “Prediction and Classification of Non-Stationary Categorical Time Series”, Journal of Multivariate Analysis, 67(2), 277–296.

    MathSciNet  Article  Google Scholar 

  9. FOKIANOS, K., and KEDEM, B. (2002), Regression Model for Time Series Analysis, Wiley Interscience.

  10. FOKIANOS, K., and KEDEM, B. (2003), “Regression Theory for Categorical Time Series”, Statistical Science, 18(3), 357–376.

    MathSciNet  Article  Google Scholar 

  11. FRIEDMAN, J., HASTIE, T., and TIBSHIRANI, R. (2001), The Elements of Statistical Learning (Vol. 1), Springer Series in Statistics, Berlin: Springer.

  12. FRIEDMAN, J.H. (2001), “Greedy Function Approximation: A Gradient Boosting Machine”, Annals of Statistics, 29(5), 1189–1232.

    MathSciNet  Article  Google Scholar 

  13. GELFAND, A.E., KOTTAS, A., and MACEACHERN, S.N. (2005), “Bayesian Nonparametric Spatial Modeling with Dirichlet Process Mixing”, Journal of the American Statistical Association 100(471), 1021–1035.

    MathSciNet  Article  Google Scholar 

  14. JACOBS, P.A., and LEWIS, P.A. (1978), “Discrete Time Series Generated by Mixtures II: Asymptotic Properties”, Journal of the Royal Statistical Society. Series B (Methodological), 40(2), 222–228.

    MathSciNet  MATH  Google Scholar 

  15. KEENAN, D.M. (1982), “A Time Series Analysis of Binary Data”, Journal of the American Statistical Association 77(380), 816–821.

    MathSciNet  Article  Google Scholar 

  16. KUSS, M. (2006), “Gaussian Process Models for Robust Regression, Classification, and Reinforcement Learning”, Ph. D. thesis, Technische Universität Darmstadt.

  17. LIN, X., and ZHANG, D. (1999), “Inference in Generalized Additive Mixed Models by Using Smoothing Splines”, Journal of the Royal Statistical Society: Series B (Statistical Methodology) 61(2), 381–400.

    MathSciNet  Article  Google Scholar 

  18. LINDQUIST, M.A., and MCKEAGUE, I. (2009), “Logistic Regression with Brownian-Like Predictors”, Journal of the American Statistical Association 104, 1575–1585.

    MathSciNet  Article  Google Scholar 

  19. MAHARAJ, E.A. (2002), “Comparison of Non-Stationary Time Series in the Frequency Domain”, Computational Statistics and Data Analysis 40(1), 131–141.

    MathSciNet  Article  Google Scholar 

  20. MAHARAJ, E.A., D’URSO, P., and GALAGEDERA, D.U. (2010), “Wavelet-Based Fuzzy Clustering of Time Series”, Journal of Classification 27(2), 231–275.

    MathSciNet  Article  Google Scholar 

  21. MCCULLAGH, P. (1984), “Generalized Linear Models”, European Journal of Operational Research 16(3), 285–292.

    MathSciNet  Article  Google Scholar 

  22. MEYN, S.P., and Tweedie, R.L. (2012), Markov Chains and Stochastic Stability, Springer Science and Business Media.

  23. MINKA, T.P. (2001), “A Family of Algorithms for Approximate Bayesian Inference”, Ph. D. thesis, Massachusetts Institute of Technology.

  24. NEVSIMALOVA, S., and SONKA, K. (1997), “Poruchy Spanku a Bdeni”, Maxdorf/ Jessenius, Parha.

  25. OPPER, M., and WINTHER, O. (2000), “Gaussian Processes for Classification: Mean Field Algorithms”, Neural Computation, 12(11), 2655-2684.

    Article  Google Scholar 

  26. QUICK, H., BANERJEE, S., CARLIN, B.P. et al. (2013), “Modeling Temporal Gradients in Regionally Aggregated California Asthma Hospitalization Data”, The Annals of Applied Statistics 7(1), 154–176.

    MathSciNet  Article  Google Scholar 

  27. SNELSON, E., RASMUSSEN, C.E., and GHAHRAMANI, Z. (2004), “Warped Gaussian Processes”, Advances in Neural Information Processing Systems 16, 337–344.

    Google Scholar 

  28. STEIN, M.L. (2012), Interpolation of Spatial Data: Some Theory for Kriging, Springer Science and Business Media.

  29. VANDENBERG-RODES, A., and SHAHBABA, B. (2015), “Dependent Matern Processes for Multivariate Time Series”, arXiv preprint arXiv:1502.03466.

  30. WANG, F., and GELFAND, A.E.(2014), “Modeling Space and Space-Time Directional Data Using Projected Gaussian Processes”, Journal of the American Statistical Association 109(508), 1565–1580.

    MathSciNet  Article  Google Scholar 

  31. WILLIAMS, C.K., and Barber, D. (1998), “Bayesian Classification with Gaussian Processes” IEEE Transactions on Pattern Analysis and Machine Intelligence, 20,(12), 1342–1351.

    Article  Google Scholar 

  32. WILLIAMS, C.K., and RASMUSSEN, C.E. (2006), “Gaussian Processes for Machine Learning”, The MIT Press 2(3), 4.

    MATH  Google Scholar 

  33. ZHOU, B., MOORMAN, D.E., BEHSETA, S., OMBAO, H., and SHAHBABA, B. (2015), “A Dynamic Bayesian Model for Characterizing Cross-Neuronal Interactions During Decision Making”, Journal of the American Statistical Association 111, 1–44.

    MathSciNet  Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Hernando Ombao.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Gao, X., Shahbaba, B. & Ombao, H. Modeling Binary Time Series Using Gaussian Processes with Application to Predicting Sleep States. J Classif 35, 549–579 (2018).

Download citation


  • Binary time series
  • Classification
  • Gaussian process
  • Latent process
  • Sleep state