Dynamic Grouped Mixture Models for Intermittent Multivariate Sensor Data

  • Naoya TakeishiEmail author
  • Takehisa Yairi
  • Naoki Nishimura
  • Yuta Nakajima
  • Noboru Takata
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9652)


For secure and efficient operation of engineering systems, it is of great importance to watch daily logs generated by them, which mainly consist of multivariate time-series obtained with many sensors. This work focuses on challenges in practical analyses of those sensor data: temporal unevenness and sparseness. To handle the unevenly and sparsely spaced multivariate time-series, this work presents a novel method, which roughly models temporal information that still remains in the data. The proposed model is a mixture model with dynamic hierarchical structure that considers dependency between temporally close batches of observations, instead of every single observation. We conducted experiments with synthetic and real dataset, and confirmed validity of the proposed model quantitatively and qualitatively.


Multivariate time-series Unevenly spaced time-series Mixture models Latent factor models Sensor data 


  1. 1.
    Adorf, H.M.: Interpolation of irregularly sampled data series - a survey. In: Astronomical Data Analysis Software and Systems IV (1995)Google Scholar
  2. 2.
    Blei, D.M., Lafferty, J.D.: Dynamic topic models. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 113–120 (2006)Google Scholar
  3. 3.
    Brockwell, P.J.: Lévy-driven continuous-time ARMA processes. In: Mikosch, T., Kreiß, J.-P., Davis, R.A., Andersen, T.G. (eds.) Handbook of Financial Time Series, pp. 457–480. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  4. 4.
    Dunia, R., Qin, S.J., Edgar, T.F., McAvoy, T.J.: Identification of faulty sensors using principal component analysis. AIChE J. 42(10), 2797–2812 (1996)CrossRefGoogle Scholar
  5. 5.
    Erdogan, E.: Statistical models for unequally spaced time series. In: Proceedings of the 5th SIAM International Conference on Data Mining, pp. 626–630 (2005)Google Scholar
  6. 6.
    Foster, G.: Wavelets for period analysis of unevenly sampled time series. Astron. J. 112, 1709–1729 (1996)CrossRefGoogle Scholar
  7. 7.
    Geweke, J.: The dynamic factor analysis of economic time-series models. In: Aigner, D.J., Goldberger, A.S. (eds.) Latent Variables in Socio-Economic Models. North-Holland, New York (1977)Google Scholar
  8. 8.
    Ghahramani, Z., Hinton, G.E.: The EM algorithm for mixtures of factor analyzers. Technical report, University of Toronto (1996)Google Scholar
  9. 9.
    Ghahramani, Z., Jordan, M.I.: Factorial hidden markov models. Mach. Learn. 29(2–3), 245–273 (1997)CrossRefzbMATHGoogle Scholar
  10. 10.
    Gupta, M.D., Huang, T.S.: Regularized maximum likelihood for intrinsic dimension estimation. In: Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence (2010)Google Scholar
  11. 11.
    Hayashi, T., Yoshida, N.: On covariance estimation of non-synchronously observed diffusion processes. Bernoulli 11, 359–379 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Jones, R.: Time series analysis with unequally spaced data. In: Hannan, E.J., Krishnaiah, P.R., Rao, M.M. (eds.) Handbook of Statistics, vol. 5. Elsevier Science, Amsterdam (1985)Google Scholar
  13. 13.
    Kermit, M., Tomic, O.: Independent component analysis applied on gas sensor array measurement data. IEEE Sens. J. 3(2), 218–228 (2003)CrossRefGoogle Scholar
  14. 14.
    Lawrence, N.: Probabilistic non-linear principal component analysis with gaussian process latent variable models. J. Mach. Learn. Res. 6, 1783–1816 (2005)MathSciNetzbMATHGoogle Scholar
  15. 15.
    Nakamura, Y., Nishijo, K., Murakami, N., Kawashima, K., Horikawa, Y., Yamamoto, K., Ohtani, T., Takhashi, Y., Inoue, K.: Small demonstration satellite-4 (SDS-4): development, flight results, and lessons learned in JAXAs microsatellite project. In: Proceedings of the 27th Annual AIAA/USU Conference on Small Satellites (2013)Google Scholar
  16. 16.
    Nigam, K., McCallum, A.K., Thrun, S., Mitchell, T.: Text classification from labeled and unlabeled documents using EM. Mach. Learn. 39(2/3), 103–134 (2000)CrossRefzbMATHGoogle Scholar
  17. 17.
    Rehfeld, K., Marwan, N., Heitzig, J., Kurths, J.: Comparison of correlation analysis techniques for irregularly sampled time series. Nonlinear Proc. Geophys. 18, 389–404 (2011)CrossRefGoogle Scholar
  18. 18.
    Rosti, A.V.I., Gales, M.J.F.: Factor analysed hidden Markov models for speech recognition. Comput. Speech Lang. 18(2), 181–200 (2004)CrossRefGoogle Scholar
  19. 19.
    Sammon, J.W.: A nonlinear mapping for data structure analysis. IEEE Trans. Comput. 18, 401–409 (1969)CrossRefGoogle Scholar
  20. 20.
    Scargle, J.: Studies in astronomical time series analysis. II - Statistical aspects of spectral analysis of unevenly spaced data. Astrophys. J. 343, 874–887 (1982)CrossRefGoogle Scholar
  21. 21.
    Schölkopf, B., Platt, J.C., Shawe-Taylor, J.C., Smola, A.J., Williamson, R.C.: Estimating the support of a high-dimensional distribution. Neural Comput. 13(7), 1443–1471 (2001)CrossRefzbMATHGoogle Scholar
  22. 22.
    Tipping, M., Bishop, C.: Mixtures of probabilistic principal component analysers. Neural Comput. 11(2), 443–482 (1999)CrossRefGoogle Scholar
  23. 23.
    Watson, M.W., Engle, R.F.: Alternative algorithms for the estimation of dynamic factor, mimic and varying coefficient regression models. J. Econometrics 23, 385–400 (1983)CrossRefzbMATHGoogle Scholar
  24. 24.
    Zumbach, G., Müller, U.: Operators on inhomogeneous time series. Int. J. Theor. Appl. Fin. 4, 147–177 (2001)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Naoya Takeishi
    • 1
    Email author
  • Takehisa Yairi
    • 1
  • Naoki Nishimura
    • 2
  • Yuta Nakajima
    • 2
  • Noboru Takata
    • 2
  1. 1.The University of TokyoTokyoJapan
  2. 2.Japan Aerospace Exploration AgencyTsukubaJapan

Personalised recommendations