Shift-Invariant Grouped Multi-task Learning for Gaussian Processes

  • Yuyang Wang
  • Roni Khardon
  • Pavlos Protopapas
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6323)


Multi-task learning leverages shared information among data sets to improve the learning performance of individual tasks. The paper applies this framework for data where each task is a phase-shifted periodic time series. In particular, we develop a novel Bayesian nonparametric model capturing a mixture of Gaussian processes where each task is a sum of a group-specific function and a component capturing individual variation, in addition to each task being phase shifted. We develop an efficient em algorithm to learn the parameters of the model. As a special case we obtain the Gaussian mixture model and em algorithm for phased-shifted periodic time series. Experiments in regression, classification and class discovery demonstrate the performance of the proposed model using both synthetic data and real-world time series data from astrophysics. Our methods are particularly useful when the time series are sparsely and non-synchronously sampled.


Root Mean Square Error Gaussian Process Gaussian Mixture Model Single Task Reproduce Kernel Hilbert Space 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Bi, J., Xiong, T., Yu, S., Dundar, M., Rao, R.: An improved multi-task learning approach with applications in medical diagnosis. In: ECML/PKDD, pp. 117–132 (2008)Google Scholar
  2. 2.
    Dinuzzo, F., Pillonetto, G., De Nicolao, G.: Client-server multi-task learning from distributed datasets. Arxiv preprint arXiv:0812.4235 (2008)Google Scholar
  3. 3.
    Bickel, S., Bogojeska, J., Lengauer, T., Scheffer, T.: Multi-task learning for HIV therapy screening. In: ICML, pp. 56–63 (2008)Google Scholar
  4. 4.
    Evgeniou, T., Micchelli, C., Pontil, M.: Learning multiple tasks with kernel methods. JMLR 6(1), 615–637 (2006)MathSciNetGoogle Scholar
  5. 5.
    Xue, Y., Liao, X., Carin, L., Krishnapuram, B.: Multi-task learning for classification with Dirichlet process priors. JMLR 8, 35–63 (2007)MathSciNetGoogle Scholar
  6. 6.
    Yu, K., Tresp, V., Schwaighofer, A.: Learning Gaussian processes from multiple tasks. In: ICML, pp. 1012–1019 (2005)Google Scholar
  7. 7.
    Schwaighofer, A., Tresp, V., Yu, K.: Learning Gaussian process kernels via hierarchical Bayes. NIPS 17, 1209–1216 (2005)Google Scholar
  8. 8.
    Pillonetto, G., Dinuzzo, F., De Nicolao, G.: Bayesian Online Multitask Learning of Gaussian Processes. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(2), 193–205 (2010)CrossRefGoogle Scholar
  9. 9.
    Rebbapragada, U., Protopapas, P., Brodley, C.E., Alcock, C.: Finding anomalous periodic time series. Machine Learning 74(3), 281–313 (2009)CrossRefGoogle Scholar
  10. 10.
    Gaffney, S.J., Smyth, P.: Curve clustering with random effects regression mixtures. In: Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics (2003)Google Scholar
  11. 11.
    Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. The MIT Press, Cambridge (2005)Google Scholar
  12. 12.
    Lu, Z., Leen, T., Huang, Y., Erdogmus, D.: A reproducing kernel Hilbert space framework for pairwise time series distances. In: ICML, pp. 624–631 (2008)Google Scholar
  13. 13.
    Seeger, M.: Gaussian processes for machine learning. International Journal of Neural Systems 14(2), 69–106 (2004)CrossRefGoogle Scholar
  14. 14.
    Protopapas, P., Giammarco, J.M., Faccioli, L., Struble, M.F., Dave, R., Alcock, C.: Finding outlier light curves in catalogues of periodic variable stars. Monthly Notices of the Royal Astronomical Society 369, 677–696 (2006)CrossRefGoogle Scholar
  15. 15.
    Udalski, A., Szymanski, M., Kubiak, M., Pietrzynski, G., Wozniak, P., Zebrun, Z.: Optical gravitational lensing experiment. photometry of the macho-smc-1 microlensing candidate. Acta Astronomica 47, 431–436 (1997)Google Scholar
  16. 16.
    Alcock, C., et al.: The MACHO Project - a Search for the Dark Matter in the Milky-Way. In: Soifer, B.T. (ed.) Sky Surveys. Protostars to Protogalaxies. Astronomical Society of the Pacific Conference Series, vol. 43, pp. 291–296 (1993)Google Scholar
  17. 17.
    Faccioli, L., Alcock, C., Cook, K., Prochter, G.E., Protopapas, P., Syphers, D.: Eclipsing Binary Stars in the Large and Small Magellanic Clouds from the MACHO Project: The Sample. Astronomy Journal 134, 1963–1993 (2007)CrossRefGoogle Scholar
  18. 18.
    Hodapp, K.W., et al.: Design of the Pan-STARRS telescopes. Astronomische Nachrichten 325, 636–642 (2004)CrossRefGoogle Scholar
  19. 19.
    Soszynski, I., Udalski, A., Szymanski, M.: The Optical Gravitational Lensing Experiment. Catalog of RR Lyr Stars in the Large Magellanic Cloud 06. Acta Astronomica 53, 93–116 (2003)Google Scholar
  20. 20.
    Wachman, G., Khardon, R., Protopapas, P., Alcock, C.: Kernels for Periodic Time Series Arising in Astronomy. In: ECML/PKDD, pp. 489–505 (2009)Google Scholar
  21. 21.
    Chudova, D., Gaffney, S.J., Mjolsness, E., Smyth, P.: Translation-invariant mixture models for curve clustering. In: SIGKDD, pp. 79–88 (2003)Google Scholar
  22. 22.
    Wachman, G.: Kernel Methods and Their Application to Structured Data. PhD thesis, Tufts University (2009)Google Scholar
  23. 23.
    Ding, H., Trajcevski, G., Scheuermann, P., Wang, X., Keogh, E.: Querying and mining of time series data: experimental comparison of representations and distance measures. VLDB 1(2), 1542–1552 (2008)Google Scholar
  24. 24.
    Osowski, S., Hoai, L., Markiewicz, T.: Support vector machine-based expert system for reliable heartbeat recognition. IEEE Transactions on Biomedical Engineering 51(4), 582–589 (2004)CrossRefGoogle Scholar
  25. 25.
    Hughes, N., Tarassenko, L., Roberts, S.: Markov models for automated ECG interval analysis. NIPS 16 (2003)Google Scholar
  26. 26.
    Kim, S., Smyth, P.: Segmental Hidden Markov Models with Random Effects for Waveform Modeling. JMLR 7, 945–969 (2006)MathSciNetGoogle Scholar
  27. 27.
    Rasmussen, C.E., Ghahramani, Z.: Infinite mixtures of Gaussian process experts. NIPS 15, 881–888 (2002)Google Scholar
  28. 28.
    Gaffney, S.J., Smyth, P.: Trajectory clustering with mixtures of regression models. In: SIGKDD, pp. 63–72 (1999)Google Scholar
  29. 29.
    Gaffney, S.J., Smyth, P.: Joint probabilistic curve clustering and alignment. NIPS 17, 473–480 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Yuyang Wang
    • 1
  • Roni Khardon
    • 1
  • Pavlos Protopapas
    • 2
  1. 1.Tufts UniversityMedfordUSA
  2. 2.Harvard-Smithsonian Center for AstrophysicsCambridgeUSA

Personalised recommendations