Unsupervised Temporal Ensemble Alignment for Rapid Annotation

  • Ashton FaggEmail author
  • Sridha Sridharan
  • Simon Lucey
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9008)


This paper presents a novel framework for the unsupervised alignment of an ensemble of temporal sequences. This approach draws inspiration from the axiom that an ensemble of temporal signals stemming from the same source/class should have lower rank when “aligned” rather than “misaligned”. Our approach shares similarities with recent state of the art methods for unsupervised images ensemble alignment (e.g. RASL) which breaks the problem into a set of image alignment problems (which have well known solutions i.e. the Lucas-Kanade algorithm). Similarly, we propose a strategy for decomposing the problem of temporal ensemble alignment into a similar set of independent sequence problems which we claim can be solved reliably through Dynamic Time Warping (DTW). We demonstrate the utility of our method using the Cohn-Kanade+ dataset, to align expression onset across multiple sequences, which allows us to automate the rapid discovery of event annotations.


Ground Truth Dynamic Time Warping Ensemble Method Augmented Lagrangian Method Alignment Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This research was supported by an Australian Research Council (ARC) Discovery Research Grant DP140100793.


  1. 1.
    Learned-Miller, E.G.: Data driven image models through continuous joint alignment. PAMI 28, 236–250 (2006)CrossRefGoogle Scholar
  2. 2.
    Cox, M., Sridharan, S., Lucey, S., Cohn, J.: Least squares congealing for unsupervised alignment of images. In: CVPR, pp. 1–8. IEEE (2008)Google Scholar
  3. 3.
    Peng, Y., Ganesh, A., Wright, J., Ma, Y.: RASL: Robust alignment via sparse and lowrank decomposition. In: CVPR. IEEE (2010)Google Scholar
  4. 4.
    Peng, Y., Ganesh, A., Wright, J., Xu, W., Ma, Y.: RASL: Robust alignment by sparse and low-rank decomposition for linearly correlated images. PAMI 34, 2233–2246 (2012)CrossRefGoogle Scholar
  5. 5.
    Zhou, F., De la Torre, F.: Generalized time warping for multi-modal alignment of human motion. In: CVPR, pp. 1282–1289. IEEE (2012)Google Scholar
  6. 6.
    Panagakis, Y., Nicolaou, M.A., Zafeiriou, S., Pantic, M.: Robust canonical time warping for the alignment of grossly corrupted sequences. In: CVPR, pp. 540–547. IEEE (2013)Google Scholar
  7. 7.
    Candès, E.J., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? JACM 58, 11 (2011)CrossRefGoogle Scholar
  8. 8.
    Müller, M.: Dynamic time warping. In: Müller, M. (ed.) Information Retrieval for Music and Motion, pp. 69–84. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  9. 9.
    Lucas, B.D., Kanade, T., et al.: An iterative image registration technique with an application to stereo vision. In: IJCAI, vol. 81, pp. 674–679 (1981)Google Scholar
  10. 10.
    Cuturi, M., Vert, J.P., Birkenes, Ø., Matsui, T.: A kernel for time series based on global alignments. In: ICASSP, vol. 2, pp. 413–416. IEEE (2007)Google Scholar
  11. 11.
    Cuturi, M.: Fast global alignment kernels. In: ICML, pp. 929–936 (2011)Google Scholar
  12. 12.
    Lorincz, A., Jeni, L.A., Szabó, Z., Cohn, J.F., Kanade, T.: Emotional expression classification using time-series kernels. In: CVPRW, pp. 889–895. IEEE (2013)Google Scholar
  13. 13.
    Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression. In: CVPRW, pp. 94–101. IEEE (2010)Google Scholar
  14. 14.
    Zhou, F., De la Torre, F.: Canonical time warping for alignment of human behavior. In: NIPS, pp. 2286–2294 (2009)Google Scholar
  15. 15.
    Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1–122 (2011)CrossRefGoogle Scholar
  16. 16.
    Dai, Y., Li, H., He, M.: A simple prior-free method for non-rigid structure-from-motion factorization. In: CVPR, pp. 2018–2025. IEEE (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.CSIROBrisbaneAustralia
  2. 2.Queensland University of TechnologyBrisbaneAustralia
  3. 3.Carnegie Mellon UniversityPittsburghUSA

Personalised recommendations