One-Shot Learned Priors in Augmented Active Appearance Models for Anatomical Landmark Tracking

  • Oliver MothesEmail author
  • Joachim Denzler
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 983)


In motion science, biology and robotics animal movement analyses are used for the detailed understanding of the human bipedal locomotion. For this investigations an immense amount of recorded image data has to be evaluated by biological experts. During this time-consuming evaluation single anatomical landmarks, for example bone ends, have to be located and annotated in each image. In this paper we show a reduction of this effort by automating the annotation with a minimum level of user interaction. Recent approaches, based on Active Appearance Models, are improved by priors based on anatomical knowledge and an online tracking method, requiring only a single labeled frame. In contrast, we propose a one-shot learned tracking-by-detection prior which overcomes the shortcomings of template drifts without increasing the number of training data. We evaluate our approach based on a variety of real-world X-ray locomotion datasets and show that our method outperforms recent state-of-the-art concepts for the task at hand.


One-shot learned detector X-ray videography Graph-based landmark tracking Animal locomotion analysis Active appearance models 



The research was supported by grant DE 735/8-3 of the German Research Foundation (DFG).


  1. 1.
    Nyakatura, J.A., Andrada, E., Blickhan, R., Fischer, M.S.: Avian bipedal locomotion. In: 5th International Symposium on Adaptive Motion of Animals and Machines (AMAM). Elsevier (2011)Google Scholar
  2. 2.
    Andrada, E., Nyakatura, J.A., Bergmann, F., Blickhan, R.: Adjustments of global and local hindlimb properties during terrestrial locomotion of the common quail (coturnix coturnix). J. Exp. Biol. (2013)Google Scholar
  3. 3.
    Sigal, L., Balan, A.O., Black, M.J.: HumanEva: synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion. Int. J. Comput. Vis. 87(1–2), 4 (2010)CrossRefGoogle Scholar
  4. 4.
    Haase, D., Andrada, E., Nyakatura, J.A., Kilbourne, B.M., Denzler, J.: Automated approximation of center of mass position in X-ray sequences of animal locomotion. J. Biomech. 46, 2082–2086 (2013)CrossRefGoogle Scholar
  5. 5.
    Haase, D., Denzler, J.: 2D and 3D analysis of animal locomotion from biplanar X-ray videos using augmented active appearance models. EURASIP J. Image Video Process. (2013)Google Scholar
  6. 6.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. (2001)Google Scholar
  7. 7.
    Amthor, M., Haase, D., Denzler, J.: Fast and robust landmark tracking in X-ray locomotion sequences containing severe occlusions. In: International Workshop on Vision, Modelling, and Visualization (VMV). Eurographics Association (2012)Google Scholar
  8. 8.
    Mothes, O., Denzler, J.: Anatomical landmark tracking by one-shot learned priors for augmented active appearance models. In: International Conference on Computer Vision Theory and Applications (VISAPP), pp. 246–254 (2017)Google Scholar
  9. 9.
    Haase, D., Denzler, J.: Anatomical landmark tracking for the analysis of animal locomotion in X-ray videos using active appearance models. In: Heyden, A., Kahl, F. (eds.) SCIA 2011. LNCS, vol. 6688, pp. 604–615. Springer, Heidelberg (2011). Scholar
  10. 10.
    Haase, D., Nyakatura, J.A., Denzler, J.: Multi-view active appearance models for the X-ray based analysis of avian bipedal locomotion. In: Mester, R., Felsberg, M. (eds.) DAGM 2011. LNCS, vol. 6835, pp. 11–20. Springer, Heidelberg (2011). Scholar
  11. 11.
    Lelieveldt, B., Üzümcü, M., van der Geest, R., Reiber, J., Sonka, M.: Multi-view active appearance models for consistent segmentation of multiple standard views: application to long- and short-axis cardiac MR images. In: International Congress Series (2003)Google Scholar
  12. 12.
    Amthor, M., Haase, D., Denzler, J.: Robust pictorial structures for X-ray animal skeleton tracking. In: International Conference on Computer Vision Theory and Applications (VISAPP). SCITEPRESS (2014)Google Scholar
  13. 13.
    Andriluka, M., Roth, S., Schiele, B.: Monocular 3D pose estimation and tracking by detection. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2010)Google Scholar
  14. 14.
    Li, L., Nawaz, T., Ferryman, J.: Pets 2015: datasets and challenge. In: 2015 12th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). IEEE (2015)Google Scholar
  15. 15.
    Zhang, L., Li, Y., Nevatia, R.: Global data association for multi-object tracking using network flows. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008. IEEE (2008)Google Scholar
  16. 16.
    Berclaz, J., Fleuret, F., Türetken, E., Fua, P.: Multiple object tracking using k-shortest paths optimization. IEEE Trans. Pattern Anal. Mach. Intell. 33, 1806–1819 (2011)CrossRefGoogle Scholar
  17. 17.
    Jiang, X., Haase, D., Körner, M., Bothe, W., Denzler, J.: Accurate 3D multi-marker tracking in X-ray cardiac sequences using a two-stage graph modeling approach. In: Wilson, R., Hancock, E., Bors, A., Smith, W. (eds.) CAIP 2013. LNCS, vol. 8048, pp. 117–125. Springer, Heidelberg (2013). Scholar
  18. 18.
    Dehghan, A., Modiri Assari, S., Shah, M.: GMMCP tracker: globally optimal generalized maximum multi clique problem for multiple object tracking. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2015)Google Scholar
  19. 19.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005. IEEE (2005)Google Scholar
  20. 20.
    Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32, 1627–1645 (2010)CrossRefGoogle Scholar
  21. 21.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004)CrossRefGoogle Scholar
  22. 22.
    Breiman, L.: Classification and Regression Trees. CRC Press, Boca Raton (2017)CrossRefGoogle Scholar
  23. 23.
    Hariharan, B., Malik, J., Ramanan, D.: Discriminative decorrelation for clustering and classification. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7575, pp. 459–472. Springer, Heidelberg (2012). Scholar
  24. 24.
    Sun, Y., Wang, X., Tang, X.: Deep convolutional network cascade for facial point detection. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2013)Google Scholar
  25. 25.
    Zhou, E., Fan, H., Cao, Z., Jiang, Y., Yin, Q.: Extensive facial landmark localization with coarse-to-fine convolutional network cascade. In: 2013 IEEE International Conference on Computer Vision Workshops (ICCVW). IEEE (2013)Google Scholar
  26. 26.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, pp. 484–498. Springer, Heidelberg (1998). Scholar
  27. 27.
    Kendall, D.G.: Shape manifolds, procrustean metrics, and complex projective spaces. Bull. London Math. Soc. 16, 81–121 (1984)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Jolliffe, I.: Principal Component Analysis. Springer Series in Statistics. Springer, Heidelberg (2002). Scholar
  29. 29.
    Berg, M., Cheong, O., Kreveld, M., Overmars, M.: Computational Geometry: Algorithms and Applications, 3rd edn. Springer, Heidelberg (2008). Scholar
  30. 30.
    Baker, S., Matthews, I.: Lucas-kanade 20 years on: a unifying framework. Int. J. Comput. Vis. 56(3), 221–255 (2004)CrossRefGoogle Scholar
  31. 31.
    Freytag, A., Schadt, A., Denzler, J.: Interactive image retrieval for biodiversity research. In: Gall, J., Gehler, P., Leibe, B. (eds.) GCPR 2015. LNCS, vol. 9358, pp. 129–141. Springer, Cham (2015). Scholar
  32. 32.
    Dijkstra, E.W.: A note on two problems in connexion with graphs. Numer. Math. 1(1), 269–271 (1959)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Bellman, R.: On a routing problem. Q. Appl. Math. 16(1), 87–90 (1958)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Computer Vision GroupFriedrich Schiller University JenaJenaGermany

Personalised recommendations