Skip to main content
SpringerLink
Log in

Relevance feedback for human motion retrieval using a boosting approach

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Content-based human motion retrieval (CBMR) has been more and more important with the rapid growth of motion capture data, but the gap between high-level semantic concepts and low-level features hinders further performance improvement. Relevance feedback is an effective tool to narrow the semantic gap and enhance the retrieval performance. However, as a type of variable-length multivariate time series (VLMTS), motion capture data has its own characteristics including high-dimensionality, demand for elastic matching, and difficulty representing different movements in a uniform feature space, which make it much more challenging to design an effective relevance feedback approach. This paper presents a novel boosting approach for CBMR and the main contributions include three aspects. First, to fit in with the characteristics of VLMTS data and meet the real-time requirement of relevance feedback, the ensemble learning framework RankBoost is introduced and k-nearest neighbors combining with dynamic time warping (KNN-DTW) is employed as its weak ranker. Second, the set of extended Boolean geometry features containing much richer geometry elements and measures is used to represent motion content, and it provides a comparatively complete feature set for designing the weak ranker of RankBoost. Third, to solve the over-fitting problem caused by the small-sample training of relevance feedback, a novel learning objective composed of minimizing empirical ranking loss and minimizing the maximum generalization loss is proposed for RankBoost ensemble learning. Experimental results on CMU database and its extended database verify the effectiveness of the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. In traditional RankBoost [27], the ranking loss on each preference pair is [[x ≥ 0]],namely the value is 0 or 1, but in RankBoost relevance feedback algorithm, its value is between −1 and 1. Although the value ranges of ranking loss are different, e x is their common upper-bound.

References

  1. CMU (2003) Motion Capture Database. http://mocap.cs.cmu.edu/.

  2. Liu F, Zhuang YT, Wu F, Pan YH (2003) 3D motion retrieval with motion index tree. Comput Vis Image Underst 92(2–3):265–284. doi:10.1016/j.cviu.2003.06.001

    Article  Google Scholar 

  3. Müller M, Röder T, Clausen M (2005) Efficient content-based retrieval of motion capture data. ACM Trans Graph 24(3):677–685. doi:10.1145/1073204.1073247

    Article  Google Scholar 

  4. Tang JKT, Leung H (2012) Retrieval of logically relevant 3D human motions by adaptive feature selection with graded relevance feedback. Pattern Recogn Lett 33(4):420–430. doi:10.1016/j.patrec.2011.06.005

    Article  Google Scholar 

  5. Chen SL, Sun ZX, Li Y, Li Q (2012) Partial Similarity Human Motion Retrieval Based on Relative Geometry Features. In: Proceedings of the 2012 Fourth International Conference on Digital Home (ICDH'12), 23–25 Nov. 2012. pp 298–303. doi:10.1109/icdh.2012.91

  6. Huang TS, Zhou XS (2001) Image retrieval with relevance feedback: from heuristic weight adjustment to optimal learning methods. In:  Proceedings of international conference on Image Processing, 07–10 Oct. 2001. pp 2–5. doi:10.1109/icip.2001.958036

  7. Zhou XS, Huang TS (2003) Relevance feedback in image retrieval: a comprehensive review. Multimedia Systems 8(6):536–544. doi:10.1007/s00530-002-0070-3

    Article  Google Scholar 

  8. Wu K, Yap KH (2006) Fuzzy SVM for content-based image retrieval. IEEE Comput Intell Mag 1(2):10–16. doi:10.1109/MCI.2006.1626490

    Article  Google Scholar 

  9. Huang SH, Wu QJ, Lai SH (2006) Improved AdaBoost-based image retrieval with relevance feedback via paired feature learning. Multimedia Systems 12(1):14–26. doi:10.1007/s00530-006-0028-y

    Article  Google Scholar 

  10. Yoon H, Yang K, Shahabi C (2005) Feature subset selection and feature ranking for multivariate time series. IEEE Trans Knowl Data Eng 17(9):1186–1198. doi:10.1109/tkde.2005.144

    Article  Google Scholar 

  11. Huang W, Gao Y, Chan KL (2010) A review of region-based image retrieval. J Signal Process Syst Signal Image Video Technol 59(2):143–161. doi:10.1007/s11265-008-0294-3

    Article  Google Scholar 

  12. Jiang W, Er G, Dai QH, Gu JW (2006) Similarity-based online feature selection in content-based image retrieval. IEEE Trans Image Process 15(3):702–712. doi:10.1109/tip.2005.863105

    Article  Google Scholar 

  13. Xing Z, Pei J, Keogh E (2010) A brief survey on sequence classification. SIGKDD Explor Newsl 12(1):40–48. doi:10.1145/1882471.1882478

    Article  Google Scholar 

  14. Li C, Khan L, Prabhakaran B (2006) Real-time classification of variable length multi-attribute motions. Knowl Inf Syst 10(2):163–183. doi:10.1007/s10115-005-0223-8

    Article  Google Scholar 

  15. Shimodaira H, Noma K, Nakai M, Sagayama S (2002) Dynamic time-alignment kernel in support vector machine. In: Dietterich TG, Becker S, Ghahramani Z (eds) Advances in neural information processing systems, 14th edn. MIT Press, USA, pp 921–928

  16. Bahlmann C, Haasdonk B, Burkhardt H (2002) Online handwriting recognition with support vector machines - a kernel approach. In: Proceedings of the Eighth International Workshop on Frontiers in Handwriting Recognition (IWFHR'02), 6–8 Aug. 2002. pp 49–54. doi:10.1109/iwfhr.2002.1030883

  17. Dongyu Z, Wangmeng Z, Zhang D, Hongzhi Z (2010) Time Series Classification Using Support Vector Machine with Gaussian Elastic Metric Kernel. In: Proceedings of the 2010 20th International Conference on Pattern Recognition (ICPR'10), 23–26 Aug. 2010. pp 29–32. doi:10.1109/icpr.2010.16

  18. Cuturi M, Vert J, Birkenes O, Matsui T (2007) A Kernel for Time Series Based on Global Alignments. In: Acoustics, Speech and Signal Processing. IEEE International Conference on, 15–20 April 2007. pp 413–416. doi:10.1109/icassp.2007.366260

  19. Lei HS, Sun BY (2007) A Study on the Dynamic Time Warping in Kernel Machines. In: Signal-Image Technologies and Internet-Based System, third International IEEE Conference on, 16–18 Dec. 2007. pp 839–845. doi:10.1109/sitis.2007.112

  20. Gudmundsson S, Runarsson TP, Sigurdsson S (2008) Support vector machines and dynamic time warping for time series. In: 2008 I.E. international joint conference on neural networks. IEEE Int Joint Conf Neural Netw (IJCNN) 1–8:2772–2776. doi:10.1109/ijcnn.2008.4634188

    Google Scholar 

  21. Ferreira CD, Santos JA, Torres RS, Goncalves MA, Rezende RC, Fan W (2011) Relevance feedback based on genetic programming for image retrieval. Pattern Recogn Lett 32(1):27–37. doi:10.1016/j.patrec.2010.05.015

    Article  Google Scholar 

  22. Stejic Z, Takama Y, Hirota K (2003) Genetic algorithm-based relevance feedback for image retrieval using local similarity patterns. Inf Process Manag 39(1):1–23. doi:10.1016/s0306-4573(02)00024-9

    Article  MATH  Google Scholar 

  23. Zhou XS, Garg A, Huang TS (2005) Nonlinear variants of biased discriminants for interactive image retrieval. Vis, Image Signal Process, IEE Proc 152(6):927–936. doi:10.1049/ip-vis:20045190

    Article  Google Scholar 

  24. Kohavi R, Sommerfield D (1995) Feature Subset Selection Using the Wrapper Method: Overfitting and Dynamic Search Space Topology. In: The First International Conference on Knowledge Discovery and Data Mining, 20–21 Aug. pp 192–197

  25. Chao MW, Lin CH, Assa J, Lee TY (2012) Human motion retrieval from hand-drawn sketch. IEEE Trans Vis Comput Graph 18(5):729–740. doi:10.1109/tvcg.2011.53

    Article  Google Scholar 

  26. Sakoe H, Chiba S (1978) Dynamic programming algorithm optimization for spoken word recognition. IEEE Trans Acoust Speech Signal Process 26(1):43–49. doi:10.1109/tassp.1978.1163055

    Article  MATH  Google Scholar 

  27. Freund Y, Iyer R, Schapire RE, Singer Y (2003) An efficient boosting algorithm for combining preferences. J Mach Learn Res 4:933–969

    MathSciNet  Google Scholar 

  28. Kovar L, Gleicher M (2004) Automated extraction and parameterization of motions in large data sets. ACM Trans Graph 23(3):559–568. doi:10.1145/1015706.1015760

    Article  Google Scholar 

  29. Deng Z, Gu Q, Li Q (2009) Perceptually consistent example-based human motion retrieval. In: Proceedings of the 2009 symposium on Interactive 3D graphics and games, Boston, Massachusetts, ACM, 1507181, pp 191–198. doi:10.1145/1507149.1507181

  30. Forbes K, Fiume E (2005) An efficient search algorithm for motion data using weighted PCA. In: Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation, Los Angeles, California. ACM, 1073377, pp 67–76. doi:10.1145/1073368.1073377

  31. Wu S, Xia S, Wang Z, Li C (2009) Efficient motion data indexing and retrieval with local similarity measure of motion strings. Vis Comput 25(5–7):499–508. doi:10.1007/s00371-009-0345-1

    Article  Google Scholar 

  32. Pradhan GN, Prabhakaran B (2009) Indexing 3-D human motion repositories for content-based retrieval. IEEE Trans Inf Technol Biomed 13(5):802–809. doi:10.1109/titb.2009.2021262

    Article  Google Scholar 

  33. Demuth B, Röder T, Müller M, Eberhardt B (2006) An information retrieval system for motion capture data. In: The 28th European conference on Advances in Information Retrieval, 10–12 Apr. pp 373–384. doi:10.1007/11735106_33

  34. Kim TH, Park SI, Shin SY (2003) Rhythmic-motion synthesis based on motion-beat analysis. ACM Trans Graph 22(3):392–401. doi:10.1145/882262.882283

    Article  Google Scholar 

  35. Liu TY (2009) Learning to rank for information retrieval. Found Trends Inf Retr 3(3):225–331. doi:10.1561/1500000016

    Article  Google Scholar 

  36. Crucianu M, Ferecatu M, Boujemaa N (2004) Relevance feedback for image retrieval: a short survey. In State of the Art in Audiovisual Content-Based Retrieval, Information Universal Access and Interaction including Datamodels and Languages (DELOS2 Report)

  37. Barbič J, Safonova A, Pan J-Y, Faloutsos C, Hodgins JK, Pollard NS (2004) Segmenting motion capture data into distinct behaviors. In: The 2004 Graphics Interface Conference, 17–19 May. pp 185–194

  38. Shen L, Chung MK (2006) Large-Scale Modeling of Parametric Surfaces Using Spherical Harmonics. In: Proceedings of the Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’06), pp 294–301. doi:10.1109/3dpvt.2006.86

  39. Friedman J, Hastie T, Tibshirani R (2000) Additive logistic regression: a statistical view of boosting. Ann Stat 28(2):337–374. doi:10.1214/aos/1016218223

    Article  MathSciNet  MATH  Google Scholar 

  40. Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182

    MATH  Google Scholar 

  41. Jungong H, Ling S, Dong X, Shotton J (2013) Enhanced computer vision with microsoft kinect sensor: a review. IEEE Trans Cybern 43(5):1318–1334. doi:10.1109/tcyb.2013.2265378

    Article  Google Scholar 

  42. Yang X, Tian Y (2014) Effective 3D action recognition using Eigen joints. J Vis Commun Image Represent 25(1):2–11. doi:10.1016/j.jvcir.2013.03.001

    Article  MathSciNet  Google Scholar 

  43. Lu X, Chia-Chih C, Aggarwal JK (2012) View invariant human action recognition using histograms of 3D joints. In: Computer Vision and Pattern Recognition Workshops (CVPRW), IEEE Computer Society Conference on, 16–21 June 2012. pp 20–27. doi:10.1109/cvprw.2012.6239233

  44. Reyes M, Dominguez G, Escalera S (2011) Feature weighting in dynamic timewarping for gesture recognition in depth data. In: Computer Vision Workshops (ICCV Workshops), 2011 I.E. International Conference on, 6–13 Nov. pp 1182–1188. doi:10.1109/iccvw.2011.6130384

  45. Kapadia M, Chiang I-k, Thomas T, Badler NI, Joseph T. Kider J (2013) Efficient motion retrieval in large motion databases. In: Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, pp 19–28. doi:10.1145/2448196.2448199

Download references

Acknowledgments

We would like to thank the anonymous reviewers for their valuable comments. This work is supported by National Natural Science Foundation of China (61272219, 61100110 and 61321491), Program for New Century Excellent Talents of Ministry of Education of China (NCET-04-0460), Science and Technology Plan of Jiangsu Province (BE2010072, BE2011058, BY2012190 and BY2013072-04), Innovation Foundation of State Key Lab for Novel Software Technology of China (ZZKT2013A12).

Author information

Authors and Affiliations

  1. State Key Lab for Novel Software Technology, Nanjing University, Nanjing, 210023, China

    Songle Chen, Zhengxing Sun, Yan Zhang & Qian Li

Authors
  1. Songle Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhengxing Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qian Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhengxing Sun.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, S., Sun, Z., Zhang, Y. et al. Relevance feedback for human motion retrieval using a boosting approach. Multimed Tools Appl 75, 787–817 (2016). https://doi.org/10.1007/s11042-014-2325-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-014-2325-3

Keywords

Search

Navigation