Abstract
In this paper we address the problem of detecting spatio-temporal interest points in video sequences and we introduce a novel detection algorithm based on the three-dimensional shearlet transform. By evaluating our method on different application scenarios, we show we are able to extract meaningful spatio-temporal features from video sequences of human movements, including full body movements selected from benchmark datasets of human actions and human-machine interaction sequences where the goal is to segment drawing activities in smaller action primitives.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Dahlke, S., Steidl, G., Teschke, G.: The continuous shearlet transform in arbitrary space dimensions. J. Fourier Anal. Appl. 16(3), 340–364 (2010)
Do, M.N., Vetterli, M.: The contourlet transform: an efficient directional multiresolution image representation. Trans. Image Process. 14, 2091–2106 (2005)
Dollár, P., Rabaud, V., Cottrell, G., Belongie, S.: Behavior recognition via sparse spatio-temporal features. In: 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, pp. 65–72. IEEE (2005)
Duan, C., Wang, S., Wang, X.G., Huang, Q.H.: MRI volume fusion based on 3D shearlet decompositions. J. Biomed. Imaging 2014, 4 (2014)
Duval-Poo, M.A., Odone, F., De Vito, E.: Edges and corners with shearlets. IEEE Trans. Image Process. 24(11), 3768–3780 (2015)
Guo, K., Labate, D.: Analysis and detection of surface discontinuities using the 3D continuous shearlet transform. Appl. Comput. Harmonic Anal. 30(2), 231–242 (2011)
Guo, K., Labate, D.: Optimally sparse representations of 3D data with \(C^2\) surface singularities using Parseval frames of shearlets. SIAM J. Math. Anal. 2, 851–886 (2012)
Guo, K., Labate, D.: Optimal recovery of 3D X-ray tomographic data via shearlet decomposition. Adv. Comput. Math. 39(2), 227–255 (2013)
Houska, R., Labate, D.: Detection of boundary curves on the piecewise smooth boundary surface of three dimensional solids. Appl. Comput. Harmonic Anal. 40(1), 137–171 (2016)
Jhuang, H., Serre, T., Wolf, L., Poggio, T.: A biologically inspired system for action recognition. In: 2007 IEEE 11th International Conference on Computer Vision, pp. 1–8 (2007)
Kutyniok, G., Labate, D.: Shearlets. Applied and Numerical Harmonic Analysis. Springer, New York (2012)
Kutyniok, G., Lemvig, J., Lim, W.Q.: Optimally sparse approximations of 3D functions by compactly supported shearlet frames. SIAM J. Math. Anal. 44(4), 2962–3017 (2012)
Kutyniok, G., Lim, W.Q., Reisenhofer, R.: Shearlab 3D: faithful digital shearlet transforms based on compactly supported shearlets. ACM Trans. Math. Softw. 42(1), 5 (2016)
Laptev, I.: On space-time interest points. Int. J. Comput. Vis. 64(2), 107–123 (2005)
Lei, B., Xiongwei, Z., Yunfei, Z., Yang, L.: Video saliency detection using 3D shearlet transform. Multimedia Tools Appl. 75(13), 7761–7778 (2016)
Mallat, S., Zhong, S.: Characterization of signals from multiscale edges. IEEE Trans. Pattern Anal. Mach. Intell. 14, 710–732 (1992)
Negi, P.S., Labate, D.: 3D discrete shearlet transform and video processing. IEEE Trans. Image Process. 21, 2944–2954 (2012)
Rao, C., Yilmaz, A., Shah, M.: View-invariant representation and recognition of actions. Int. J. Comput. Vis. 50(2), 203–226 (2002)
Schuldt, C., Laptev, I., Caputo, B.: Recognizing human actions: a local SVM approach. In: Proceedings of the 17th International Conference on Pattern Recognition, ICPR 2004, vol. 3, pp. 32–36. IEEE (2004)
Taylor, G.W., Fergus, R., LeCun, Y., Bregler, C.: Convolutional learning of spatio-temporal features. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6316, pp. 140–153. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15567-3_11
Wang, H., Ullah, M.M., Klaser, A., Laptev, I., Schmid, C.: Evaluation of local spatio-temporal features for action recognition. In: BMVC 2009-British Machine Vision Conference, pp. 124.1–124.11. BMVA Press (2009)
Willems, G., Tuytelaars, T., Van Gool, L.: An efficient dense and scale-invariant spatio-temporal interest point detector. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5303, pp. 650–663. Springer, Heidelberg (2008). doi:10.1007/978-3-540-88688-4_48
Yi, S., Labate, D., Easley, G.R., Krim, H.: A shearlet approach to edge analysis and detection. IEEE Trans. Image Process. 18(5), 929–941 (2009)
Acknowledgements
The authors would like to thank Alessia Vignolo for providing the drawing data used in the experiments.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Malafronte, D., Odone, F., De Vito, E. (2017). Detecting Spatio-Temporally Interest Points Using the Shearlet Transform. In: Alexandre, L., Salvador Sánchez, J., Rodrigues, J. (eds) Pattern Recognition and Image Analysis. IbPRIA 2017. Lecture Notes in Computer Science(), vol 10255. Springer, Cham. https://doi.org/10.1007/978-3-319-58838-4_55
Download citation
DOI: https://doi.org/10.1007/978-3-319-58838-4_55
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-58837-7
Online ISBN: 978-3-319-58838-4
eBook Packages: Computer ScienceComputer Science (R0)