Abstract
In this work, a formalization of the problem of analyzing motion flows in a video sequence is proposed. A method for solving the problem that leverages the optical-flow field and separates the motion of the camera from that of the objects is given. The optical flow is used to generate motion maps that help analyze and describe movements in any area of interest.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
REFERENCES
G. Birkhoff and J. D. Lipson, “Heterogeneous algebras,” J. Combin. Theory 8, 115‒133 (1970). https://doi.org/10.1016/S0021-9800(70)80014-X
D. J. Butler, J. Wulff, G. B. Stanley, and M. J. Black, “A naturalistic open source movie for optical flow evaluation,” in Computer Vision–ECCV 2012, Ed. by A. Fitzgibbon, S. Lazebnik, P. Perona, Y. Sato, and C. Schmid, Lecture Notes in Computer Science, vol. 7577 (Springer, Berlin, 2012). https://doi.org/10.1007/978-3-642-33783-3_44
U. Grenander, General Pattern Theory: A Mathematical Study of Regular Structures (Clarendon, Oxford, 1993).
I. B. Gurevich and V.V. Yashina, “Operations of descriptive image algebras with one ring,” Pattern Recognit. Image Analysis. 16, 298–328 (2006). https://doi.org/10.1134/S1054661806030035
I. B. Gurevich, “A class of descriptive logical transformations in problems of image understanding,” in Abstracts 5th Int. Seminar on Digital Image Processing and Comput. Graphics, Ed. by A. Fitzgibbon, S. Lazebnik, P. Perona, Y. Sato, and C. Schmid (Samara, 1994).
A. Dosovitskiy, P. Fischer, E. Ilg, P. Häusser, C. Hazirbas, V. Golkov, P. van der Smagt, D. Cremers, and T. Brox, “FlowNet: Learning optical flow with convolutional networks”, in IEEE Int. Conf. on Computer Vision (ICCV), Santiago, 2015 (IEEE, 2015), pp. 2758‒2766. https://doi.org/10.1109/ICCV.2015.316
G. Farnebäck, “Two-frame motion estimation based on polynomial expansion”, in Image Analysis, Ed. by J. Bigun and T. Gustavsson, Lecture Notes in Computer Science, vol. 2749 (Springer, Berlin, 2003), pp. 363–370 (2003). https://doi.org/10.1007/3-540-45103-X_50
Yu. I. Zhuravlev, “On algebraic approach to solution of problems of pattern recognition or classification,” Probl. Kibern. 33, 5–68 (1978).
Yu. I. Zhuravlev, Selected Works (Magister, Moscow, 1998).
D. Patel and S. Upadhyay, “Optical flow measurement using Lucas kanade method,” Int. J. Comput. Appl. 61, 6‒10 (2013). https://doi.org/10.5120/9962-4611
A. Ranjan and M. J. Black, “Optical flow estimation using a spatial pyramid network,” in IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Honolulu, Hawaii, 2017 (IEEE, 2017), pp. 4161–4170. https://doi.org/10.1109/CVPR.2017.291
S. Rao and H. Wang, “Optical flow estimation via weighted guided filtering with non-local steering kernel,” Visual Comput. (2021). https://doi.org/10.1007/s00371-021-02349-2
G. X. Ritter, J. N. Wilson, and J. L. Davidson, “Image algebra: An overview,” Comput. Vision, Graphics, Image Process. 49, 297‒331 (1990). https://doi.org/10.1016/0734-189X(90)90106-6
G. X. Ritter and J. N. Wilson, Handbook of Computer Vision Algorithms in Image Algebra, 2nd ed. (CRC Press, Boca Raton, Fla., 2001). https://doi.org/10.1201/9781420042382
S. R. Sternberg, “Language and architecture for parallel image processing,” in Proc. Conf. on Pattern Recognition in Practice, Amsterdam, 1980 (1980), pp. 35‒44.
D. Q. Sun, S. Roth, and M. J. Black, “Secrets of optical flow estimation and their principles,” in IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, San Francisco, 2010 (IEEE, 2010), pp. 2432‒2439. https://doi.org/10.1109/CVPR.2010.5539939
D. Sun, X. Yang, M.-Y. Liu, and J. Kautz, “PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume,” in IEEE/CVF Conf. on Computer Vision and Pattern Recognition, Salt Lake City, Utah, 2018 (IEEE, 2018), pp. 8934‒8943. https://doi.org/10.1109/CVPR.2018.00931
Funding
This research is partially funded by the Development and Research of Descriptive Methods of Dynamic Image Analysis for Automation of Diagnostic Procedures RFBR/BRFBR project (project no. 20-57-00025/F20R-134); special funds for basic scientific research in Provincial Universities from Zhejiang Shuren University (project nos. 2021XZ018 and 2022XZ014); the Natural Science Foundation of Zhejiang Province, China (project no. LQ21F020025); and the National Foreign Expert Program (project nos. G20200216025, G2021016001L, G2021016002L, G2021016028L).
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
COMPLIANCE WITH ETHICAL STANDARDS
This article is a completely original work of its authors; it has not been published before and will not be sent to other publications until the PRIA Editorial Board decides not to accept it for publication.
Conflict of Interests
The process of writing and the content of the article do not give grounds for raising the issue of a conflict of interest.
Additional information
Olga V. Nedzved. Graduated from Belarus State University in 1997. She received her PhD degree in Computer Science from the United Institute of Informatics Problems of the National Academy of Sciences, Minsk, Belarus. She is an associate professor of the Biological Faculty of Belarusian State University. Scientific interests: analysis of medical images, mathematical simulation of biological processes, biophysics. Author of more than 50 publications.
Sergey V. Ablameyko. Born in 1956. He received his DipMath in 1978, PhD in 1984, D.Sc. in 1990, and became a professor in 1992. Rector (President) of Belarusian State University from 2008 to 2017 and professor since 2017. He is on the editorial board of Pattern Recognition and Image Analysis, Supercomputers, and many other international and national journals. He is a Fellow of the International Association for Pattern Recognition, Academician of Belarusian Engineering Academy, Academician of National Academy of Sciences of Belarus, Academician of the European Academy, Russian Academy of Natural Sciences, Russian Space Academy, and many others. He was the first vice president of the International Association for Pattern Recognition (IAPR) (2006–2008) and the president of the Belarusian Association for Image Analysis and Recognition. He is the chairman of the BSU Academic Council that awards PhD and D.Sc. degrees. He was awarded the State Prize of Belarus (the highest national scientific award) in 2002, Skoryna Belarusian Medal, Russian Award of Friendship, and many other awards. Scientific interests: image analysis, pattern recognition, digital geometry, knowledge-based systems, geographical information systems, and medical imaging. Author of more than 600 publications.
Igor’ B. Gurevich. Born on August 24, 1938. He graduated from Moscow Power Engineering Institute in 1961 (Automatic Control and Electrical Engineering) and defended his Candidate’s dissertation in physics and mathematics at the Moscow Institute of Physics and Technology in 1975. Leading researcher at the Federal Research Center Computer Science and Control of the Russian Academy of Sciences. He has been working since 1960 as an engineer, researcher, and lecturer in industry, research institutions, medicine, and universities, and, since 1985, he has been working in the USSR/Russian Academy of Sciences. Area of expertise: mathematical theory of image analysis, image mining, image understanding, mathematical theory of pattern recognition, theoretical computer science, medical informatics, applications of pattern-recognition and image-analysis techniques in biology, medicine, and in automation of scientific research, and knowledge-based systems. Gurevich suggested, proved, and developed—with his pupils—the descriptive approach to image analysis and recognition (DAIA). Within DAIA, a new class of image algebra was introduced, defined, and studied (descriptive image algebras); new types of image models were introduced, classified, and studied; axioms of descriptive theory of image analysis were introduced; a common model of the image-recognition process was defined and studied; new settings of image-analysis and -recognition problems were introduced; the notion of image equivalence was introduced and studied; new classes of image-recognition algorithms were defined and studied; and an image formalization space was introduced, defined, and studied. These results have been used to develop software kits for image analysis and recognition and for the solution of important and difficult applied problems of automated biomedical image analysis. Gurevich is the author of two monographs; 307 papers in peer reviewed journals and proceedings indexed in Web of Science, Scopus, and Russian Science Citation Index on the platform of Web of Science; and 31 invited papers at international conferences. He is holder of 8 patents. Web of Science: 22 papers; Scopus: 76 papers, 287 citations in 148 documents; Hirsh index is 10; Russian Science Citation Index on the platform of Web of Science: 129 papers; 910 citations; Hirsh index is 11. He is the vice-chairman of the National Committee for Pattern Recognition and Image Analysis of the Russian Academy of Sciences, Member of the International Association for Pattern Recognition (IAPR) Governing Board (representative of Russia), and the IAPR Fellow. He has been the primary investigator of 63 R&D projects as part of national and international research programs. He is the Vice-Editor-in-Chief of Pattern Recognition and Image Analysis, an international journal of the RAS, member of the editorial boards of several international scientific journals, and member of the program and technical committees of many international scientific conferences. He has teaching experience at Moscow State University, Russia (Assistant Professor); Dresden Technical University, Germany (Visiting Professor); and George Mason University, USA (Research Fellow). He has supervised six PhD students and many graduate and master students.
Vera V. Yashina. Born September 13, 1980. Diploma Mathematician, Moscow State University (2002). Cand. Sci. (Phys.–Math.) (Theoretical Foundations of Informatics), 2009, Dorodnicyn Computing Center of the Russian Academy of Sciences, Moscow. Leading researcher at the Department for Recognition, Security, and Analysis of Information at the Federal Research Center Computer Science and Control of the Russian Academy of Sciences. She has been working at the Russian Academy of Sciences since 2001. Scientific expertise: mathematical theory of image analysis, image algebras, models, and medical informatics. Her principal results were obtained in mathematical theory of image analysis: descriptive image algebras with one ring were defined, classified and studied; a new topological image formalization space was specified and studied; descriptive generating trees were defined, classified, and studied. These results were applied in biomedical image analysis. She is the scientific secretary of the National Committee for Pattern Recognition and Image Analysis of the Presidium of the Russian Academy of Sciences. She is a member of the Educational and Membership Committees of the International Association for Pattern Recognition. She is a vice chair of Technical Committee no. 16 on Algebraic and Discrete Mathematical Techniques in Pattern Recognition and Image Analysis of the International Association for Pattern Recognition. She has been a member of many R&D projects as a part of national and international research programs. Member of Editorial Board of Pattern Recognition and Image Analysis, an international journal of the RAS. Author of 79 papers in peer reviewed journals, conference and workshop proceedings. Web of Science: 11 papers; Hirsh index is 4; Scopus: 40 papers, 162 citations in 75 papers; Hirsh index is 8; Russian Science Citation Index on the platform of Web of Science: 56 papers; 255 citations; Hirsh index is 9. She was awarded several times for the best young scientist papers presented at international conferences. Teaching experience: Moscow State University. She has supervised several graduate and master students.
Tiaojuan Ren, Master of Engineering, Professor. Received the B.S. degree from Zhejiang University in 1987 and the M.S. degree from Korea Giant Buddha University in 2005. She is the leader of Information and Communication Engineering in Zhejiang Province. Her research interests include network security, image processing, and wireless-mobile communication. She has presided and participated in many scientific research projects and has published several dozen academic articles.
Fangfang Ye born in 1980, Anhui province, China. She received her Ph.D. degrees in Control Theory and Control Engineering from Zhejiang University, Hangzhou, China in 2014. Her research interests focus on image segmentation, medical imaging processing, and deep learning. She is also a lecturer at Zhejiang Shuren University, Hangzhou, China. She has participated in many scientific research projects and has published dozens of academic articles.
Translated by M. Talacheva
Rights and permissions
About this article
Cite this article
Nedzved, O.V., Ablameyko, S.V., Gurevich, I.B. et al. Describing Motion of Dynamic Objects for a Moving Camera. Pattern Recognit. Image Anal. 32, 301–311 (2022). https://doi.org/10.1134/S1054661822020158
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1134/S1054661822020158