Skip to main content
SpringerLink
Log in

Swarm intelligence based object tracking

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

A Correction to this article was published on 05 May 2023

This article has been updated

Abstract

Though object tracking is a very old problem still there are several challenges to be solved; for instance, variation of illumination of light, noise, occlusion, sudden start and stop of moving object, shading etc. In this paper we propose a dual approach for object tracking based on optical flow and swarm Intelligence. The optical flow based KLT tracker, tracks the dominant points of the target object from first frame to last frame of a video sequence; whereas swarm Intelligence based PSO tracker simultaneously tracks the boundary information of the target object from second frame to last frame of the same video sequence. The boundary information of the target object is captured by the polygonal approximation of the same. The dual approach to object tracking is inherently robust with respect to the above stated problems. We compare the performance of the proposed dual tracking algorithm with several benchmark datasets and in most of the cases we obtain superior results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26

Similar content being viewed by others

Data Availability

In this research work we use 3 datasets for our experiment. Below we provide a table which include those 3 dataset’s availability information along with their location. This data availability information properly summarize the data we use in our approach.

Availability of data

Data availability statement

Policy

Dataset-1

The data that support the findings of this study are openly available in Online Object Tracking: A Benchmark at http://cvlab.hanyang.ac.kr/tracker_benchmark/, [86]

All

Dataset-2

The data that support the findings of this study are openly available in Long-Term Visual Object Tracking Benchmark at https://amoudgl.github.io/tlp/, [48]

All

Dataset-3

The data that support the findings of this study are openly available in Object Tracking Evaluation 2012 from The KITTI Vision Benchmark Suite at http://www.cvlibs.net/datasets/kitti/eval_tracking.php, [27]

All

Change history

References

  1. Ackerman Phillip L (2000) Domain-specific knowledge as the dark matter of adult intelligence: Gf/Gc, personality and interest correlates. The Journals of Gerontology Series B: Psychological Sciences and Social Sciences, 69–84

  2. Ackerman Phillip L, Beier ME, Boyle MO (2005) Working memory and intelligence: The same or different constructs?. Psychological bulletin, New York

    Google Scholar 

  3. Ahmed H, Glasgow J (2012) Swarm intelligence: concepts, models and applications. School of computing, Queens University technical report, New York

    Google Scholar 

  4. Ashton MC, Lee K (2006) Minimally biased g-loadings of crystallized and non-crystallized abilities. Intelligence :469–477

  5. Aslani S, Mahdavi-Nasab H (2013) Optical flow based moving object detection and tracking for traffic surveillance. Int J Electr Comput Energetic Electron Commun Eng 84:1252–1256

    Google Scholar 

  6. Babenko B, Yang M-H (2011) Robust object tracking with online multiple instance learning. IEEE Trans Pattern Anal Mach Intell 61:1619–1632

    Article  Google Scholar 

  7. Barron JL, Fleet DJA, Beauchemin SS (1994) Performance of optical flow techniques. Int J Comput Vis 59:43–77

    Article  Google Scholar 

  8. Bertinetto L, Valmadre J, Henriques JF, Vedaldi A, Torr PHS (2016) Fully-convolutional siamese networks for object tracking. Eur Conf Comput Vis 59:850–865

    Google Scholar 

  9. Bertinetto L, Valmadre J, Henriques JF, Vedaldi A, Torr PHS (2021) Fully-convolutional siamese networks for object tracking. arXiv:1606.09549v3

  10. Birchfield S (2001) Derivation of kanade-lucas-tomasi tracking equation, unpublished notes. 59

  11. Blair C (2006) How similar are fluid cognition and general intelligence? a developmental neuroscience perspective on fluid cognition as an aspect of human cognitive ability. Behav Brain Sci :109–125

  12. Kwolek B (2013) Multi-object tracking using particle swarm optimization on target interactions. Springer, Berlin, pp 109–125

  13. Brox T, Rosenhahn B, Cremers D, Seidel H-P (2006) High accuracy optical flow serves 3-D pose tracking: exploiting contour and flow based constraints. In: European conference on computer vision. Springer, pp 98–111

  14. Buxton B, Buxton H (1984) Computation of optic flow from the motion of edge features in image sequences. Image Vis Comput 2(2):59–75

    Article  Google Scholar 

  15. Cattell R (1963) Theory of fluid and crystallized intelligence: A critical experiment. J Educ Psychol 54(1):1–22

    Article  Google Scholar 

  16. Cattell RB (1987) Theory of Intelligence: Its structure, growth and action. Elsevier

  17. Chen Z, Jianzhong T, Tang L (2011) Tracking of moving object based on optical flow detection. Computer Science and Network Technology (ICCSNT) :1096–1099

  18. Chen C, Li D (2021) Research on the detection and tracking algorithm of moving object in image based on computer vision technology. Wireless Communications and Mobile Computing

  19. Chen C-H, Yan M-C (2011) PSO-Based multiple people tracking. Communications in Computer and Information Science :1096–1099

  20. Cui Jinshi, Ye Liu, Xu Yuandong, Zhao H, Zha H (2013) Tracking generic human motion via fusion of low-and high-dimensional approaches. Elsevier :996–1002

  21. De Melo VV, Delbem ACB (2012) Investigating smart sampling as a population initialization method for differential evolution in continuous problems. Information Sciences :36–53

  22. Eberhart R, Kennedy J (1995) A new optimizer using particle swarm theory. In: MHS’95. Proceedings of the sixth international symposium on micro machine and human sciences, pp 39–43

  23. Everingham M, Gool LV, Williams CKI, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. In: MHS’95. International journal of computer vision, pp 303–338

  24. Fakheredine K (2012) A fast-multiplying PSO algorithm for real-time multiple object tracking. International journal of computer Applications :1–6

  25. Freeman H (1961) On the encoding of arbitrary geometric configurations. IRE Trans Electr Comput: 260–268

  26. Geiger A, Lenz P, Urtasun R (2012a) Are we ready for autonomous driving? the kitti vision benchmark suite. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3354–3361

  27. Geiger A, Lenz P, Stiller C, Urtasun R (2012b) Object tracking evaluation 2012 from the KITTI vision benchmark suite. In: Conference on computer vision and pattern recognition (CVPR)

  28. Gerard Wolff J (2013) The SP theory of intelligence: an overview. Information :283–341

  29. Grabner H, Leistner C, Bischof H (2008) Semi-supervised on-line boosting for robust tracking. European Conference on Computer Vision :234–247

  30. Henriques JF, Caseiro R, Martins P, Batista J (2012) Exploiting the circulant structure of tracking-by-detection with kernels. European Conference on Computer Vision :702–715

  31. Horn JL (1968) Organization of abilities and the development of intelligence. Psychological Review :242

  32. Horn BKP, Schunck BG (1993) Determining optical flow: a retrospective. Artif Intell (Elsevier) 59:81–87

    Article  MATH  Google Scholar 

  33. Hsu C, Dai G-T (2012) Multiple object tracking using particle swarm optimization. World Academy of Science Engineering and Technology: 41–44

  34. Husseini S (2017) A survey of optical flow techniques for object tracking. World Academy of Science Engineering and Technology :41–44

  35. Jepson A, Blacki MJ (1993) Mixture models for optical flow computation. In: Proceedings of IEEE conference on computer vision and pattern recognition

  36. Jeremy R, Gray CC, Braver TS (2003) Neural mechanisms of general fluid intelligence. Nature Neuroscience :316–318

  37. Jia X, Yang H, Lu M-H (1993) Visual tracking via adaptive structural local sparse appearance model. Computer vision and pattern recognition (CVPR) :1822–1829

  38. Jiang M, Luo YP, Yang SY (2007) Stochastic convergence analysis and parameter selection of the standard particle swarm optimization algorithm. Information Processing Letters :8–16

  39. John V, Trucco E, Ivekovic S (2010) Markerless human articulated tracking using hierarchical particle swarm optimisation. Image and Vision Computing :1530–1547

  40. Kale K, Dhulekar SP (2015) Pravin moving object tracking using optical flow and motion vector estimation. IEEE :1–6

  41. Kim SW, Yun K, Kwang M, Kim SJ, Choi JY (2012) Beyond semi-supervised tracking: Tracking should be as simple as detection, but not simpler than recognition. Machine Vision and Applications :1015–1028

  42. LEE R, Chen I-Y (2020) The time complexity analysis of neural network model configurations. In: 2020 International conference on mathematics and computers in science and engineering (MACISE)

  43. Liu Y, Cui J, Zhao H, Zha H (2012) Fusion of low-and high-dimensional approaches by trackers sampling for generic human motion tracking. In: Proceedings of the 21st international conference on pattern recognition (ICPR2012), IEEE, pp 899–901

  44. Marcus G (2018) Deep learning: a critical appraisal. arXiv:1801.00631

  45. McGrew KS (2005) The Cattell-Horn-Carroll theory of cognitive abilities: past, present, and future. Contemporary intellectual assessment: theories, tests, and issues :136–181

  46. Melo VV (2009) Thyago SPC duque and alexandre CB alexandre, efficiency enhancement of ECGA through population size management. In: Ninth international conference on intelligent systems design and applications, ISDA’09, pp 19–24

  47. Misra R, Ray KS (2021) Dual approach for object tracking based on optical flow and swarm intelligence. arXiv:1808.08186

  48. Moudgil A, Gandhi V (2017) Long-Term Visual object tracking benchmark, ISDA’09. arXiv:1712.01358

  49. Nam Hyeonseob, Han Bohyung (2016) Learning multi-domain convolutional neural networks for visual tracking. Computer Vision and Pattern Recognition (CVPR) :4293–4302

  50. Nam H, Han B (2016) Learning multi-domain convolutional neural networks for visual tracking. arXiv:1510.07945v2

  51. Nguyen XS, Dubuisson S, Gonzale C (2013) Hierarchical annealed particle swarm optimization for articulated object tracking. Computer Analysis of Images and Patterns :319–326

  52. Perez P, Vermaak J, Gangnet M (2002) Color-based probabilistic tracking. In: European conference on computer vision, pp 661–675

  53. Potok T (2016) Neuromorphic computing architectures, models, and applications. DOE workshop report, Oak Ridge National Laboratory

  54. Ray KS (2021) Quest for I(Intelligence) in AI(Artificial Intelligence): a non-elusive attempt. Artifical 833 Intelligence-Latest Advances, New paradigms and Novel Applications, infotech

  55. Ray BK, Ray KS (1992) Detection of significant points and polygonal approximation of digitized curves. Pattern Recogn Lett 59:443–452

    Article  Google Scholar 

  56. Ray KS, Ray BK (2013) Polygonal approximation and scale-space analysis of closed digital curves. CRC Press :661–675

  57. Richards M, Ventura D (2004) Choosing a starting configuration for particle swarm optimization. IEEE Int Joint Conf Neural :2309–2312

  58. Rohler AB, Chen S (2011) An analysis of sub-swarms in multi-swarm systems. Australasian Joint Conference on Artificial Intelligence :271–280

  59. Rymut B, Kwolek B (2014) Real-time multiview human pose tracking using GPU-accelerated particle swarm optimization. Concurr Comput Pract Experience 27(6):1551–1563. https://doi.org/10.1002/cpe.3329

    Article  Google Scholar 

  60. Schneider WJ, McGrew KS (2012) The Cattell-Horn-Carroll model of intelligence. In: Flanagan DP, Harrison PL (eds) Contemporary intellectual assessment: Theories, tests, and issues, pp 99–144

  61. Schwarz LA, Mkhitaryan A, Mateus D, Navab N (2012) Human skeleton tracking from depth data using geodesic distances and optical flow. Image and Vision Computingl :217–226

  62. Schwarz LA, Mkhitaryan A, Mateus D, Navab N (2012) Human skeleton tracking from depth data using geodesic distances and optical flow. Image and Vision Computingl :217–226

  63. Sevilla-Lara L, Learned-Miller E (2012) Distribution fields for tracking. In: Conference on computer vision and pattern recognition

  64. Shi Y, Eberhart RC (1998) A modified particle swarm optimizer. In: 1998 IEEE international conference on evolutionary computation proceedings. IEEE world congress on computational intelligence (Cat. No.98TH8360), pp 69–73

  65. Shi J, Tomasi V (1994) Good features to track, computer vision and pattern recognition. In: Proceedings CVPR’94., 1994 IEEE computer society conference, pp 593–600

  66. Shin J, Kim S, Kang S, Lee S-w, Paik J, Abidi B, Abidi M (1994) Optical flow-based real-time object tracking using non-prior training active feature model. Real-Time Imaging, Elsevier :204–218

  67. Shin J, Kim S, Kang S, Lee S-W, Paik J, Abidi B, Abidi M (2005) Optical flow-based real-time object tracking using non-prior training active feature model. Real-Time Imaging Elsevier

  68. Simpson AJR (2015) On-the-fly learning in a perpetual learning machine, arXiv:1509.00913

  69. Song Y, Ma C, Gong L, Zhang J, Lau RWH, Yang M-H (2012) Detection of moving objects with a moving camera using non-panoramic background model. Mach Vis Appl :1015–1028

  70. Song Y, Ma C, Gong L, Zhang J, Lau RWH, Yang M-H (2017) Crest: convolutional residual learning for visual tracking. IEEE International Conference on Computer Vision (ICCV) :2574–2583

  71. Song Y, Ma C, Gong L, Zhang J, Lau R, Yang M-H (2017) CREST: convolutional residual learning for visual tracking. arXiv:1708.00225v1

  72. Stiefelhagen R, Bernardin K, Bowers R, Garofolo J, Mostefa D, Soundararajan P (2006) The CLEAR 2006 evaluation, international evaluation workshop on classification of events. Activities and Relationships :1–44

  73. Stiefelhagen R, Bernardin K, Bowers R, Garofolo J, Mostefa D, Soundararajan P (2010) Dense point trajectories by GPU-accelerated large displacement optical flow. European conference on computer vision :438–451

  74. Tomasi C, Kanade T (1991) Detection and tracking of point feature, school of computer science. Carnegie Mellon Univ, Pittsburgh

    Google Scholar 

  75. Tsutsui H, Miura J, Shirai Y (2001) Optical flow-based person tracking by multiple cameras. In: Conference documentation international conference on multisensor fusion and integration for intelligent systems

  76. Wolff JG (2004) Unifying computing and cognition: the SP theory and its applications. arXiv preprint cs/04010097

  77. Wolff JG (2014) Application of the SP theory of intelligence to the understanding of natural vision and the development of computer vision. SpringerPlus :552

  78. Wolff JG (2014) Autonomous robots and the SP theory of intelligence. IEEE Access :1629–1651

  79. Wolff JG (2016) Commonsense reasoning, commonsense knowledge, and the SP theory of intelligence, arXiv:1609.07772

  80. Wolff JG (2016) Information compression, multiple alignment, and the representation and processing of knowledge in the brain. Frontiers in psychology

  81. Wolff JG (2016) The SP theory of intelligence: its distinctive features and advantages. IEEE Access 4:216–246

    Article  Google Scholar 

  82. Wolff JG (2017) Strengths and potential of the SP theory of intelligence in general. human-like artificial intelligence. IEEE Access 4:216–246

    Article  Google Scholar 

  83. Wolff JG (2018) Solutions to problems with deep learning. arXiv:1801.05457

  84. Wu W-Y (2003a) An adaptive method for detecting dominant points. Pattern Recognition :2231–2237

  85. Wu W-Y (2003b) Dominant point detection using adaptive bending value. Image and Vision Computing

  86. Wu Y, Jongwoo L, Ming-Hsuan Y (2013) Online Object Tracking: A Benchmark. In: Conference on computer vision and pattern recognition, IEEE, p 4

  87. Xia G, Ludwig SA (2017) Sequential particle swarm optimization for visual tracking, IEEE Conference on Computer Vision and Pattern Recognition, Image and Vision Computing

  88. Xiaoqin Z, Weiming H, Li X, Mingliang Z (2008) IEEE Conference on Computer Vision and Pattern Recognition. Image and Vision Computing

  89. Xueyan L, Zheng X (2012) Real-time compressive tracking. European conference on computer vision :864–877

  90. Xueyan L, Zheng X (2015) Swarm size and inertia weight selection of particle swarm optimizer in system identification. In: 2015 4th international conference on computer science and network technology (ICCSNT), pp 1554–1556

  91. Yun S, Choi J, Yoo Y, Yun K, Choi JY (2017) Action-decision networks for visual tracking with deep reinforcement learning. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 1349–1358, DOI https://doi.org/10.1109/CVPR.2017.148

  92. Yun S, Choi J, Yoo Y, Yun Choi K, Young J (2017) Action-decision networks for visual tracking with deep reinforcement learning. Computer Vision Foundation

  93. Zhang BJ (2014) Monocular video human motion tracking based on Hybrid PSO. J Multimed 59:892–899

    Google Scholar 

  94. Zhang J, Ma S, Sclaroff S (2014) MEEM: robust tracking via multiple experts using entropy minimization. European conference on computer vision pp 188–203

  95. Zheng Y, Meng Y (2008) Swarm intelligence based dynamic object tracking. IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence)

  96. Zheng Y, Yan M (2007) Adaptive object tracking using particle swarm optimization. International Symposium on Computational Intelligence in Robotics and Automation: 188–203

Download references

Author information

Author notes

    Authors and Affiliations

    1. S.A.Jaipuria College, 10 Raja Naba Krishna Street, Kolklata, West Bengal, India

      Rajesh Misra

    2. GLA University, 17KM Stone, NH2, Mathura, 281406, UP, India

      Kumar S. Ray

    Authors
    1. Rajesh Misra
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Kumar S. Ray
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to Rajesh Misra.

    Ethics declarations

    Conflict of interest

    The authors of this article certify that there is NO conflict of Interest with any other party financial or non-financial in the subject matter or materials discussed in this manuscript.

    Additional information

    Publisher’s note

    Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

    Kumar S. Ray contributed equally to this work.

    The original online version of this article was revised: The author name of reference [12] was incorrectly listed as ”Bogdan K” instead of ”Kwolek B” in the original publication of this article.

    Rights and permissions

    Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Misra, R., Ray, K.S. Swarm intelligence based object tracking. Multimed Tools Appl 82, 28009–28039 (2023). https://doi.org/10.1007/s11042-023-14343-y

    Download citation

    • Received:

    • Revised:

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s11042-023-14343-y

    Keywords

    Search

    Navigation