Skip to main content
SpringerLink
Log in

A Unified Framework for Joint Moving Object Detection and Tracking in the Sky and Underwater

  • Conference paper
  • First Online:
Proceedings of the International Conference on Aerospace System Science and Engineering 2021 (ICASSE 2021)

Part of the book series: Lecture Notes in Electrical Engineering ((LNEE,volume 849))

Included in the following conference series:

Abstract

The ability to detect and locate the moving object in a video is a fundamental procedure in applications of computer vision. However, these tracking methods still face some challenges, and are contradictory among different tasks. In this paper, a unified framework for joint moving object detection and tracking in the sky and underwater is proposed. This framework meets the requirements of two real applications: (i) tracking unmanned aerial vehicle (UAV) in the sky; and (ii) tracking unmanned underwater vehicle (UUV) in water. It consists of three key steps: (i) moving object detection by pixel classification; (ii) data association by blob detection; and (iii) object tracking by efficient convolution operator. Finally, analysis on the accuracy of the proposed framework is provided. Experimental results on real-world datasets and object tracking benchmark (OTB) demonstrate the advantage of the tracking method compared with some state-of-the-art trackers, in terms of accuracy and robustness. In addition, to the best of the authors’ knowledge, there is no previously published work for joint moving target detection and tracking in the sky and underwater.

This work was supported in part by the National Natural Science Foundation of China under Grant 61603249, Grant 61673262, and the Wuhan Second Ship Design and Research Institute.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 229.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 299.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 299.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Zhongliang J, Han P, Yuankai L, Peng D (2018) Non-cooperative target tracking: fusion and control. Algorithms and advances. Springer International Publishing, Berlin

    Google Scholar 

  2. Pan H, Jing Z, Qiao L, Li M (2018) Visible and infrared image fusion using \( \ell \)0-generalized total variation model. Sci China Inf Sci 61(4):049103

    Article  MathSciNet  Google Scholar 

  3. Bolme DS, Beveridge JR, Draper BA, Lui YM (2010) Visual object tracking using adaptive correlation filters. In: IEEE computer society conference on computer vision and pattern recognition 2010, pp 2544–2550

    Google Scholar 

  4. Henriques J, Caseiro R, Martins P, Batista J (2012) Exploiting the circulant structure of tracking-by-detection with kernels 7575:702–715

    Google Scholar 

  5. Danelljan M, Häger G, Khan FS, Felsberg M (2015) Learning spatially regularized correlation filters for visual tracking. In: 2015 IEEE international conference on computer vision (ICCV), 2015, pp 4310–4318

    Google Scholar 

  6. Danelljan M, Hager G, Khan FS, Felsberg M (2017) Discriminative scale space tracking. IEEE Trans Pattern Anal Mach Intell 39(8):1561–1575

    Article  Google Scholar 

  7. Danelljan M, Robinson A, Shahbaz Khan F, Felsberg M (2016) Beyond correlation filters: learning continuous convolution operators for visual tracking. ECCV 9909:472–488

    Google Scholar 

  8. Galoogahi HK, Fagg A, Lucey S (2017) Learning background-aware correlation filters for visual tracking. In: IEEE international conference on computer vision (ICCV) 2017, pp 1144–1152

    Google Scholar 

  9. Danelljan M, Bhat G, Khan FS, Felsberg M (2017) Eco: Efficient convolution operators for tracking. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 6931–6939

    Google Scholar 

  10. Gladh S, Danelljan M, Khan FS, Felsberg M (2016) Deep motion features for visual tracking. In: 2016 23rd international conference on pattern recognition (ICPR), 2016, pp 1243–1248

    Google Scholar 

  11. Wu H, Li W, Li W, Liu G (2020) A real-time robust approach for tracking uavs in infrared videos. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR) workshops

    Google Scholar 

  12. Maki T et al. (2019) Autonomous tracking of sea turtles based on multibeam imaging sonar: toward robotic observation of marine life. In: 12th IFAC conference on control applications in marine systems, robotics, and vehicles CAMS 2019. IFAC-PapersOnLine, vol 52, no 21, pp 86–90

    Google Scholar 

  13. Jiang M, Feng X, Song S, Herrmann JM, Li S (2019) Underwater loop-closure detection for mechanical scanning imaging sonar by filtering the similarity matrix with probability hypothesis density filter. IEEE Access 99:1

    Google Scholar 

  14. Barnich O, Van Droogenbroeck M (2011) Vibe: a universal background subtraction algorithm for video sequences 20(6)

    Google Scholar 

  15. Wu Y, Lim J, Yang M (2013) Online object tracking: a benchmark. In: IEEE conference on computer vision and pattern recognition 2013, pp 2411–2418

    Google Scholar 

  16. Bouwmans T (2011) Recent advanced statistical background modeling for foreground detection: a systematic survey. Recent Patents Comput Sci 4:147–176

    Google Scholar 

  17. Stauffer C, Grimson WEL (1999) Adaptive background mixture models for real-time tracking. In: Proceedings. 1999 IEEE computer society conference on computer vision and pattern recognition (Cat. No PR00149), vol 2, pp 246–252

    Google Scholar 

  18. Lee D-S (2005) Effective gaussian mixture learning for video background subtraction. IEEE Trans Pattern Anal Mach Intell 27(5):827–832

    Article  Google Scholar 

  19. Kim K et al. (2005) Real-time foreground-background segmentation using codebook model. Real-Time Imag 11(3):172–185. Special issue on video object processing

    Google Scholar 

  20. Krungkaew R, Kusakunniran W (2016) Foreground segmentation in a video by using a novel dynamic codebook. In: 2016 13th international conference on electrical engineering/electronics, computer, telecommunications and information technology (ECTI-CON), pp 1–6

    Google Scholar 

  21. Varghese A, Sreelekha G (2017) Sample-based integrated background subtraction and shadow detection. Ipsj Trans Comput Vis Appl 9(1):25

    Article  Google Scholar 

  22. Chang O, Constante P, Gordon A, Singaña M (2017) A novel deep neural network that uses space-time features for tracking and recognizing a moving object J Artif Intell Soft Comput Res (2), in press

    Google Scholar 

  23. Kang K, Li H, Yan J, Zeng X, Yang B, Xiao T, Zhang C, Wang Z, Wang R, Wang X, Ouyang W (2018) T-cnn: tubelets with convolutional neural networks for object detection from videos. IEEE Trans Circuits Syst Video Technol 28(10):2896–2907

    Article  Google Scholar 

  24. Maddalena L, Petrosino A (2008) A self-organizing approach to background subtraction for visual surveillance applications. IEEE Trans Image Process 17(7):1168–1177

    Article  MathSciNet  Google Scholar 

  25. Vijayan M, Raguraman P, Mohan R (2021) A fully residual convolutional neural network for background subtraction. Pattern Recognit Lett 146:63–69

    Article  Google Scholar 

  26. Chen M, Wei X, Yang Q, Li Q, Wang G, Yang M (2018) Spatiotemporal gmm for background subtraction with superpixel hierarchy. IEEE Trans Pattern Anal Mach Intell 40(6):1518–1525

    Article  Google Scholar 

  27. Li XR, Jilkov VP (2010) Survey of maneuvering target tracking. Part ii: motion models of ballistic and space targets. IEEE Trans Aerosp Electron Syst 46(1):96–119

    Google Scholar 

  28. Bertinetto L, Valmadre J, Henriques JF, Vedaldi A, Torr P (2016) Fully-convolutional siamese networks for object tracking 2016:850–865

    Google Scholar 

  29. Mueller M, Smith N, Ghanem B (2016) A benchmark and simulator for uav tracking

    Google Scholar 

  30. Mandal M, Kumar LK, Vipparthi SK (2020) Mor-uav: a benchmark dataset and baselines for moving object recognition in uav videos. Assoc Comput Mach

    Google Scholar 

  31. Danelljan M, Bhat G, Khan FS, Felsberg M (2019) Atom: accurate tracking by overlap maximization. IEEE/CVF conference on computer vision and pattern recognition (CVPR) 2019, pp 4655–4664

    Google Scholar 

  32. Fu C, Zhang Y, Huang Z, Duan R, Xie Z (2019) Part-based background-aware tracking for uav with convolutional features. IEEE Access 7:79 997–80 010

    Google Scholar 

  33. Chopra S, Hadsell R, LeCun Y (2005) Learning a similarity metric discriminatively, with application to face verification. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), vol 1, pp 539–546

    Google Scholar 

  34. Modalavalasa N (2012) An efficient implementation of tracking using kalman filter for underwater robot application. Int J Comput Sci Eng Inf 2(2):67–78

    Google Scholar 

  35. Rao J, Dinesh U, Koteswara Rao S, Jagan B (2017) Active sonar target tracking using extended kalman filter. Int J Pure Appl Math 117(12):301–309

    Google Scholar 

  36. Kumar D (2021) Hybrid unscented kalman filter with rare features for underwater target tracking using passive sonar measurements. Optik - Int J Light Electron Opt 226(3):165813

    Article  Google Scholar 

  37. Jeong TT (2007) Particle phd filter multiple target tracking in sonar image. IEEE Trans Aerosp Electron Syst 43(1):409–416

    Article  MathSciNet  Google Scholar 

  38. Xie S, Chen J, Luo J, Xie P, Tang W (2012) Detection and tracking of underwater object based on forward-scan sonar. In: Intelligent robotics and applications. Springer, Berlin, pp 341–347

    Google Scholar 

  39. Wang J, Shan T, Englot B (2019) Underwater terrain reconstruction from forward-looking sonar imagery. In: International conference on robotics and automation (ICRA) 2019, pp 3471–3477

    Google Scholar 

  40. “Opencv,” https://opencv.org/. Accessed 20 April 2021

  41. Lukežic A, Vojíc T, Zajc LC, Matas J, Kristan M (2017) Discriminative correlation filter with channel and spatial reliability. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), 2017, pp 4847–4856

    Google Scholar 

  42. Li Y, Fu C, Ding F, Huang Z, Lu G (2020) Autotrack: towards high-performance visual tracking for uav with automatic spatio-temporal regularization. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), 2020, pp 11 920–11 929

    Google Scholar 

  43. Huang Z, Fu C, Li Y, Lin F, Lu P (2019) Learning aberrance repressed correlation filters for real-time uav tracking. In: ICCV

    Google Scholar 

  44. Ptz camera. http://www.sectec.com.cn/products/. Accessed 13 April 2021

  45. The blueprint m750d. https://www.blueprintsubsea.com. Accessed 13 April 2021

Download references

Author information

Authors and Affiliations

  1. Shanghai Jiao Tong University, 200240, Shanghai, SH, China

    Xia Wu, Han Pan, Meng Xu & Zhongliang Jing

  2. Xidian University, 710071, Shaanxi Xi’an, China

    Min Bao

Authors
  1. Xia Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Han Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Meng Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhongliang Jing
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Min Bao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Han Pan or Zhongliang Jing .

Editor information

Editors and Affiliations

  1. School of Aeronautics and Astronautics, Shanghai Jiao Tong University, Shanghai, China

    Zhongliang Jing

  2. Moscow Aviation Institute, Moscow, Russia

    Dmitry Strelets

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wu, X., Pan, H., Xu, M., Jing, Z., Bao, M. (2023). A Unified Framework for Joint Moving Object Detection and Tracking in the Sky and Underwater. In: Jing, Z., Strelets, D. (eds) Proceedings of the International Conference on Aerospace System Science and Engineering 2021. ICASSE 2021. Lecture Notes in Electrical Engineering, vol 849. Springer, Singapore. https://doi.org/10.1007/978-981-16-8154-7_17

Download citation

  • DOI: https://doi.org/10.1007/978-981-16-8154-7_17

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-16-8153-0

  • Online ISBN: 978-981-16-8154-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation