Skip to main content
SpringerLink
Log in

A survey on horizon detection algorithms for maritime video surveillance: advances and future techniques

  • Survey
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

On maritime images, the horizon is a linear shape separating the sea and non-sea regions. This visual cue is essential in several sea video surveillance applications, including camera calibration, digital video stabilization, target detection and tracking, and distance estimation of detected targets. Given the nature of these applications, the horizon detection algorithm must satisfy robustness and real-time constraints. Our first aim in this paper is to provide a comprehensive review of horizon detection algorithms. After analyzing assumptions and test results reported in the horizon detection literature, we found a high trade-off between robustness and real-time performance. Thus, our second aim is to propose and describe three workable techniques to reduce this trade-off. The first technique aims to increase the robustness against contrast-degraded horizons. The non-sea region right above the horizon mainly depicts the sky, coast, ship, or combination of these classes. Thus, the second technique suggests a way to handle such class variation. While we believe that the last two techniques require relatively little computations, the third technique concerns using an alternative convolutional neural network (CNN) architecture to avoid a significant quantity of redundant computations in a previous CNN-based algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. MODD: https://www.vicos.si/Downloads/MODD; SMD: https://sites.google.com/site/dilipprasad/home/singapore-maritime-dataset.

  2. At a given edge pixel p, this direction is perpendicular to the gradient direction of p.

  3. The Radon transform is based on the same voting idea of Hough transform. Also, the Radon space \(R(\rho ,\theta )\) is interpreted in the same way as the Hough space \(H(\rho ,\theta )\).

  4. Processing time results are taken from experiments in [26].

  5. \(\approx \) 0.6 s on \(1920\times 1080\) images using Intel E5-1680 CPU.

  6. This space is quantized with a coordinate descent-based process that accounts for the performance and efficiency factors.

  7. N is not constant because the number of semantic lines may vary from one scene to another.

  8. A positive patch corresponds to a horizon edge pixel.

  9. In this context, FPs correspond to non-horizon edge pixels (i.e., noise), whereas FNs correspond to missing horizon edge pixels.

  10. We set N = 10 for high-resolution images.

References

  1. Moreira, R., Ebecken, N., Alves, A.S., Livernet, F., Campillo-Navetti, A.: A survey on video detection and tracking of maritime vessels. Int. J. Recent Res. Appl. Stud. 20 (2014)

  2. Prasad, D.K., Rajan, D., Rachmawati, L., Rajabally, E., Quek, C.: Video processing from electro-optical sensors for object detection and tracking in a maritime environment: a survey. IEEE Trans. Intell. Transp. Syst. 18, 1993–2016 (2017)

    Article  Google Scholar 

  3. Petković, M., Vujović, I., Kuzmanić, I.: An overview on horizon detection methods in maritime video surveillance. Trans. Marit. Sci. 9, 106–112 (2020)

    Article  Google Scholar 

  4. Pires, N., Guinet, J., Dusch, E.: ASV: an innovative automatic system for maritime surveillance. Navigation 58, 1–20 (2010)

    Google Scholar 

  5. Wei, H., et al.: Automated intelligent video surveillance system for ships in optics and photonics. In: Global Homeland Security V and Biometric Technology for Human Identification VI, vol. 7306, p. 73061N (2009)

  6. Zhang, W., Naik, S.M.: Camera auto-calibration by horizon estimation. US Patent 8,259,174 (2012)

  7. Ali, A., Smrz, P.: Camera auto-calibration for complex scenes. In: Thirteenth International Conference on Machine Vision, vol 11605, p. 116051W (2021)

  8. Fefilatyev, S., Goldgof, D., Shreve, M., Lembke, C.: Detection and tracking of ships in open sea with rapidly moving buoymounted camera system. Ocean Eng. 54, 1–12 (2012)

    Article  Google Scholar 

  9. Schwendeman, M., Thomson, J.: A horizon-tracking method for shipboard video stabilization and rectification. J. Atmos. Oceanic Tech. 32, 164–176 (2015)

    Article  Google Scholar 

  10. Morris, D.D., Colonna, B.R., Snyder, F.D.: Image-based motion stabilization for maritime surveillance. In: Image Processing: Algorithms and Systems V, vol. 6497, p. 64970F (2007)

  11. Cai, C., Weng, X., Zhu, Q.: Sea-skyline-based image stabilization of a buoy-mounted catadioptric omnidirectional vision system. EURASIP J. Image Video Process. 2018, 1 (2018)

    Article  Google Scholar 

  12. Van den Broek, S.P., Bouma, H., Degache, M.A.: Discriminating small extended targets at sea from clutter and other classes of boats in infrared and visual light imagery. In: Signal and Data Processing of Small Targets 2008, vol. 6969, p. 69690B (2008)

  13. Ettinger, S.M., Nechyba, M.C., Ifju, P.G., Waszak, M.: Towards flight autonomy: vision-based horizon detection for micro air vehicles. In: Florida Conference on Recent Advances in Robotics 2002 (2002)

  14. Cornall, T.D., Egan, G.K., Price, A.: Aircraft attitude estimation from horizon video. Electron. Lett. 42, 744–745 (2006)

    Article  Google Scholar 

  15. Grelsson, B., Felsberg, M., Isaksson, F.: Highly accurate attitude estimation via horizon detection. J. Field Robot. 33, 967–993 (2016)

    Article  Google Scholar 

  16. Abbass, M.Y., et al.: A survey on online learning for visual tracking. Vis. Comput. 37, 993–1014 (2021)

    Article  Google Scholar 

  17. Kristan, M., Kenk, V.S., Kovačič, S., Perš, J.: Fast image-based obstacle detection from unmanned surface vehicles. IEEE Trans. Cybern. 46, 641–654 (2015)

    Article  Google Scholar 

  18. Gershikov, E., Libe, T., Kosolapov, S.: Horizon line detection in marine images: which method to choose? Int. J. Adv. Intell. Syst. 6 (2013)

  19. Grishin, V.A., Maslov, I.A.: Horizon line stability observations over the sea. J. Navig. 71, 437–453 (2018)

    Article  Google Scholar 

  20. Bao, G.-Q., Xiong, S.-S., Zhou, Z.-Y.: Vision-based horizon extraction for micro air vehicle flight control. IEEE Trans. Instrum. Meas. 54, 1067–1072 (2005)

    Article  Google Scholar 

  21. Zhang, H., Yin, P., Zhang, X., Shen, X.: A robust adaptive horizon recognizing algorithm based on projection. Trans. Inst. Meas. Control. 33, 734–751 (2011)

    Article  Google Scholar 

  22. Hough, P.V.: Method and means for recognizing complex patterns. US Patent 3,069,654 (1962)

  23. Shen, Y.-F., Rahman, Z.-U., Krusienski, D., Li, J.: A visionbased automatic safe landing-site detection system. IEEE Trans. Aerosp. Electron. Syst. 49, 294–311 (2013)

    Article  Google Scholar 

  24. Shen, Y.-F., Krusienski, D., Li, J., Rahman, Z.-U.: A hierarchical horizon detection algorithm. IEEE Geosci. Remote Sens. Lett. 10, 111–114 (2012)

    Article  Google Scholar 

  25. Lipschutz, I., Gershikov, E., Milgrom, B.: New methods for horizon line detection in infrared and visible sea images. Int. J. Comput. Eng. Res. 3, 1197–1215 (2013)

    Google Scholar 

  26. Prasad, D.K., Rajan, D., Rachmawati, L., Rajabally, E., Quek, C.: MuSCoWERT: multi-scale consistence of weighted edge Radon transform for horizon detection in maritime images. JOSA A 33, 2491–2500 (2016)

    Article  Google Scholar 

  27. Jeong, C.Y., Yang, H.S., Moon, K.: Fast horizon detection in maritime images using region-of-interest. Int. J. Distrib. Sens. Netw. 14, 1550147718790753 (2018)

    Article  Google Scholar 

  28. Jeong, C., Yang, H.S., Moon, K.: A novel approach for detecting the horizon using a convolutional neural network and multiscale edge detection. Multidimens. Syst. Signal Process. 30, 1187–1204 (2019)

    Article  MATH  Google Scholar 

  29. Li, F., et al.: Sea-sky line detection using gray variation differences in the time domain for unmanned surface vehicles. SIViP 15, 139–146 (2021)

    Article  Google Scholar 

  30. Grelsson, B., Felsberg, M.: Probabilistic Hough voting for attitude estimation from aerial fisheye images. In: Scandinavian Conference on Image Analysis, pp. 478–488 (2013)

  31. Bouma, H., de Lange, D.-J.J., van den Broek, S.P., Kemp, R.A., Schwering, P.B.: Automatic detection of small surface targets with electro-optical sensors in a harbor environment. In: Electro- Optical Remote Sensing, Photonic Technologies, and Applications II, vol. 7114, p. 711402 (2008)

  32. Praczyk, T.: A quick algorithm for horizon line detection in marine images. J. Mar. Sci. Technol. 23, 164–177 (2018)

    Article  Google Scholar 

  33. Liu, J., Li, H., Liu, J., Xie, S., Luo, J.: Real-time monocular obstacle detection based on horizon line and saliency estimation for unmanned surface vehicles. Mob. Netw. Appl. 26, 1–14 (2021)

    Article  Google Scholar 

  34. Zou, X., et al.: A Novel Water-Shore-Line Detection Method for USV Autonomous Navigation. Sensors 20, 1682 (2020)

    Article  Google Scholar 

  35. Von Gioi, R.G., Jakubowicz, J., Morel, J.-M., Randall, G.: LSD: a fast line segment detector with a false detection control. IEEE Trans. Pattern Anal. Mach. Intell. 32, 722–732 (2008)

    Article  Google Scholar 

  36. Li, K., Yao, J., Lu, X., Li, L., Zhang, Z.: Hierarchical line matching based on line-junction–line structure descriptor and local homography estimation. Neurocomputing 184, 207–220 (2016)

    Article  Google Scholar 

  37. Gershikov, E.: Is color important for horizon line detection? In: 2014 International Conference on Advanced Technologies for Communications (ATC 2014), pp. 262–267 (2014)

  38. Zafarifar, B., Weda, H., et al.: Horizon detection based on skycolor and edge features. In: Visual Communications and Image Processing 2008, vol. 6822, p. 682220 (2008)

  39. Dumble, S.J., Gibbens, P.W.: Horizon profile detection for attitude determination. J. Intell. Robot. Syst. 68, 339–357 (2012)

    Article  Google Scholar 

  40. Cornall, T., Egan, G.: Calculating attitude from horizon vision. In: Eleventh Australian International Aerospace Congress, Melbourne (2005)

  41. Liang, D., Liang, Y.: Horizon detection from electro-optical sensors under maritime environment. IEEE Trans. Instrum. Meas. 69, 45–53 (2019)

    Article  Google Scholar 

  42. Wang, B., Su, Y., Wan, L.: A sea-sky line detection method for unmanned surface vehicles based on gradient saliency. Sensors 16, 543 (2016)

    Article  Google Scholar 

  43. Sun, Y., Fu, L.: Coarse-fine-stitched: a robust maritime horizon line detection method for unmanned surface vehicle applications. Sensors 18, 2825 (2018)

    Article  Google Scholar 

  44. Fefilatyev, S., Smarodzinava, V., Hall, L.O., Goldgof, D.B.: Horizon detection using machine learning techniques. In: 2006 5th International Conference on Machine Learning and Applications (ICMLA’06), pp. 17–21 (2006)

  45. Jeong, C., Yang, H., Moon, K.: Horizon detection in maritime images using scene parsing network. Electron. Lett. 54, 760–762 (2018)

    Article  Google Scholar 

  46. Praczyk, T., et al.: Concept and First Results of Optical Navigational System. Transactions on maritime science 8, 46–53 (2019)

    Article  Google Scholar 

  47. Cane, T., Ferryman, J.: Evaluating deep semantic segmentation networks for object detection in maritime surveillance. In: 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1–6 (2018)

  48. Kumeechai, P., Jiriwibhakorn, S.: Effective horizon detection on complex seas using back propagation neural network. In: 2019 16th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON), pp. 790–793 (2019)

  49. Zhan, W., et al.: Adaptive Semantic Segmentation for Unmanned Surface Vehicle Navigation. Electronics 9, 213 (2020)

    Article  Google Scholar 

  50. Lee, J.-T., Kim, H.-U., Lee, C., Kim, C.-S.: Semantic line detection and its applications. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3229–3237 (2017)

  51. Zhao, K., Han, Q., Zhang, C.-B., Xu, J., Cheng, M.-M.: Deep hough transform for semantic line detection. IEEE Trans. Pattern Anal. Mach. Intell. (2021)

  52. Lin, T.-Y., et al.: Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117–2125 (2017)

  53. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.-C.: Mobilenetv2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520 (2018)

  54. Kuanar, S., Mahapatra, D., Bilas, M., Rao, K.: Multi-path dilated convolution network for haze and glow removal in nighttime images. Vis. Comput. 1–14 (2021)

  55. Khmag, A., Al-Haddad, S., Kalantar, B., et al.: Single image dehazing using second-generation wavelet transforms and the mean vector L2-norm. Vis. Comput. 34, 675–688 (2018)

    Article  Google Scholar 

  56. Das, B., Ebenezer, J.P., Mukhopadhyay, S.: A comparative study of single image fog removal methods. Vis. Comput. 1–17 (2020)

  57. Xiao, C., Gan, J.: Fast image dehazing using guided joint bilateral filter. Vis. Comput. 28, 713–721 (2012)

    Article  Google Scholar 

  58. Lee, S., Yun, S., Nam, J.-H., Won, C.S., Jung, S.-W.: A review on dark channel prior based image dehazing algorithms. EURASIP J. Image Video Process. 2016, 1–23 (2016)

    Article  Google Scholar 

  59. Sermanet, P., et al.: Overfeat: integrated recognition, localization and detection using convolutional networks. arXiv preprint arXiv:1312.6229 (2013)

Download references

Funding

No funding was received for conducting this work.

Author information

Authors and Affiliations

  1. Laboratory of Informatics, Signals & Telecommunications (LIST), Faculty of Science and Techniques of Tangier (FST), Abdelmalek-Essaadi University (UAE), Ziaten. BP: 416, Tangier, Morocco

    Yassir Zardoua, Abdelali Astito & Mohammed Boulaala

Authors
  1. Yassir Zardoua
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abdelali Astito
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammed Boulaala
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yassir Zardoua.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zardoua, Y., Astito, A. & Boulaala, M. A survey on horizon detection algorithms for maritime video surveillance: advances and future techniques. Vis Comput 39, 197–217 (2023). https://doi.org/10.1007/s00371-021-02321-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02321-0

Keywords

Search

Navigation