Skip to main content
Log in

Semantic Segmentation-Based Road Marking Detection Using Around View Monitoring System

  • Published:
Journal of Shanghai Jiaotong University (Science) Aims and scope Submit manuscript

Abstract

Road marking detection is an important branch in autonomous driving, understanding the road information. In recent years, deep-learning-based semantic segmentation methods for road marking detection have been arising since they can generalize detection result well under complicated environments and hold rich pixel-level semantic information. Nevertheless, the previous methods mostly study the training process of the segmentation network, while omitting the time cost of manually annotating pixel-level data. Besides, the pixel-level semantic segmentation results need to be fitted into more reliable and compact models so that geometrical information of road markings can be explicitly obtained. In order to tackle the above problems, this paper describes a semantic segmentation-based road marking detection method using around view monitoring system. A semiautomatic semantic annotation platform is developed, which exploits an auxiliary segmentation graph to speed up the annotation process while guaranteeing the annotation accuracy. A segmentation-based detection module is also described, which models the semantic segmentation results for the more robust and compact analysis. The proposed detection module is composed of three parts: vote-based segmentation fusion filtering, graph-based road marking clustering, and road-marking fitting. Experiments under various scenarios show that the semantic segmentation-based detection method can achieve accurate, robust, and real-time detection performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. AKITA T. Lane keeping assist device for vehicle: US7216023 [P]. 2007-05-08.

  2. GRUYER D, BELAROUSSI R, REVILLOUD M. Accurate lateral positioning from map data and road marking detection [J]. Expert Systems with Applications, 2016, 43:1–8.

    Article  Google Scholar 

  3. KIM D, KIM B, CHUNG T, et al. Lane-level localization using an AVM camera for an automated driving vehicle in urban environments [J]. IEEE/ASME Transactions on Mechatronics, 2017, 22(1): 280–290.

    Article  Google Scholar 

  4. JEONG J, CHO Y, KIM A. Road-SLAM: Road marking based SLAM with lane-level accuracy [C] //2017 IEEE Intelligent Vehicles Symposium (IV). Los Angeles, CA: IEEE, 2017: 1736–1473.

    Google Scholar 

  5. HU J, YANG M, XU H, et al. Mapping and localization using semantic road marking with centimeter-level accuracy in indoor parking lots [C]//2019 IEEE Intelligent Transportation Systems Conference (ITSC). Auckland: IEEE, 2019: 4068–4073.

    Google Scholar 

  6. DENG L, YANG M, HU B, et al. Semantic segmentation-based lane-level localization using around view monitoring system [J]. IEEE Sensors Journal, 2019, 19(21): 10077–10086.

    Article  Google Scholar 

  7. YANG B, FANG L, LI Q, et al. Automated extraction of road markings from mobile Lidar point clouds [J]. Photogrammetric Engineering & Remote Sensing, 2012, 78(4): 331–338.

    Article  Google Scholar 

  8. HATA A, WOLF D. Road marking detection using LIDAR reflective intensity data and its application to vehicle localization [C]//17th International IEEE Conference on Intelligent Transportation Systems (ITSC). Qingdao: IEEE, 2014: 584–589.

    Google Scholar 

  9. LIU W, ZHANG H, DUAN B, et al. Vision-based realtime lane marking detection and tracking [C]//2008 11th International IEEE Conference on Intelligent Transportation Systems. Beijing: IEEE, 2008: 49–54.

    Google Scholar 

  10. LEE S, KIM J, YOON J S, et al. VPGNet: Vanishing point guided network for lane and road marking detection and recognition [C]//2017 IEEE International Conference on Computer Vision (ICCV). Venice: IEEE, 2017: 1965–1973.

    Google Scholar 

  11. BAILO O, LEE S, RAMEAU F, et al. Robust road marking detection and recognition using density-based grouping and machine learning techniques [C]//2017 IEEE Winter Conference on Applications of Computer Vision (WACV). Santa Rosa, CA: IEEE, 2017: 760–768.

    Google Scholar 

  12. AMAYO P, BRULS T, NEWMAN P. Semantic classification of road markings from geometric primitives [C]//2018 21st International Conference on Intelligent Transportation Systems (ITSC). Maui, HI: IEEE, 2018: 387–393.

    Google Scholar 

  13. TRAN L A, LE M H. Robust U-Net-based road lane markings detection for autonomous driving [C]//2019 International Conference on System Science and Engineering (ICSSE). Dong Hoi: IEEE, 2019: 62–66.

    Google Scholar 

  14. QIN B, LIU W, SHEN X, et al. A general framework for road marking detection and analysis [C]//16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013). The Hague: IEEE, 2013: 619–625.

    Google Scholar 

  15. WU Y, YANG T, ZHAO J, et al. VH-HFCN based parking slot and lane markings segmentation on panoramic surround view [C]//2018 IEEE Intelligent Vehicles Symposium (IV). Changshu: IEEE, 2018: 1767–1772.

    Google Scholar 

  16. JIANG W, WU Y, GUAN L T, et al. DFNet: semantic segmentation on panoramic images with dynamic loss weights and residual fusion block [C]//2019 International Conference on Robotics and Automation (ICRA). Montreal, QC: IEEE, 2019: 5887–5892.

    Google Scholar 

  17. ROMERA E, 脕LVAREZ J M, BERGASA L M, et al. ERFNet: Efficient residual factorized ConvNet for real-time semantic segmentation [J]. IEEE Transactions on Intelligent Transportation Systems, 2018, 19(1): 263–272.

    Article  Google Scholar 

  18. CORDTS M, OMRAN M, RAMOS S, et al. The cityscapes dataset for semantic urban scene understanding [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV: IEEE, 2016: 3213–3223.

    Google Scholar 

  19. ACUNA D, LING H, KAR A, et al. Efficient interactive annotation of segmentation datasets with polygon-RNN++ [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT: IEEE, 2018: 859–868.

    Google Scholar 

  20. CAESAR H, UIJLINGS J, FERRARI V. COCO-stuff: Thing and stuff classes in context [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT: IEEE, 2018: 1209–1218.

    Google Scholar 

  21. ACHANTA R, SHAJI A, SMITH K, et al. SLIC superpixels compared to state-of-the-art superpixel methods [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(11): 2274–2282.

    Article  Google Scholar 

  22. SCARAMUZZA D, MARTINELLI A, SIEGWART R. A toolbox for easily calibrating omnidirectional cameras [C]//2006 IEEE/RSJ International Conference on Intelligent Robots and Systems. Beijing: IEEE, 2006: 5695–5701.

    Google Scholar 

  23. FISCHLER M A, BOLLES R C. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography [M]//Readings in computer vision. Amsterdam: Elsevier, 1987: 726–740.

    Google Scholar 

  24. KING B. Step-wise clustering procedures [J]. Journal of the American Statistical Association, 1967, 62(317): 86–101.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ming Yang  (杨 明).

Additional information

Foundation item: the National Natural Science Foundation of China (Nos. U1764264 and 61873165), and the Shanghai Automotive Industry Science and Technology Development Foundation (No. 1807)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, H., Yang, M., Deng, L. et al. Semantic Segmentation-Based Road Marking Detection Using Around View Monitoring System. J. Shanghai Jiaotong Univ. (Sci.) 27, 833–843 (2022). https://doi.org/10.1007/s12204-021-2401-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12204-021-2401-6

Key words

CLC number

Document code

Navigation