Skip to main content
SpringerLink
Log in

Vehicle lane markings segmentation and keypoint determination using deep convolutional neural networks

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Lane detection is used to detect the lane markings in a road scene between which the vehicle is driving and provide the accurate location and shape of each lane marking. It serves as one of the key techniques to enable modern, assisted, and autonomous driving systems. However, lane detection poses several challenges. The lane markings vary in their shapes, colors, and patterns. The lack of distinct features and the presence of several occlusions on the roads makes the use of conventional methods using handcrafted features less robust and computationally expensive. In this study, we propose a compact and efficient multi-stage Convolutional Neural Network (CNN) architecture which can learn both the lane markings segmentation and also the localization and shape of each lane in the form of key-points. The proposed model combines a lane mask proposal network with a lane key-point determination network to accurately predict the key-points that describe the left and right lane-markings of the vehicle lanes. The high running speed and low computational cost of the proposed method make it suitable for being deployed in the real world vehicle systems. Through simulation results, we also show that the proposed method is robust to a variety of weather conditions and highway driving scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Amaradi P, Sriramoju N, Li Dang GS, Tewolde, Kwon J (2016) Lane following and obstacle detection techniques in autonomous driving vehicles. IEEE International Conference on Electro Information Technology (EIT), Grand Forks, ND, pp 0674–0679

  2. Ayachi R, Afif M, Yahia Said, and Atri M (2018) Strided convolution instead of max pooling for memory efficiency of convolutional neural networks. In: International conference on the Sciences of Electronics, Technologies of Information and Telecommunications, pp 234–243. Springer, Cham

  3. Badrinarayanan V, Kendall A, Cipolla R, Segnet (2017) A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Article  Google Scholar 

  4. Bellis EA, Page J (2008) National Motor Vehicle Crash Causation Survey (NMVCCS) SAS Analytical Users Manual.

  5. Chougule S, Ismail A, Soni A, Kozonek N, Narayan V, Schulze M (2018) An efficient encoder-decoder CNN architecture for reliable multilane detection in real time. In: 2018 IEEE Intelligent Vehicles Symposium (IV), pp 1444–1451. IEEE, Piscataway

  6. Gayko JE (2012) Lane departure and lane keeping. Handbook of Intelligent Vehicles 1:689–708

    Article  Google Scholar 

  7. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp 1026–1034

  8. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770-778

  9. Hou Y, Ma Z, Liu C, Loy CC (2019) Learning lightweight lane detection cnns by self attention distillation. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1013–1021

  10. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861

  11. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167

  12. Kervadec H, Bouchtiba J, Desrosiers C, Granger E, Dolz J, Ayed IB, 2019, May. Boundary loss for highly unbalanced segmentation. In: International conference on medical imaging with deep learning, pp 285–296

  13. Li Mingfa L Yuanyuan, J Min (2018) Lane detection based on connection of various feature extraction methods. Advances in multimedia. Hindawi, London, vol 24, pp 1687–5680

  14. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

  15. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

  16. Muthalagu R, Bolimera A, Kalaichelvi V (2020) Lane detection technique based on perspective transformation and histogram analysis for self-driving cars. Comput Electr Eng 85:106653

    Article  Google Scholar 

  17. Neven D, Brabandere BD, Georgoulis S, Proesmans M, Luc Van Gool (2008) Towards end-to-end lane detection: an instance segmentation approach.. In: 2018 IEEE intelligent vehicles symposium (IV), pp 286–291. IEEE, Piscataway

  18. Neven D, De Brabandere B, Georgoulis S, Proesmans M, Van Gool L (2018) Towards end-to-end lane detection: an instance segmentation approach. In: 2018 IEEE intelligent vehicles symposium (IV), pp 286–291. IEEE, Piscataway

  19. Oliveira M, Santos V, Sappa AD (2015) Multimodal inverse perspective mapping. Inf Fusion 24:108–121

    Article  Google Scholar 

  20. Paszke A, Chaurasia A, Kim S, Culurciello E (2016) Enet: A deep neural network architecture for real-time semantic segmentation. arXiv preprint arXiv:1606.02147

  21. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical image computing and computer-assisted intervention. Springer, Cham, pp 234–241

    Google Scholar 

  22. Singh S (2015, February) Critical reasons for crashes investigated in the National Motor Vehicle Crash Causation Survey. (Traffic Safety Facts Crash•Stats. Report No. DOT HS 812 115). National Highway Traffic Safety Administration, Washington, DC

  23. Shelhamer E, Long J, Darrell T (2017) Fully convolutional networks for semantic segmentation. In: IEEE Transactions on Pattern Analysis and Machine Intelligence, vol 39, no 4, pp 640–651. https://doi.org/10.1109/TPAMI.2016.2572683

  24. Tabelini L, Berriel R, Paixão TM, Badue C, Alberto F De Souza, Oliveira-Santos T (2020) PolyLaneNet: Lane estimation via deep polynomial regression. arXiv preprint arXiv:2004.10924

  25. Talib ML, Rui X, Ghazali KH, Mohd. Zainudin N, Ramli S (2013) Comparison of edge detection technique for lane analysis by improved hough transform, vol 8237. Springer, Cham, pp 176–183

  26. The tuSimple lane challenge. http://benchmark.tusimple.ai/

  27. Visvikis C et al (2008) Study on lane departure warning and lane change assistant systems.. Transport Research Laboratory Project Rpt PPR, 374

  28. Wu H, Zhang J, Huang K, Liang K, Yizhou Yu (2019) Fastfcn: Rethinking dilated convolution in the backbone for semantic segmentation. arXiv preprint arXiv:1903.11816

  29. Xing Y et al (2018) Advances in vision-based lane detection: algorithms, integration, assessment, and perspectives on ACP-based parallel vision. IEEE/CAA J Automat Sin 5(3):645–661. https://doi.org/10.1109/JAS.2018.7511063

    Article  Google Scholar 

  30. Pan X, Shi J, Luo P, Wang X, Tang X (2018) Spatial as deep: Spatial CNN for traffic scene understanding. AAAI

  31. Yan X, Li Y (2017) A method of lane edge detection based on Canny algorithm. 2017 Chinese Automation Congress (CAC), pp 2120–2124

  32. Yosinski J, Clune J, Nguyen A, Fuchs T, Lipson H (2015) Understanding neural networks through deep visualization. arXiv preprint arXiv:1506.06579

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Birla Institute of Technology and Science Pilani, Dubai Campus, Dubai, UAE

    Raja Muthalagu

  2. Department of Electrical and Electronics Engineering, Birla Institute of Technology and Science Pilani, Dubai Campus, Dubai, UAE

    Anudeepsekhar Bolimera & V. Kalaichelvi

Authors
  1. Raja Muthalagu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anudeepsekhar Bolimera
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. V. Kalaichelvi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Raja Muthalagu.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Muthalagu, R., Bolimera, A. & Kalaichelvi, V. Vehicle lane markings segmentation and keypoint determination using deep convolutional neural networks. Multimed Tools Appl 80, 11201–11215 (2021). https://doi.org/10.1007/s11042-020-10248-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-10248-2

Search

Navigation