Skip to main content
SpringerLink
Log in

LaneDraw: Cascaded lane and its bifurcation detection with nested fusion

  • Article
  • Published:
Science China Technological Sciences Aims and scope Submit manuscript

Abstract

Lane and its bifurcation detection is a vital and active research topic in low cost camera-based autonomous driving and advanced driver assistance system (ADAS). The common lane detection pipeline usually predicts lane segmentation mask firstly, and then makes line fitting by parabola or spline post-processing. However, if the speed of the lane and its bifurcation detection is fast and robust enough, we think curve fitting is not a necessary step. The goal of this work is to get accurate lane segmentation, identification of every lane, adaptability of lane numbers and the right combination of lane bifurcation. In this work, we relabeled lane and its bifurcation with solid line if the image of TuSimple dataset has both of them. In the data training process, we apply a data balance strategy for the heavily biased lane and non-lane data. In such a way, we develop a competitive cascaded instance lane detection model and propose a novel bifurcation pixel embedding nested fusion method based on full binary segmentation pixel embedding with self-grouping cluster, called LaneDraw. Our method discards curve fitting process, therefore it reduces the complexity of post-processing and increases detection speed at 35 fps. Moreover, the proposed method yields better performance and high accuracy on the relabeled TuSimple dataset. To the best of our knowledge, this is the first attempt in 2D lane and bifurcation detection, which more often happens in actual situations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ziegler J, Bende P, Dang T, et al. Trajectory planning for bertha—A local, continuous method. In: Proceedings of the 2014 IEEE Intelligent Vehicles Symposium. IEEE, 2014. 450–457

  2. Li Q, Chen L, Li M, et al. A sensor-fusion drivable-region and lane-detection system for autonomous vehicle navigation in challenging road scenarios. IEEE Trans Veh Technol, 2014, 63: 540–555

    Article  Google Scholar 

  3. Suzuki K. An analysis of driver’s steering behaviour during auditory or haptic warnings for the designing of lane departure warning system. JSAE Rev, 2003, 24: 65–70

    Article  Google Scholar 

  4. Neven D, De Brabandere B, Georgoulis S, et al. Towards end-to-end lane detection: An instance segmentation approach. In: Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2018. 286–291

  5. Philion J. Fastdraw: Addressing the long tail of lane detection by adapting a sequential prediction network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, 2019. 11582–11591

  6. Pan X, Shi J, Luo P, et al. Spatial as deep: Spatial cnn for traffic scene understanding. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence. New Orleans, 2018

  7. López A, Serrat J, Cañero C, et al. Robust lane markings detection and road geometry computation. Int J Automot Technol, 2010, 11: 395–407

    Article  Google Scholar 

  8. Yim Y U, Oh S. Three-feature based automatic lane detection algorithm (tfalda) for autonomous driving. IEEE Trans Intell Trans Syst, 2003, 4: 219–225

    Article  Google Scholar 

  9. Borkar A, Hayes M, Smith M T. A novel lane detection system with efficient ground truth generation. IEEE Trans Intell Transp Syst, 2012, 13: 365–374

    Article  Google Scholar 

  10. Deusch H, Wiest J, Reuter S, et al. A random finite set approach to multiple lane detection. In: Proceedings of the 2012 15th International IEEE Conference on Intelligent Transportation Systems. IEEE, 2012. 270–275

  11. Cheng H Y, Jeng B S, Tseng P T, et al. Lane detection with moving vehicles in the traffic scenes. IEEE Trans Intell Transp Syst, 2006, 7: 571–582

    Article  Google Scholar 

  12. Chen Z, Chen Z. Rbnet: A deep neural network for unified road and road boundary detection. In: Proceedings of the International Conference on Neural Information. Springer, 2017. 677–687

  13. Ghafoorian M, Nugteren C, Baka N, et al. El-gan: Embedding loss driven generative adversarial networks for lane detection. In: Proceedings of the European Conference on Computer Vision (ECCV). Munich, 2018

  14. Hou Y, Ma Z, Liu C, et al. Learning to steer by mimicking features from heterogeneous auxiliary networks. AAAI, 2019, 33: 8433–8440

    Article  Google Scholar 

  15. Lee S, Kim J, Shin Yoon J, et al. Vpgnet: Vanishing point guided network for lane and road marking detection and recognition. In: Proceedings of the IEEE International Conference on Computer Vision. Venice, 2017. 1947–1955

  16. Kim J, Park C. End-to-end ego lane estimation based on sequential transfer learning for self-driving cars. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu, 2017. 30–38

  17. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv: 1409.1556

  18. Paszke A, Chaurasia A, Kim S, et al. Enet: A deep neural network architecture for real-time semantic segmentation. arXiv: 1606.02147

  19. Bar Hillel A, Lerner R, Levi D, et al. Recent progress in road and lane detection: A survey. Machine Vision Appl, 2014, 25: 727–745

    Article  Google Scholar 

  20. Bertozzi M, Broggi A. GOLD: A parallel real-time stereo vision system for generic obstacle and lane detection. IEEE Trans Image Process, 1998, 7: 62–81

    Article  Google Scholar 

  21. Wang Y, Teoh E K, Shen D. Lane detection and tracking using B-Snake. Image Vision Comput, 2004, 22: 269–280

    Article  Google Scholar 

  22. Aly M. Real time detection of lane markers in urban streets. In: Proceedings of the 2008 IEEE Intelligent Vehicles Symposium. IEEE, 2008. 7–12

  23. Linarth A, Angelopoulou E. On feature templates for particle filter based lane detection. In: Proceedings of the 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC). IEEE, 2011. 1721–1726

  24. Niu J, Lu J, Xu M, et al. Robust lane detection using two-stage feature extraction with curve fitting. Pattern Recognition, 2016, 59: 225–233

    Article  Google Scholar 

  25. Mammeri A, Boukerche A, Tang Z. A real-time lane marking localization, tracking and communication system. Comput Commun, 2016, 73: 132–143

    Article  Google Scholar 

  26. Borkar A, Hayes M, Smith M. T. Robust lane detection and tracking with ransac and kalman filter. In: Proceedings of the 2009 16th IEEE International Conference on Image Processing (ICIP). IEEE, 2009. 3261–3264

  27. Lee J W. A machine vision system for lane-departure detection. Comput Vision Image Understand, 2002, 86: 52–78

    Article  Google Scholar 

  28. Shen H, Ling R, Mao J G, et al. Steering control strategy guide by two preview vision cues. Sci China Tech Sci, 2012, 55: 2662–2670

    Article  Google Scholar 

  29. Hur J, Kang S N, Seo S W. Multi-lane detection in urban driving environments using conditional random fields. In: Proceedings of the 2013 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2013. 1297–1302

  30. Wang C Y, Zhao W Z, Xu Z J, et al. Path planning and stability control of collision avoidance system based on active front steering. Sci China Tech Sci, 2017, 60: 1231–1243

    Article  Google Scholar 

  31. Xiao J, Luo L, Yao Y, et al. Lane detection based on road module and extended kalman filter. In: Proceedings of the Pacific-Rim Symposium on Image and Video Technology. Springer, 2017. 382–395

  32. Gansbeke W, Brabandere B, Neven D, et al. End-to-end lane detection through differentiable least-squares fitting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Seoul, 2019

  33. Jaderberg M, Simonyan K, Zisserman A, et al. Spatial transformer networks. In: Proceedings of the Advances in Neural Information Processing Systems. Montreal, 2015

  34. Handa A, Bloesch M, Pǎtrǎucean V, et al. gvnn: Neural network library for geometric computer vision. In: Proceedings of the European Conference on Computer Vision Workshops. Amsterdam, 2016

  35. Bronstein M M, Bruna J, LeCun Y, et al. Geometric deep learning: Going beyond euclidean data. IEEE Signal Process Mag, 2017, 34: 18–42

    Article  Google Scholar 

  36. Chen P-R, Lo S-Y, Hang H-M, et al. Efficient road lane marking detection with deep learning. In: Proceedings of the 2018 IEEE 23rd International Conference on Digital Signal Processing (DSP). IEEE, 2018. 1–5

  37. Homayounfar N, Ma W-C, Kowshika Lakshmikanth S, et al. Hierarchical recurrent attention networks for structured online maps. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, 2018. 3417–3426

  38. Kendall A, Hawke J, Janz D, et al. Learning to drive in a day. In: Proceedings of the 2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019. 8248–8254

  39. Zou Q, Jiang H, Dai Q, et al. Robust lane detection from continuous driving scenes using deep neural networks. arXiv: 1903.02193v1

  40. Dalal N, Triggs B. Histograms of oriented gradients for human detection. In: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05). IEEE, 2005. 886–893

  41. Cortes C, Vapnik V. Support-vector networks. Mach Learn, 1995, 20: 273–297

    MATH  Google Scholar 

  42. Xie S, Tu Z. Holistically-nested edge detection. In: Proceedings of the IEEE International Conference on Computer Vision. Santiago, 2015. 1395–1403

  43. De Brabandere B, Neven D, Van Gool L. Semantic instance segmentation with a discriminative loss function. arXiv: 1708.02551

  44. TuSimple. http://benchmark.tusimple.ai/#/t/1. Accessed: 2018–09-08

  45. Kingma D P, Ba J. Adam: A method for stochastic optimization. arXiv: 1412.6980v9

Download references

Author information

Authors and Affiliations

  1. Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China

    KeYan Ren, HaoChen Hou, SiYang Li & TianYi Yue

  2. Beijing Engineering Research Center for IoT Software and Systems, Beijing, 100124, China

    KeYan Ren, HaoChen Hou, SiYang Li & TianYi Yue

Authors
  1. KeYan Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. HaoChen Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. SiYang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. TianYi Yue
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to KeYan Ren.

Additional information

This work was supported by the National Key Research and Development Project (Grant No. 2019YFC1511003), the National Natural Science Foundation of China (Grant No. 61803004), and the Aeronautical Science Foundation of China (Grant No. 20161375002).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ren, K., Hou, H., Li, S. et al. LaneDraw: Cascaded lane and its bifurcation detection with nested fusion. Sci. China Technol. Sci. 64, 1238–1249 (2021). https://doi.org/10.1007/s11431-020-1702-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11431-020-1702-2

Keywords

Search

Navigation