Abstract
In this study, a lane following method based on deep learning from surround view images for autonomous driving of a ground vehicle is proposed. Previous methods can be suffered from false detection by hand-craft feature extraction in color-based binarization especially when the surround view images are exposed to unfavorable conditions such as strong shadow, sunlight reflections or shallow puddles on the roads. Thus the proposed method adopts a modified convolutional neural network structure to estimate the 6 coefficients of the left and right lane lines modeled by two quadratic functions from the surround view images of a vehicle. Then, a desired steering wheel angle is calculated using Stanley method to make a test vehicle follow a test lane autonomously by the proposed method. Autonomous driving experiment of the test vehicle using the proposed method was carried out on the test lane with various unfavorable conditions high-curvature lane of a test field. Experiment results showed that the vehicle was self-driven autonomously and stably without any lane departures on the test lane.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Adhikari SP, Kim H (2010) Dynamic programming and curve fitting based road boundary detection. In: Proceedings of the 9th WSEAS international conference on computational intelligence, man-machine systems and cybernetics, pp 236–240
Aly M (2008) Real time detection of lane markers in urban streets. In: 2008 IEEE intelligent vehicles symposium. IEEE, pp 7–12. https://doi.org/10.1109/ivs.2008.4621152
Bertozzi M, Broggi A (1998) GOLD: a parallel real-time stereo vision system for generic obstacle and lane detection. IEEE Trans Image Process 7:62–81. https://doi.org/10.1109/83.650851
Chae H, Min KC, Yi K (2017) Model predictive control based automated driving lane change control algorithm for merge situation on highway intersection. SAE Techn Pap. https://doi.org/10.4271/2017-01-1441
Chiu K-Y, Lin S-F (2005) Lane detection using color-based segmentation. In: IEEE proceedings. Intelligent vehicles symposium, IEEE, pp 706–711. https://doi.org/10.1109/ivs.2005.1505186
Chrysalidis N, Ghobadi SE, Heitmann (2018) A path planning for a common lateral control system architecture. In: AmE 2018-automotive meets electronics; 9th GMM-symposium. VDE, pp 1–4
Ding J, Li K, Hedrick K (2015) A robust lane-keeping ‘co-pilot’ system using LBMPC method. SAE Int J Passeng Cars-Electron Electr Syst 8:219–228. https://doi.org/10.4271/2015-01-0322
Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: European conference on computer vision. Springer, pp 184–199. https://doi.org/10.1007/978-3-319-10593-2_13
Farhat W, Sghaier S, Faiedh H, Souani C (2018) Design of efficient embedded system for road sign recognition. J Ambient Intell Humaniz Comput 10(2):491–507. https://doi.org/10.1007/s12652-017-0673-3
Fritsch J, Kühnl T, Kummert F (2014) Monocular road terrain detection by combining visual and spatial information. IEEE Trans Intell Transp Syst 15:1586–1596. https://doi.org/10.1109/TITS.2014.2303899
Ghazaei G, Alameer A, Degenaar P, Morgan G, Nazarpour K (2017) Deep learning-based artificial vision for grasp classification in myoelectric hands. J Neural Eng 14:036025. https://doi.org/10.1088/1741-2552/aa6802
Gietelink O, Ploeg J, De Schutter B, Verhaegen M (2006) Development of advanced driver assistance systems with vehicle hardware-in-the-loop simulations. Veh Syst Dyn 44:569–590. https://doi.org/10.1080/00423110600563338
Han KY, Lee M, Lee Y-S (2018) Implementation of autonomous driving of a ground vehicle for narrow high-curvature roads using surround view images. Multimed Tools Appl 1:1–20. https://doi.org/10.1007/s11042-018-6485-4
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Hillel AB, Lerner R, Levi D, Raz G (2014) Recent progress in road and lane detection: a survey. Mach Vis Appl 25:727–745. https://doi.org/10.1007/s00138-011-0404-2
Hsu J-Y, Ku K-L, Jhang T-K, Yeh C-J, Lin H-P (2017) Implementation of phase lead controller for lane following system based on identified vehicle lateral model. In: 2017 international automatic control conference (CACS). IEEE, pp 1–6. https://doi.org/10.1109/cacs.2017.8284236
Kim D, Kim B, Chung T, Yi K (2016) Lane-level localization using an AVM camera for an automated driving vehicle in urban environments. IEEE/ASME Trans Mechatron 22:280–290. https://doi.org/10.1109/TMECH.2016.2533635
LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1:541–551. https://doi.org/10.1162/neco.1989.1.4.541
Lee S, et al. (2017) Vpgnet: vanishing point guided network for lane and road marking detection and recognition. In: Proceedings of the IEEE international conference on computer vision, pp 1947–1955. https://doi.org/10.1109/iccv.2017.215
Lu M, Wevers K, Van Der Heijden R (2005) Technical feasibility of advanced driver assistance systems (ADAS) for road traffic safety. Transp Plan Technol 28:167–187. https://doi.org/10.1080/03081060500120282
McCall JC, Trivedi MM (2006) Video-based lane estimation and tracking for driver assistance: survey, system, and evaluation. IEEE Trans Intell Transp Syst 7:20–37. https://doi.org/10.1109/tits.2006.869595
Mineta K, Unoura K, Ikeda T (2003) Development of a lane mark recognition system for a lane keeping assist system. SAE Tech Pap. https://doi.org/10.4271/2003-01-0281
Montavon G, Orr G, Müller K-R (1998) Neural networks: tricks of the trade. Springer, Berlin. https://doi.org/10.1007/3-540-49430-8
Neven D, De Brabandere B, Georgoulis S, Proesmans M, Van Gool L (2018) Towards end-to-end lane detection: an instance segmentation approach. In: 2018 IEEE intelligent vehicles symposium (IV), pp 286–291. https://doi.org/10.1109/ivs.2018.8500547
Nguyen V, Kim H, Jun S, Boo K (2018) A study on real-time detection method of lane and vehicle for lane change assistant system using vision system on highway. Eng Sci Technol Int J 21:822–833. https://doi.org/10.1016/j.jestch.2018.06.006
Pan SJ, Yang Q (2009) A survey on transfer learning. IEEE Trans Knowl Data Eng 22:1345–1359. https://doi.org/10.1109/TKDE.2009.191
Russakovsky O et al (2015) ImageNet large scale visual recognition challenge. Int J Comput Vis 115:211–252. https://doi.org/10.1007/s11263-015-0816-y
Russell SJ, Norvig P (2016) Artificial intelligence: a modern approach. Pearson Education Limited, Malaysia
Shi J, Wu J, Anisetti M, Damiani E, Jeon G (2017) An interval type-2 fuzzy active contour model for auroral oval segmentation. Soft Comput 21(9):2325–2345. https://doi.org/10.1007/s00500-015-1943-7
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:14091556
Sugawara T, Altmannshofer H, Kakegawa S (2017) Applications of road edge information for advanced driver assistance systems and autonomous driving. In: Advanced microsystems for automotive applications. Springer, pp 71–86. https://doi.org/10.1007/978-3-319-66972-4_7
Szegedy C et al. (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9. https://doi.org/10.1109/cvpr.2015.7298594
Thrun S et al (2006) Stanley: the robot that won the DARPA grand challenge. J Field Robot 23:661–692. https://doi.org/10.1007/978-3-030-11292-9_39
Varga D, Sziranyi T (2017) Robust real-time pedestrian detection in surveillance videos. J Ambient Intell Humaniz Comput 8(1):79–85. https://doi.org/10.1007/s12652-016-0369-0
Wu J, Anisetti M, Wu W, Damiani E, Jeon G (2016) Bayer demosaicking with polynomial interpolation. IEEE Trans Image Process 25(11):5369–5382. https://doi.org/10.1109/TIP.2016.2604489
Yoo H, Yang U, Sohn K (2013) Gradient-enhancing conversion for illumination-robust lane detection. IEEE Trans Intell Transp Syst 14:1083–1094. https://doi.org/10.1109/TITS.2013.2252427
Acknowledgements
This work was supported by the Incheon National University Research Grant in 2016.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Lee, M., Han, K.Y., Yu, J. et al. A new lane following method based on deep learning for automated vehicles using surround view images. J Ambient Intell Human Comput 14, 1–14 (2023). https://doi.org/10.1007/s12652-019-01496-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-019-01496-8