Introduction and related work

The stumbling of drivers on the road is a vital risk factor in road safety. Therefore, driver assistance systems (DAS) are developed, implemented, and adopted by many manufacturers. The LD is a essential subsystem in DAS. The main component in vehicle localization is lane detection (LD) which is employed in the lane departure warning (LDW) system for DAS in autonomous vehicles. Many research papers have reported LD related problem statements; however, few of them have addressed the deployment of LD for the industrial purpose in an integrated framework. Using IoT is increasing recently which yields better performance and cheaper solutions for complicated real-life problems. Nevertheless, relying on one module to achieve road safety is not enough in practice. Instead, an alternative solution should be ready in place of device failure. To achieve this, a cloud computing environment is used to enhance the model robustness and to make the faster decisions. Cloud computing uses remote resources which saves the cost of servers and other equipment. In addition, hardware failures do not lead to data loss because of networked backups. The architecture of the IoT-based framework (IBF) for LD and tracking is presented in Fig. 1. Simplified vanishing point detection method is employed in [1], and a scan-line method is applied to detect lane ridge features. Multi-LD algorithm that is robust to the challenging road conditions has been proposed in [2]. An adaptive threshold has been applied to extract strong lane features from images with obstacles and barely visible lanes. Then, an improved RANdom SAmple Consensus algorithm has been introduced using the feedback from lane edge angles and the curvature of lane history to prevent false LD. Dynamic ROI extraction, edge detection, and Hough straight-line detection have been applied to extract the lane line in [3]. The model predictive control has been applied to track the extracted lane line and the front wheel steering angle has been corrected by the fuzzy controller based on the yaw angle and the yaw rate. Some reported articles have addressed the issue of LD and tracking using vision-based techniques without or with little knowledge of road geometry [4, 5]. These techniques mostly depend on the color threshold and work well on highways and urban avenues under daylight or white lighting conditions. YCbCr color model has been used to focus on the most important visual information contained in the Y component, and to reduce time complexity [6]. In another method, the HSV color space is employed to achieve an 86.21% accuracy in detection rate [7]. Under daylight or white light conditions, the white lane markers and yellow lane markers almost preserve individual true colors. Therefore, global thresholding on different color plane in YCbCr or HSV color model efficiently segments the color planes to extract the lane markers. However, in the tunnel because of the color light, the color of the lane markers does not preserve their true colors. Hence, the global thresholding methods yield poor results in the segmentation of lane markers. Therefore, in this paper, an efficient LD method is developed to detect the lane markers efficiently in the tunnel either with artificial colored light as well as on highway under the daylight.

Fig. 1
figure 1

Block diagram of the IBF

Full size image
Fig. 2
figure 2

Generalized flowchart for LD scheme

Full size image

The general LD procedure starts with preprocessing the input frame to remove the perspective noisy information from the image. The preprocessing stage is followed by ROI isolation where lane markers are likely to be presented. Within ROI, extraction of features of lane marker is carried out. The lane marker detection stage is divided into two categories, feature-based and model-based. The feature-based LD methods and lane marker candidates are identified by some features like color, shape, and orientation. Whereas, in the model-based LD methods, search for lane markers systematically according to either the geometric measurements perceived in the road scene or the persistent information between successive frames. The candidate lane markers extracted from the previous stage are then validated to reduce the false-positive rate. Either linear or curved lane fitting is applied to validate lane markers as the output of the LD process. This output is used for many purposes including lane departure warning, automatic lane centering, and adaptive cruise control systems. In this paper, a model-based lane marker detection is proposed to adress the problem arrises due to the color light in tunnel. Figure 2 shows the generalized flowchart of LD. Image preprocessing is carried out to reduce the distortion and noise present in the captured image. Distortion is caused by the perspective effect when acquiring the image using normal cameras. To reduce the distortion, inverse perspective mapping (IPM) is employed to convert the input image into the bird’s view image [2, 8,9,10,11]. The IPM is also used in calculation of lane width between left- and right-lane boundaries [12]. Despite its efficiency, the IPM is sometimes avoided because of its significant processing time of around 35.3 ms/frame [13]. The LD potentiality is affected by noise due to different surrounding illumination sources, on-lateral-road objects, and weather conditions. A shadow invariant method is introduced [14] using the maximally stable extremal region (MSER)-based approach in the blue color channel as well as using Hough transform. Employing the averaging of the pixel values of preceeding frames approach is introduced in [2] to improve the low-quality lane markers. Noise smoothing Gaussian filter to remove the noise of the mounted camera is dealt in [15]. The image pyramid approach is adopted in [16] to diminish the details and to present the high-frequency data. A four-level Gaussian pyramid model is employed to reduce the image dimensions and to make the edge drawing lines algorithm works effectively for the lowest resolution image at the top level of pyramids.

In [17], the ROI containing lane markers is considered as the bottom third of the input image. The extraction of vanishing point (VP) is used in [2, 8, 12, 14, 18,19,20,21,22, 26]. The VP is considered to be the point in which the most extracted lines of image are intersected. Adaptive ROI based on the longitudinal velocity changes of the vehicle is introduced [3] in which the upper boundary line moves down and up according to autonomous vehicle speed. The ROI extraction based on the minimum safe distance between the ego vehicle and the vehicle in front of it is proposed in [23]. In this paper, authors have reported that 150-pixel height is enough to cover 35 meters ahead when the vehicle speeds at 110 km/h. Generally, lane markers within the ROI are extracted using either predefined model or features. In feature-based LD scheme, lane marker candidates are initially defined by some features such as the orientations to x-axis [8], the width–height ratio [14], length, angles, and y-intercept in Hough transform [11]. The model-based LD techniques search for lane markers in a systematic method according to either the geometric measurements perceived in the road scene or the persistent information between successive frames. The temporal–spatial information matching method is suggested in [24]. In this paper, the top-view binary image is searched linearly for lane markers and, thereafter, the extracted markers are fitted using a cubic B-spline method. An improved version of the random sample consensus (RANSAC) algorithm is dealt in [10]. In this algorithm, two lane fittings, straight-line fitting model in the near field to the vehicle, and third-degree curve fitting for the far-field models are presented. The RANSAC is also employed in [12] to reduce the false-positive rate by removing the outliers from the lane candidates pool. A sliding window-based approach is used to extract the left- and right-lane markers using the highest peaks from the sliding window histogram for a horizontal scan line. The extracted lane markers are then connected using a polynomial fitting method and the final output is validated using a multi-sensor fusion method. A parallel constraint is applied [25] to an open snake model to detect broken lane markers. The lateral curvatures of the road are estimated in [15], and subsequently, the control points are extracted using vector fuzzy connectedness (VFC) technique. Road boundaries are built using a non-uniform B-spline interpolation method.

Fig. 3
figure 3

Flowchart of the proposed IBF scheme

Full size image

In few articles, the authors report the test results using their LD methods in tunnels [19, 26]. A real-time illumination invariant LDA (IILDA) using a third-order polynomial function of the longitudinal distance between vehicle and lane is proposed in [26]. In [26], it is reported that the average detection rate in the tunnel under daylight and in the night time is 91.17% in approximately 34.3 ms/frame. The lowest detection rate for this method observed when entering the tunnel is 87.4% because of sudden changing in illumination. A two-stage feature extraction (TSFE) method is proposed in [19] to detect two boundaries of lanes. To enhance robustness, lane boundary is taken as collection of small line segments. Thereafter, a modified Hough transform is applied to extract small line segments of the lane contour, which are then divided into clusters using the density based spatial clustering of applications with noise clustering algorithm. Then, the lanes are identified by curve fitting. False-negative test cases are reported in [19] when exiting from tunnel to the high way. A field-programmable gate array (FPGA)-based dual-stage lane detection (DSLD) algorithm [31] to cope with real-world challenges such as cast shadows, occlusion of lane markers, brightness variations, wear, etc. In the first stage, Sobel operator and adaptive threshold are used to extract lane edges, followed by Hough transform to extract the road markers is proposed. The second stage of the algorithm operates on the original grayscale image and identifies stripe features near several candidate points with highest probabilities to find the landmarks. These extracted features are then used to detect the lane boundaries with high accuracy.

The review of the literature reveals that the color-based solution alone is not enough for correct LD. Under daylight or white light, the white and yellow lane markers almost preserve their true colors. Therefore, global thresholding on different color plane in YCbCr or HSV color model can efficiently segment the color planes to extract the lane markers. However, in the tunnel because of the color light condition, the color of the lane markers does not preserve their true colors. Hence, the global thresholding methods produce poor results in the segmentation of lane markers. To alleviate this problem, in this paper, an efficient LD method is proposed to detect the lane markers efficiently in the tunnel with artificial colored light as well as highway under daylight condition. The review indicates that none of the previous articles addressed this issue of development of a suitable LD framework using IoT and cloud computing techniques under artificial colored light in tunnels and on highways. It may be noted that in many real-life situations, artificial colored light in tunnel and highway are experienced by drivers. Therefore, the main objective of this research is to develop a concrete model using vision-based LD and tracking method and cloud computing which can be used for driverless tasks in smart cities. The proposed method is shown to be efficient when testing on tunnel traffic images captured by a vehicular onboard camera. Time complexity of the proposed method is found to be less than that of other reported methods. Therefore, the proposed method fulfills one of the real-time application requirements.

This section has dealt with the current state of vehicle localization. It also presents the motivation and the key objective of the current investigation. Sect. 2 details the proposed framework and its modules in detail. Sect. 3 reports the findings and contribution of the proposed framework as well as comparison with the results obtained by existing work. Conclusion and future extension of this work are presented in Sect. 4.

The proposed lane detection and tracking module on IoT framework

The architecture of the IBF for LD and tracking is illustrated in Fig. 1. Three modules connect each other using a 5G network to sustain safety while autonomous driving. The corresponding flowchart is shown in Fig. 3. The 5G mobile networks require an end-to-end latency of within 1 ms, (including the wireless section; the required one-way latency of the wired section, particularly in 5G mobile networks, is about \(100\, \upmu \text {s}\)). The camera in the VB module acquires the road image and passes it to the lane detection algorithm (LDA) which detects both sides of the lane and the captured image along with the localized lane markers are stored in the SD card for further processing. If LDA fails to detect the equations for lane lines, then the current image accompanying the last stored image is sent to the CM where the current lane markers are detected using information from the last image. Thereafter, the amount of lane departure is measured, and if this value is bigger than a certain threshold, \( \tau \), then the overall control is delivered to man-powered remote which controls the vehicle in the vehicle remote controller (VR) module.

Vehicle board module

The IoT device to be employed in the implementation of the proposed scheme has the following specifications: Raspberry Pi 4 Model B, 1.5 GHz 64-bit quad-core Arm Cortex-A72 CPU, has three RAM options (2 GB, 4 GB, 8 GB), gigabit Ethernet, integrated 802.11ac/n wireless LAN, and Bluetooth 5.0. That enables the IoT module to execute the computational processes fast and to give a real-time decision. Raspberry Pi 4 Model B is installed in the vehicle, and it is chosen, because it decodes the video using the H265 standard and gives 4K P60 quality. Moreover, it can work at a high temperature up to \( 50^{\circ } \) which enables installing it outside the vehicle compartment. The Raspberry Pi Camera Module v2 is attached with Raspberry Pi to enable video recording of 8MP for the traffic scene and to store them on the SD card to be processed by the LDA. It is the major part of the VB module and it does the LD tasks in tunnels and on the high ways. The performance of the proposed algorithm is robust and does not get affected when illumination conditions change, like when colored artificial light presents.

The color-based thresholding is observed to have poor detection potentiality of lane markers under the colored light condition of the tunnel. However, the structural features are invariant to day light as well as colored light. Therefore, the knowledge of lane structure is considered to extract the structural features of the lanes in the proposed LD approach. The vanishing point (VP)-based method is used in LDA to identify the region of interest (ROI). Thereafter, lane markers are extracted using textural features that are extracted using the standard deviation filter. Lane markers are segmented using associated geometric characteristics, and then, these are clustered and fitted to find the formula of lateral lines for the lane. The output of the LDA is represented in two equations, one equation for each side of the lane, and if this output cannot be met, then the currently captured image along with the previous image is sent to the CM to be processed using LD using the cloud algorithm. The flowchart of the proposed structural feature-based LD algorithm is placed in Fig. 4. The details of LDA are explained in the following subsections.

Fig. 4
figure 4

Flowchart of the proposed LD algorithm

Full size image

The region of interest (ROI) extraction

The bottom half of the road image contains most lane segments, while the top half shows other objects not related to lane markers. The VP-based ROI extraction is a standard method in LD algorithms in which the ROI is considered to be the region under the VP. The VP is defined as the point in which the most extracted lines are intersected [26]. In this work, first, edges are extracted from the gray image by applying the Canny method because of its robustness against noise [27]. Second, the line segments are extracted using Hough transform. The 2-D accumulation array with the same size as the inputted raw image is used to get the VP coordinates. Each cell of the accumulative array is increased by 1 whenever it satisfies an extracted line equation. Finally, the cell with the maximum value is marked, and its indices are used to identify the VP coordinates. Figures  5 and  6 demonstrate the aforementioned steps.

Fig. 5
figure 5

Linear segments extraction using Hough transform. a Original tunnel image. b Canny edges. c Hough lines. d Hough transform

Full size image
Fig. 6
figure 6

Vanishing point extraction. a Accumulative map. b Extracted vanishing point

Full size image

Standard deviation filter

The standard deviation (SD) filter is a textural filter that provides information on the local intensity variation. The response of SD filter is smaller when the texture is smoother, and hence, the SD filter is used in this paper to act as an indicator of the degree of variability of pixel values in a region. This SD filter calculates the SD for the neighbor pixels to the pixel of interest. The SD at each pixel over a \( 3\times 3 \) neighborhood on each RGB color plane of the ROI is evaluated. The response of the SD filter at each pixel in a particular color plan is obtained using (1):

$$\begin{aligned} s_{k}\left( i,j\right) =\sqrt{\frac{\sum _{m=-1}^{1}\sum _{n=-1}^{1}\left[ {x}_{k}(i+m,j+n)-\mu _{k}(i,j)\right] ^{2}}{[(r\times {c})-1]}}, \end{aligned}$$
(1)

where k = 1, 2, and 3 for color planes R, G, and B, respectively. The mean \( \mu _{k} \) value is defined as given in Eq. (2)

$$\begin{aligned} \mu _{k}(i,j)=\frac{1}{9}\sum _{m=-1}^{1}\sum _{n=-1}^{1}x_{k}(i+m,j+n). \end{aligned}$$
(2)

The symbol \( x_{k}(i,j) \) represents the pixel at (i, j) position in the kth color plane. Based on the response of SD filter in each color plane, a monochromatic SD plane \( S_{q} \) is generated as in [28] using Eq. (3):

$$\begin{aligned} {S_{q}}=0.2989{S_{1}}+0.5870{S_{2}}+0.11405{S_{3}}. \end{aligned}$$
(3)

The \({{S}}_{q} \) plane for highway ROI and tunnel ROI is shown in Fig. 7c, d. From these two figures, it is observed that the SD filter has a very significant response in smooth areas. This variation is suppressed using the Gaussian smoothing kernel of size \( 3\times 3 \) for \( \sigma =8 \). The Gaussian mask and the associated weights for \( \sigma =8 \) are given in Eqs. (4) and (5), respectively:

$$\begin{aligned} G(x,y)= & {} \frac{1}{2\pi \sigma ^{2}}e^{-\frac{x^{2}+y^{2}}{2\sigma ^{2}}} \end{aligned}$$
(4)
$$\begin{aligned} {G_{(3\times 3, 8)}}= & {} \begin{bmatrix} 0.1105 &{}\quad 0.1114 &{}\quad 0.1105\\ 0.1114 &{}\quad 0.1123 &{}\quad 0.1114\\ 0.1105 &{}\quad 0.1114&{}\quad 0.1105 \end{bmatrix}. \end{aligned}$$
(5)

The smoothed ROI is defined by Eq. (6) and is shown in Fig. 8:

$$\begin{aligned} {S_{ROI}}={G_{(3\times 3, 8)}}*{{S_{q}}}. \end{aligned}$$
(6)
Fig. 7
figure 7

The \( S_{q} \) plane for highway ROI and tunnel ROI. (a) ROI image on the highway. (b) ROI image in the tunnel. (c) Standard deviation filter results for (a). (d) Standard deviation filter results for (b)

Full size image
Fig. 8
figure 8

The smoothed ROI. (a) The smoothed ROI on the highway. (b) The smoothed ROI in the tunnel

Full size image

Lane edge detection

The lane markers appear generally in four directions, vertical, horizontal, primary, and secondary diagonal [24]. The left and right lines for a lane are parallel in real world. Moreover, the perspective projection of these lines does not appear in horizontal or vertical direction in the image plane [29]. Instead, they tend to give high responses for \( 45^{\circ } \) and \( 135^{\circ } \) filter kernels [30, 31]. Some other proposed methods search within an angle range for a better LD template matching [18, 20]. Equation (7) shows the Sobel kernels \( A_{45^{\circ }} \) and \( A_{135^{\circ }} \) used to detect the lane markers

$$\begin{aligned} {A_{45^{\circ }}}= \begin{bmatrix} 0 &{}\quad 1 &{}\quad 2\\ -1 &{}\quad 0 &{}\quad 1\\ -2 &{}\quad -1 &{}\quad 0 \end{bmatrix}; \quad {A_{135^{\circ }}}= \begin{bmatrix} 2 &{} \quad 1 &{}\quad 0\\ 1 &{} \quad 0 &{}\quad -1\\ 0 &{}\quad -1 &{} \quad -2 \end{bmatrix}. \end{aligned}$$
(7)

The responses of these masks in the smoothed SD image are as in Eq. (8)

$$\begin{aligned} \begin{aligned} {\omega _{45^{\circ }}}&={A_{45^{\circ }}}*{{S_{ROI}}}\\ {\omega _{135^{\circ }}}&={A_{135^{\circ }}}*{{S_{ROI}}}. \end{aligned} \end{aligned}$$
(8)

The resultant of \( \omega _{45^{\circ }} \) and \(\omega _{135^{\circ }} \) is given in Eq. (9):

$$\begin{aligned} {R}=\sqrt{{\omega _{45^{\circ }}}^{2}+{\omega _{135^{\circ }}}^{2}}. \end{aligned}$$
(9)

A threshold, \( \alpha \), is used to binarize the final image R using the condition given in Eq. (10):

$$\begin{aligned} M_{ij}=\left\{ \begin{aligned} 1,\quad&if\,R(i,j)\ge \alpha \\ 0,\quad&otherwise. \end{aligned} \right. \end{aligned}$$
(10)

The value \( \alpha \) in Eq. (10) is set as 70 which is obtained by trial and error basis. Figure 9 shows the left and right edges of lane along with some redundant edge points not belonging to lane edges of a highway and tunnel ROIs. These redundant edges are further reduced through the connected component clustering approach.

Fig. 9
figure 9

The extracted \( 45^{\circ } \) and \( 135^{\circ } \) edges for the ROIs. (a) On the highway. (b) In the tunnel

Full size image
Fig. 10
figure 10

The binary connected components for ROI images. (a) On the highway. (b) In the tunnel

Full size image
Fig. 11
figure 11

\(\theta \) and \( y_{i} \) properties of a component

Full size image

Connected components detection and clustering

Binary connected components are detected using the 8-neighbor pixel connectivity. Figure 10a, b shows the different regions based on binary pixels in Fig. 9a, b. It is observed from these figures that the connected regions over the lane are discontinued. Moreover, other connected regions that do not belong to the lane would reduce the accuracy of the lane fitting stage. Consequently, these outliers would generate false lane markers which would reduce the safety level in autonomous vehicles. Therefore, there is a need to cluster the connected components to detect the candidate lane markers and minimize the outliers. Two parameters of each region, component angle \( \theta \) and y-axis intercept \( y_{i} \), are considered for the clustering the regions in Fig. 11. To calculate \( \theta \) of a region, first, the center of the corresponding region \( \left( x_{c},y_{c} \right) \) is calculated from Eq. (11), and then, the slope m of the longest diagonal line which passes through \( \left( x_{c},y_{c} \right) \) is evaluated to measure \( \theta \) as given in Eq. (13)

$$\begin{aligned} \begin{aligned}&x_{c}=\frac{\sum _{1}^{n}x_{i}}{n}\\&y_{c}=\frac{\sum _{1}^{n}y_{i}}{n}, \end{aligned} \end{aligned}$$
(11)

where \( \left( x_{i},y_{i} \right) \) refers to the ith pixel location in a region in line segment in Fig. 11, and n is the number of connected components of respective region. Let m be the slope of the largest diagonal line passing through \( \left( x_{c},y_{c} \right) \) of a region, as shown in Fig. 11. This largest diagonal line can be defined as in Eq. (12):

$$\begin{aligned} y_{i}=y_{c}-x_{c}(\tan \theta ). \end{aligned}$$
(12)

Then, the parameter \( \theta \) is given by (13):

$$\begin{aligned} \theta =\arctan (m). \end{aligned}$$
(13)

Regions satisfying both conditions of Eq. (14) are marked with same label:

$$\begin{aligned} \begin{aligned}&\left| \theta _{i}-\theta _{j} \right| \le \varepsilon _{1}\\&\left| y_{i}-y _{j} \right| \le \varepsilon _{2}, \end{aligned} \end{aligned}$$
(14)

where \( \varepsilon _{1} \) and \( \varepsilon _{2} \) equal 0.035 and 0.016, respectively. It is observed from the Fig. 12 that all the clusters belonging to a particular lane mark have similar label. Outlier clusters not belonging to lane mark increase the false detection of lane mark. To suppress these outliers, the regions with the nearest orientation to \( 135^{\circ } \) and \( 45^{\circ } \) are considered as the candidate lane marker regions. Therefore, any region having an orientation between (\( 30^{\circ } \) and \( 60^{\circ } \)) or (\( 120^{\circ } \)–\( 150^{\circ } \)) is considered as belonging to lane mark. Any region does not satisfy these two conditions is deleted. If none of the lane mark regions in Fig. 12 falls in the above orientation criteria, then information from the previous frame is compared with the under-processing frame.

Fig. 12
figure 12

Connected components clustering for ROI images. (a) On the highway. (b) In the tunnel

Full size image

Least-square line fitting

The least-square method finds the coefficient in such a way that the cost function (sum of the square of the deviations) between the data and the estimated ones are minimized. The lane markers generally take straight-line shapes within the ROI. Because it contains only a few portions of the road ahead of the vehicle [21]. Therefore, a polynomial fitting of first degree has been considered. Accordingly, the least-square approach has chosen to achieve the instantaneous curve line fitting. The polynomial fitting of the first degree considered in this work may cause double lines in each side of the lane. This problem cannot be avoided by minimizing \( \varepsilon _{1} \) and \(\varepsilon _{2}\) in Eq. (14) which may lead to increase of the number of clusters and, consequently, to a wrong LD. Therefore, the fitted lines with the nearest slope angle of \( 45^{\circ } \) and \( 135^{\circ } \) are considered as lane borders and the other fitted lines, if any, are discarded. Figure 13 shows the output of the line fitting algorithm and the final result after suppression.

The proposed approach employs structural-based features instead of the colored light feature-based thresholding for the lane marker extraction process. The use knowledge of the lane geometry is more potential features compared to the color features in capturing the lane marker points. Therefore, the proposed structure-based features efficiently detect the lanes under colored as well as in day light conditions. Consequently, the road image is segmented well which leads to correct detection.

Fig. 13
figure 13

The line fitting results and the final result after suppression . (a) Line fitting for ROI image on highway. (b) Line fitting for ROI image in tunnel. (c) Suppression of final lines for ROI image on highway. (d) Suppression of final lines for ROI image in tunnel

Full size image

Lane tracking

Tracking helps in case of inaccurate detection or occlusion caused by imperfect lane markers or by a vehicle which makes lane departure at the time of image acquiring. The Kalman filter is employed in this work to perform the lane tracking process as it helps to converge to real values faster than other methods. It is observed from Eq. (15) that Kalman gain \( k_\mathrm{g} \) becomes small if inaccurate detection happens which leads to large error, \( e_{m} \), in measurement. Consequently, the estimated value for tth frame becomes approximately the same as it in the previous \( (t-1)\)th frame. That signifies that detection relies more on information from the previous frame compared to that of the current ones:

$$\begin{aligned} \begin{aligned} k_\mathrm{g}&=\frac{e_{s}}{e_{s}+e_{m}}\\ s_{t}&=s_{(t-1)}+k_\mathrm{g}[m-s{(t-1)}]\\ e_{s{(t)}}&=(1-k_\mathrm{g})(e_{s{(t-1)}}), \end{aligned} \end{aligned}$$
(15)

where \( k_\mathrm{g} \) is Kalman gain, \( e_{s} \) is the error in estimation, \( e_{m} \) is the error in measurement, and \( s_{t} \) is the estimated value in the current frame. \( s_{(t-1)} \) is the estimated value for lane marker position in the previous frame, and the m is the measured position of lane marker in the current frame.

Cloud module and vehicle remote controller

The cloud module (CM) provides more reliability to the IBF, because the possibility of hardware failure is less than it in the VB. The main role of the CM is to differentiate between temporal failure and crucial failure. The crucial failure occurs when the vehicle has departed the lane which causes a non-safety problem for the ego vehicle and the other vehicles, as well. In such a case, the CM decides that VR should take over the driving remotely until the destination is reached. In case of VB failure, the CM receives two successive frames, \( f_{1} \) and \( f_{2} \), and applies the LDA on \( f_{2} \). If the lane markers are not correctly detected, then the LDA is applied on the \( f_{1} \) and the amount of departure, \( \alpha \), is calculated. The value of \( \alpha \) is compared to a safe threshold, \( \tau =10 \) cm; if \( \alpha \le \tau \), then the failure is classified as a crucial and the full control for driving the vehicle is immediately deliver to the VR. The VR controls the driving of the autonomous vehicle using a secure 5G connectivity.

Fig. 14
figure 14

LD results in different conditions. (a) Daylight. (b) Night time. (c) Rainy weather. (d) Occlusion

Full size image
Fig. 15
figure 15

LD results in different conditions. (a) Tree shadow. (b) Tunnel entering. (c) Inside tunnel. (d) Exiting from tunnel

Full size image

Experimental results

Based on the visual perception, the proposed framework efficiently localizes the vehicle within the lane markers in all the images of Figs. 14 and 15. It is also noticed that the proposed LDA does not detect the lane markers perfectly in case of blurred vision, small-radius curvature, and if there is another object which has the same features as lane markers such as footpath. In Fig. 16a, the left-lane markers are not properly detected, which signifies that the method relies on the information of previous frames. According to Fig. 16b, it is observed that if lane markers are deeply curved, it produces a wrong detection in the far field from the ego vehicle. The cross-walk boundaries in Fig. 16c act like lane markers and hence lead to false detection. The performance of proposed LDA is tested using Caltech [32] and DIML [26] datasets. Caltech contains road images for traffic scene with \( 640 \times 480 \) resolution. The DIML contains videos with \( 1280 \times 800 \) resolution and 15 frames per second. Three tunnel scenarios are considered under daylight and in the night time, entering, inside, and exiting of tunnel. The results of different tunnel scenarios of the proposed LDA and a comparison with method reported in [26] are shown in Table 1. The correct detection rate is obtained, and the average detection time in millisecond per frame is calculated. The detection rate, R, is calculated using Eq. (16):

Fig. 16
figure 16

Wrong detection cases . (a) Blurred vision. (b) Small-radius curved lane. (c) Cross-walk ahead the vehicle

Full size image
Table 1 Comparison between the percentage detection rate in the proposed method and IILD [26] in tunnel scenes, DIML
Full size table
Table 2 Comparison of detection rate of different methods
Full size table
$$\begin{aligned} R=\frac{C}{N}\times 100\%, \end{aligned}$$
(16)

where C is the number of true positive samples out of total N samples. The maximum detection time of the proposed method is 54 ms. Whereas, the average detection time of the proposed framework is found to be 31 ms/frame which fulfills the requirement of real-time application. The detection time depends on the number of redundant responses in (10) due to noise or similar structure(s) in the ROI. If the number of redundant points increases, then the time required to extract the connected components and connected component clustering also increases. Furthermore, when the detection process fails it relies on the previous lane marking data. This information is provided by the detected lane markers from the previous frame. The time of normal LD is added to the time of tracking which makes the detection time longer. The detected lines have matched with the ground truth based on \( \theta \) and \( y_{i} \) parameters. The findings from the proposed LDA are compared with those reported methods in [19, 26, 31]. Table 2 shows experimental results in different weather and lighting conditions. In the Caltech dataset, the scene is visualized through a circular window. Commonly, the camera captures the scene in a rectangular window. The proposed method when uses the circular vision of this dataset of the error in ROI extraction stage becomes high. However, the proposed method still produces improved LD results.

Conclusion and future work

A novel vehicle localization framework is proposed in this paper along with LD and tracking scheme. The framework consists of three modules. The VB module is the main module that runs the LDA using Raspberry Pi 4 Model B equipped with a compatible camera. The second module on cloud CM ensures the robustness of the VB and detects the lane markers in case of any failure in VB. Moreover, the CM measures the amount of lane departure which has been made by the vehicle due to the failure and provides this information to the VR stage, which finally controls the vehicle to save lives and properties. A novel and illumination invariance lane detection algorithm is also proposed. The vanishing point-based ROI extraction is employed to reduce the time complexity and to enhance detection accuracy. The input image is filtered to extract structural features using the standard deviation filter, followed by the Gaussian filter to reduce noise. The candidate lane markers are detected using \( 45^{\circ } \) and \( 135^{\circ } \) filter kernels for edge detection. The connected components are clustered according to the cluster slope and y-intercept. The least-square lane fitting approach is used to form the left and right line equations for the lane. Finally, the lane is tracked using the Kalman filter. The experimental results demonstrate the robustness of the proposed LDA in tunnel scenarios and on highways, as well. The proposed framework is shown to be more efficient than other reported works. The average detection time fulfills real-time application requirements. But still, limitation and shortcomings exist in the proposed scheme. The fitting algorithm used does not work well in case of high curvature lane. The differentiation between left and right lines of the lane is required in DAS. Therefore, the proposed LDA needs further improvement to overcome this shortcoming. Enhancement of blurred vision which could happen because of shaking or under rainy weather conditions needs to be addressed and solved for achieving a robust and efficient vehicle localization framework. Furthermore, for ROI extraction and efficient detection of curved lane, Deep Learning methods such as RCNN, LSTM, and GAN can also be gainfully employed which can be take up as a future extension of the paper.