Data-Driven Risk Assessment with Scenario Classification for Collision Avoidance in Left/Right Turn Across Path Conflicts

Abstract

Traffic safety is one of the vital issues in developing autonomous and assisted driving systems. To achieve higher level of driving automation, it becomes necessary to have a reliable Risk Assessment (RA) method that not only responds to current traffic conditions but also anticipates future scenario propagation. Regardless of the driver intentions, traditional deterministic risk indicators like Time-To-Collision (TTC) have proven effective but fall short in addressing the inherent uncertainty in future propagation, especially under conflict scenarios influenced by interdependent decisions of traffic participants. Acceleration for Collision Avoidance (ACA) emerges as a flexible risk criterion adaptable to different collision-avoidance maneuvers. Focusing on the Left/Right Turn across path conflicts, this work aims to propose an innovative surrogate risk indicator for collision avoidance with the combination of ACA criteria and scenario classification using Hidden Markov Models (HMM). Based on a near-miss video database, we further train and evaluate the presented model, supplying interpretability and adaptability of risk assessment in complex conflict scenarios.

1 Research Background

In the development of future autonomous and advanced driver assistant systems (ADAS), a key technology is a reliable risk assessment method, which should not only react according to the current traffic states but also predict their future evolution to make appropriate actions.

The deterministic methods usually integrate the simple vehicle model like the constant yaw rate and acceleration (CYRA) with quantitative index like the Time-To-X metrics (such as the Time-To-Collision), Post-encroachment time, and responsibility-sensitive safety (RSS). These common measures of the collision risk are proven to be effective in evaluating the severity of conflicts. However, the deterministic computation methods are inherently unable to reflect the uncertainty of future traffic evolution. To overcome this limitation, probabilistic inference methods such as Hidden Markov Model (HMM), Dynamic Bayesian Networks (DBN), Support Vector Machines (SVM), Recurrent Neural Network (RNN, include LSTM) are evaluated to be powerful in intention inference or direct trajectory prediction.

For the risk assessment problem in conflicts, the decision making of traffic participants are influences by each other, resulting in different near-collision scenarios. Therefore, single deterministic matrix usually failed to reflect the actual severity. Recently, the concept of Acceleration for Collision Avoidance (ACA) appears as a more flexible risk criterion that can adaptive to various collision avoidance maneuvers. In this work, we focus on the Left/Right Turn Across Path (TAP) scenarios and combine the ACA criteria with the scenario classification based on HMM to propose a new surrogate indicator, which will be verified using a near-miss video database [1].

2 ACA Indicators in Left/Right Turn Across Path Conflicts

Figure 1 illustrates the potential ACA risk indicators in the Left/Right turn across path dilemma which can be divided into two scenarios. In scenario I, the ego vehicle passes the interaction before the turning vehicle entering the conflict area, following the driving intention of the ego vehicle driver. In scenario II, the ego vehicle lets the turning vehicle pass first and carried out a waiting maneuver, which would lead the vehicle to a complete stop or a relatively slow speed due to the driver braking maneuver. The calculation of proposed risk indicators based on the real-time measurements are expressed as follows, respectively.

$$\begin{aligned} R_1^{\text {TAP}} = ACA_{\text {pass}}^{\text {TAP}} \end{aligned}$$

(1)

$$\begin{aligned} R_2^{\text {TAP}} = \left\{ \begin{array}{l} ACA_{\text {yield}}^{\text {TAP}}, \text {if: } \frac{\left( V_{e 0}+G_{e 0} \cdot \tau \right) }{ACA_{\text {yield}}^{\text {TAP}}}+\tau <t_{out}\\ ACA_{\text {stop}}^{\text {TAP}}, \text { others} \end{array}\right. \end{aligned}$$

(2)

with

$$\begin{aligned} \left\{ \begin{array}{l} ACA_{\text {yield}}^{\text {TAP}}=\frac{G_{e 0} \cdot \tau ^2-2 t_{o u t}\left( V_{e 0}+G_{e 0} \cdot \tau \right) +2d_{e c}}{\left( t_{\text {out }}-\tau \right) ^2} \\ ACA_{\text {stop}}^{\text {TAP}}=\frac{\left( V_{e 0}+G_{e 0} \cdot \tau \right) ^2}{G_{e 0} \cdot \tau ^2+2 V_{e 0} \cdot \tau +2\left( X_{e 0}-d_{ec}\right) } \\ ACA_{\text {pass}}^{\text {TAP}}=\frac{G_{e 0} \cdot \tau ^2-2 t_{\text {in }}\left( V_{e 0}+G_{e 0} \cdot \tau \right) +2\left( d_{e c}+L_e\right) }{\left( t_{\text {in }}-\tau \right) ^2} \end{array}\right. \end{aligned}$$

(3)

Fig. 1.

Two collision avoidance scenarios and corresponding risk-indicator

Where, the real-time measurements include the target vehicle speed $V_{e0}$, acceleration $G_{e0}$, displacement from vehicle front side to the cross-point $d_{ec}$, and the entry/leaving time of the opposite vehicle $t_{in}$ and $t_{out}$ calculated by the opposite vehicle speed. The target vehicle length is $L_e$ and the human response delay is expressed by $\tau$. Here we note that even though the two scenario can be clearly identified, e.g., for a sample case from the data-base, its hard to select the risk indicator according to the real-time interactive situation.

Fig. 2.

Framework of the proposed risk assessment method

3 Stochastic Risk Indicator

In this work, we propose the conflict scenario prediction method based on HMM. The process of the presented method is illustrated in Fig. 2. For training the HMM models representing different conflict scenarios, the trails of mobility features (e.g., relative position, velocity) of vehicles in conflict situations are obtained. As the observation probabilistic model, Gaussian mixture model (GMM) is applied to characterize the continuous mobility features.

With trained HMM models for a given trail observation of the mobility features O(t), we propose the stochastic risk indicator under Left/Right turn with conflicts, expressed as follows

$$\begin{aligned} \hat{R}^{\text {TAP}}(t) = \sum _{i=1,2} \omega _i \cdot R_i^{\text {TAP}} \end{aligned}$$

(4)

where, the weighting factor $\varSigma \omega _i=1, i=1,2$ indicates the predicted probability of scenario i, which can be calculated by the following logistic function

$$\begin{aligned} \omega _1 = \frac{1}{1+e^{-(\log P(O(t)|\lambda _1)-\log P(O(t)|\lambda _2))}}, \omega _2 = 1 - \omega _1 \end{aligned}$$

(5)

The probabilities $\log P(O(t)|\lambda _i)$ with regard to the observation O(t) under different HMM models $\lambda _i$ can be calculated by forward algorithm.

4 Scenario Prediction with GMM-HMM

Specifically, one particular HMM expressed by the parameter set $\lambda _i$ is trained for each conflict scenario $i = 1, 2$. Each HMM model $\lambda$ (index i is omitted) includes Q possible hidden states H, continuous output set V, state transmission probabilities $A=\{a_{q,p}\}$, observation GMM model $\varTheta$, and initial state probabilities $\pi$. The trail of each feature is represented by a column vector, expressed as

$$\begin{aligned} \boldsymbol{x_n} = [x_{n,1}, x_{n,2},..., x_{n,T}]^T \end{aligned}$$

(6)

where t is the time step index. For each scenario, all the mobility features from different conflict events are truncated to the same length T, and the set of all features can be expressed as the following data matrix, i.e.,

$$\begin{aligned} \boldsymbol{X} = [\boldsymbol{x_1}, \boldsymbol{x_2},..., \boldsymbol{x_N}] \end{aligned}$$

(7)

In this work, we apply the GMM to characterize the continuous mobility features, together with the training process of the HMM model. Every column in the data matrix represents the trail of every mobility features whose probability density can be fit by a super position of M Gaussian distribution, with the following expression

$$\begin{aligned} P\left( \boldsymbol{x_n}\right) =\sum _{m=1}^M \sum _{q=1}^Q \omega _{n, q, m} P\left( \boldsymbol{x_n} \mid \mathcal {N}\left( \mu _{n, q, m}, \sigma _{n, q, m}\right) \right) \end{aligned}$$

(8)

where $\omega _{n, q, m}$ means the weight when the type n feature probability is modeled by the m Gaussian component from the q hidden state, and $\mu _{n, q, m}, \sigma _{n, q, m}$ are the corresponding mean value and standard deviation of the Gaussian distribution component. Let $\varTheta _n=\{\omega _{n, q, m}, \mu _{n, q, m},\sigma _{n, q, m}\}$ express the GMM parameter set for the feature n, the observation model $\varTheta = \{\varTheta _1, \varTheta _2,..., \varTheta _N\}$ includes the parameter set for all N types of mobility features. For the HMM training, we use the maximum likelihood estimation (ME) algorithm to determine all the GMM parameters which most likely fit the data set $\boldsymbol{x_n}$, satisfying

$$\begin{aligned} \varTheta _n=\arg \max _{\varTheta _n} P\left( \boldsymbol{x_n}\right) =\arg \max _{\varTheta _n} \prod _t^{T} P\left( \boldsymbol{x_n}(t)\right) \end{aligned}$$

(9)

The training algorithm is started with an initial guess of $\varTheta$ and then the probability of taking the value of $\boldsymbol{x_n}(t)$ by the Gaussian component of m from the hidden state q can be calculated as follows

$$\begin{aligned} \gamma _{n, q, m, t}=\frac{ \omega _{n, q, m} \mathcal {N}\left( \boldsymbol{x_n}(t) \mid \mu _{n, q, m}, \sigma _{n, q, m}\right) }{ \sum _{j=1}^M \omega _{n, q, j} \mathcal {N}\left( \boldsymbol{x_n}(t) \mid \mu _{n, q, j}, \sigma _{n, q, j}\right) } \cdot \gamma _{t}(q) \end{aligned}$$

(10)

where, $\gamma _t(q) = P(Q_t = q \mid \boldsymbol{X}, \lambda )$. Then the corresponding updating rule for $\varTheta _n$ can be obtained as

$$\begin{aligned} w_{n,q,m} = \frac{\sum _{t=1}^{T} \gamma _{n,q,m,t}}{\sum _{t=1}^{T} \gamma _t(q)} \end{aligned}$$

(11)

$$\begin{aligned} \mu _{n,q,m} = \frac{\sum _{t=1}^{T} \gamma _{n,q,m,t} \boldsymbol{x_n}(t)}{\sum _{t=1}^{T} \gamma _{n,q,m,t}} \end{aligned}$$

(12)

$$\begin{aligned} \varSigma _{n,q,m} = \frac{\sum _{t=1}^{T} \gamma _{n, q, m, t} (\boldsymbol{x_n}(t) - \mu _{n,q,m} )^2}{\sum _{t=1}^{T} \gamma _{n, q, m, t}} \end{aligned}$$

(13)

The complete algorithm is given as follows

5 Results and Analysis

A near-miss video database that focuses on collisions, near-misses, and normal driving data in real traffic has been developed in prior studies [1]. By using a drive-recorder installed on the front shield of vehicles, front-view video data, vehicle speed, accelerations, and GPS data were collected. The mobility features of the turning vehicle were obtained through image processing based on front-view videos. In the database we focus on the right turn across path conflicts and manually divide them into the two scenarios shown in Fig. 1. Then we trained the two GMM-HMM models respectively with the corresponding data trail of the mobility features. We first examine the accuracy of GMM-HMM models regarding the mobility feature predictions, as shown in Fig. 3. According to the prediction results, we notice that as general models of the two scenario classifications, the object vehicle’s mobility features sometimes cannot be exactly predicted. However, the Ego vehicle’s future behavior can be reasonably anticipated. In cases from scenario I, the Ego vehicle usually slows down to a low speed or stops and in scenario II cases, the global longitudinal/lateral speeds do not change much.

Fig. 3.

Prediction results of Ego vehicle and Obj vehicle positions, (a) case from scenario I, (b) case from scenario II

Fig. 4.

Likelihood estimation of test cases, left: Comparison with different models, right: real-time estimation of a single case (observation within 0.5 s)

The scenario prediction results regarding the likelihoods of the 10 test cases are shown in Fig. 4. Treating the case no. 7 as a real-time scenario, the weight factor $\omega _i$ variation is also illustrated. The test cases no. 1-no. 5 are selected from the scenario I and others from scenario II. According to the calculation results, all the cases are correctly identified with the corresponding class. Some cases are with remarkable significance like case no. 1. We also note the real-time estimation sometimes leads to contrary prediction with real-time observations.

6 Conclusion

Focusing on the Left/Right Turn across path conflicts, this work proposed an innovative surrogate risk indicator including scenario classification using GMM-HMM models, which show good performances in scenario classifications.