Forschung im Ingenieurwesen

, Volume 83, Issue 2, pp 149–161 | Cite as

Prediction of highway lane changes based on prototype trajectories

  • David AugustinEmail author
  • Marius Hofmann
  • Ulrich Konigorski


The vision of automated driving is to increase both road safety and efficiency, while offering passengers a convenient travel experience. This requires that autonomous systems correctly estimate the current traffic scene and its likely evolution. In highway scenarios early recognition of cut-in maneuvers is essential for risk-aware maneuver planning. In this paper, a statistical approach is proposed, which advantageously utilizes a set of prototypical lane change trajectories to realize both early maneuver detection and uncertainty-aware trajectory prediction for traffic participants. Generation of prototype trajectories from real traffic data is accomplished by Agglomerative Hierarchical Clustering. During clustering, the alignment of the cluster prototypes to each other is optimized and the cohesion of the resulting prototype is limited when two clusters merge. In the prediction stage, the similarity of observed vehicle motion and typical lane change patterns in the data base is evaluated to construct a set of significant features for maneuver classification via Boosted Decision Trees. The future trajectory is predicted combining typical lane change realizations in a mixture model. B‑splines based trajectory adaptations guarantee continuity during transition from actually observed to predicted vehicle states. Quantitative evaluation results demonstrate the proposed concept’s improved performance for both maneuver and trajectory prediction compared to a previously implemented reference approach.

Prädiktion von Fahrstreifenwechselmanövern basierend auf prototypischen Trajektorien


Die Vision des automatisierten Fahrens ist es, sowohl die Verkehrssicherheit als auch die Effizienz zu erhöhen und gleichzeitig den Fahrgästen ein komfortables Reiseerlebnis zu bieten. Dies erfordert, dass autonome Systeme die aktuelle Verkehrssituation und deren wahrscheinliche Entwicklung korrekt einschätzen. In Autobahnszenarien ist die frühzeitige Erkennung von Einschermanövern für eine risikobewusste Manöverplanung unerlässlich. In diesem Beitrag wird ein statistischer Ansatz präsentiert, der eine Reihe von prototypischen Fahrstreifenwechsel-Trajektorien nutzt, um sowohl eine frühzeitige Manövererkennung als auch eine Trajektorienvorhersage für Verkehrsteilnehmer unter Berücksichtigung von Unsicherheiten zu realisieren. Die Generierung von Prototyptrajektorien aus realen Verkehrsdaten erfolgt durch Agglomerative Hierarchical Clustering. Während des Clusterings wird die Ausrichtung der Cluster-Prototypen zueinander optimiert und die Kohäsion des resultierenden Prototypen bei der Fusion zweier Cluster beschränkt. Im Zuge der Prädiktion werden die Ähnlichkeiten der beobachteten Fahrzeugbewegungen zu typischen Spurwechselmustern in der Datenbank ausgewertet, um signifikante Merkmale für die Manöverklassifizierung mittels Boosted Decision Trees zu konstruieren. Die zukünftige Trajektorie wird vorhergesagt, indem typische Fahrstreifenwechselrealisierungen in einem Mischungsmodell kombiniert werden. B‑Splines-basierte Trajektorienanpassungen garantieren Kontinuität an den Übergängen von beobachteten zu vorhergesagten Fahrzeugzuständen. Quantitative Bewertungsergebnisse zeigen, dass das vorgeschlagenen Konzept die Ergebnisse sowohl für die Manöver- als auch für die Trajektorienvorhersage im Vergleich zu einem zuvor implementierten Referenzansatz verbessert.

1 Introduction

The development of automated driving functions is a central activity of industry and science. Currently, first assistance systems combining both longitudinal and lateral guidance are introduced on the market. These systems are mainly designed to assist on highway-like roads. The structured environment limits the variety of possible scenarios which need to be autonomously handled by the assistance system. Nevertheless, all public roads are still dominated by human drivers. In order to autonomously drive in public traffic the assistance systems need to correctly evaluate the current traffic scene and its likely evolution. This includes estimating the driving intentions of traffic participants, detecting maneuvers being performed, and predicting future motion of surrounding traffic. All these tasks are non-deterministic and uncertainties in maneuver and motion estimation need to be considered. This is especially important for automated driving functions of SAE level 3 or higher which assume the whole driving task in certain situations to allow the driver to focus on self-chosen activities [1].

In this contribution, highly automated driving (HAD) on highways is addressed with the focus on early lane change maneuver detection and uncertainty-aware trajectory prediction. Sudden cut-in maneuvers of traffic participants merging onto the driving lane of the HAD vehicle can easily cause dangerous situations due to undercutting the safety distance. Therefore, early lane change maneuver detection is crucial to increase safety and comfort via driving strategy adaptation. The ability of anticipating future events if often referred to as “driving with foresight”. For trajectory planning, a reliable motion prediction is also required to ensure collision-free operation. Before outlining the basic concepts of the proposed approach in Sect. 1.2 an overview of related work in the field of motion prediction is provided.

1.1 Related Work

Motion prediction is a research area since decades. The survey articles [2] and [3] provide overviews about a variety of published approaches and fields of application related to road transport. Additionally, both provide suggestions on how motion prediction approaches can be categorized. One way is to classify motion models based on their (increasing) level of abstraction into “physics-based”, “maneuver-based” and “interaction-aware” [2]. In [3] a distinction into “pure-prediction”, “driver-intention-recognition” and “knowledge-based” is introduced. Approaches modeling the driver intention as a discrete random variable are, additionally, subdivided according to the applied classification algorithms into “discriminative” and “generative”. When traffic interaction needs to be considered mainly generative classifiers are applied [3]. A maneuver can be regarded as the realization of a driver’s intention. Therefore the task of maneuver detection is related to driver intention recognition. Dependent on whether the features used to infer the intention take into account inter-vehicle dependencies, one can differentiate between “interaction-aware” or pure “maneuver-based” methods.

Focusing on driver intention recognition, a large number of approaches do assess the traffic environment on an abstract level while neglecting the prediction of physical motion which is crucial for criticality assessment and trajectory planning. The opposite concept relying purely on physically-based motion prediction is only valid for a limited time horizon of less than a second [2]. Approaches that realize both motion detection and trajectory prediction can be classified as methods of pattern recognition and methods fusing dynamic motion models with behavior descriptions [4]. Methods of pattern recognition do not only consider the current state of an object but also its history. These states are compared to a database of previously generated motion patterns, each linked to a specific maneuver type. In scenarios like intersections, in which the road topology predefines typical trajectories, the patterns can be directly created from map data [5]. In [6] trajectory prediction for lane changes is realized combining a physical motion model with artificially created motion patterns. At each time step a new set of motion patterns is generated based on the current vehicle state, the road parameters, and the detected maneuver. Another possibility for pattern creation is to analyze real traffic data. The approach [7] utilizes a set of recorded highway lane change trajectories as prototypes in a Bayesian network classifier for maneuver detection. Clustering methods are not applied [8]. Motion prediction for traffic participants is realized using multi-agent simulation. In [9] a cluster-based technique that learns the typical motion patterns using pairwise clustering is designed for a visual pedestrian tracking system placed in an entry hall. For static environments like interior rooms of buildings or intersections it is fairly easy to define regions of start and end points of motion patterns compared to environments with varying topology like highways.

1.2 Proposed Approach

In this contribution we propose an extension to a previously published approach for lane change maneuver detection and uncertainty-aware trajectory prediction in highway scenarios [10]. The approach is centered around the idea that the variety of lane change realizations can be clustered into typical motion patterns which support early maneuver recognition and trajectory prediction. Similar lane change courses are aggregated into groups, each represented by a prototype trajectory. Figure 1 shows a schematic diagram of the proposed approach.
Fig. 1

Process diagram of the proposed approach. Maneuver and trajectory estimates can be used as inputs for HAD maneuver and trajectory planning

The objectives of learning stage are to produce a set of representative lane change trajectories, which can be utilized to formulate descriptive features for maneuver recognition, and to learn a convenient classification algorithm. Prototype trajectories are generated from recorded highway footage of traffic participants applying Agglomerative Hierarchical Clustering (AHC) in combination with a preprocessing step optimizing the trajectory alignment. Additionally, constraints are introduced to improve cluster cohesion and define the end condition for pairwise clustering. To reduce the diversity of lane change embodiments and thus limit the number of required representatives we categorize lane change trajectories solely by their lateral course. This enables maneuvers of different absolute velocity but similar lateral movement to be cumulated in the same cluster. Each lane change cluster is represented by lateral position and velocity prototypes. Details of typical motion pattern extraction from real traffic data are provided in Sect. 3.

In the estimation stage, which is covered in Sect. 4, we buffer the latest states for each Vehicle Of Interest (VOI). The vehicle state histories are referred to as partially observed trajectories. A distance value is computed for each pair of partial and prototype trajectory under optimal alignment for lateral position and velocity independently. For each lane change type [Lane Change Left (LCL) or Lane Change Right (LCR)] we select the pairs with minimal distances as best matches. The distance values of the best matches for LCL and LCR are utilized to construct features for maneuver detection. Additionally, the lateral offset of a vehicle within its lane and its lateral velocity is applied for maneuver inference. All these features are derived from motion states of each VOI, individually. Vehicle interdependencies or infrastructure information are not taken into account. This limits the possible prediction horizon because potential causes for lane change maneuvers are not considered. Maneuvers can only be detected during execution. On the other hand, approaches with lower abstraction level limit the requirements on environment sensing system (Sect. 2), i.e. there is no need to detect the vehicle driving in front of a VOI in order to reason about its influence on driving intentions. Due to the choice of features the proposed approach can be categorized as maneuver-based driver intention estimation.

The extensions compared to our previous publication [10] mainly lie in advances of prototype usage for both maneuver detection and maneuver-based trajectory prediction. We train an ensemble for decision trees by adaptive boosting for maneuver classification and discuss its performance in comparison to the generative classifier Gaussian Discriminant Analysis (GDA). The trajectories of the VOIs are predicted according to the estimated maneuvers. In case of lane changes the future lateral motion is expressed as a weighted sum of all maneuver-related prototypes forming a Gaussian Mixture Model (GMM). The weight of each prototype correlates with its proximity to the partially observed trajectory. Initial offsets between the actual lateral position of a VOI and the trajectory prediction are compensated by applying a B-spline approximation. Longitudinal motion is covered independently by a (nearly) constant acceleration model. Both classification and motion prediction are evaluated with a test set of recorded highway data. The results are discussed in Sect. 6.

2 Environment Sensing

The test vehicle, depicted in Fig. 2, is equipped with a close-to-series sensor setup. For target tracking two mid-range radars are mounted centrally underneath front and rear bumper facing in longitudinal direction. Four short-range radars are mounted at the vehicle corners to cover the vehicle’s lateral area. One monocular camera is attached to the inside of the front windshield. A Kalman-filter based tracking system fuses information of all sensors. While radars have their strength in measuring radial distances and radial velocities, their angular resolution is imprecise compared to camera measurements. As our approach requires a high lateral accuracy, we focus on camera-detected or radar-and-camera-detected objects only for training and evaluation. This limits the scope of application to vehicles in the front area, which are of concern for cut-in maneuver detection.
Fig. 2

The Opel HAD test vehicle is equipped with radars and a monocular camera for environment detection

The tracking system produces, for each sensed object, a local estimate of its state vector \(\mathbf{X}^{\mathrm{(target)}}\):
where \(x\) and \(y\) are local Cartesian coordinates, \(\theta\) the relative heading angle, and \(v\) the absolute velocity.
The camera system detects lane markings and approximates their course as cubic functions in the local coordinate system:
with polynomial coefficients \(c_{3}\), \(c_{2}\), \(c_{1}\), and \(c_{0}\).
To be independent of the ego motion of the test vehicle, the states of all tracked objects are transformed into the Frenet frame. Centerlines of corresponding driving lanes of each traffic object are utilized as reference curves. Vehicle trajectories \(\mathbf{\gamma}_{i}(t):[0,T_{i}]\rightarrow\mathbb{C}\) are expressed in the lane-relative configuration space \(\mathbb{C}\) which is \([s,\dot{s},d,\dot{d}]\) with \(s\) being the traveled distance from the start of the maneuver, \(\dot{s}\) the velocity in longitudinal direction, while \(d\) and \(\dot{d}\) being the lateral offset to the lane’s centerline and the lateral velocity, respectively. To cut down diversity of maneuver executions, the configuration space \(\mathbb{C}\) is reduced to \([d,\dot{d}]\) for cluster learning and representation. The lateral offset, \(d\), can be derived using the minimal distance of the vehicle to its left, \(d_{\mathrm{l}}\), and right lane markings, \(d_{\mathrm{r}}\):
Thereby, the lane width is taken into account inherently. To reduce computational effort for the calculation of \(d_{\mathrm{l}}\) and \(d_{\mathrm{r}}\), lane marker polynomials are locally linearized. The maximal error of this approximation is below our measurement accuracy. The lateral velocity, \(\dot{d}\), is derived by applying a low-pass filtered difference quotient. The longitudinal velocity, \(\dot{s}\), is assumed to be equal to the vehicle’s absolute velocity, \(v\). The traveled distance, \(s\), between two consecutive measurements is computed applying a constant velocity model [10].

3 Motion Pattern Recognition

Due to diversity of traffic situations and individual driving styles the courses of lane changes highly vary. Nevertheless, we assume that the variety of lane change realizations can be mapped to a finite number of typical motion patterns. Such a pattern represents a group of similar maneuver executions. As the number of clusters and the course of their representing trajectories are unknown, real highway traffic data is analyzed by a prototype-based clustering technique called Agglomerative Hierarchical Clustering. Each maneuver type is analyzed separately. The structured highway environment limits vehicle maneuvers with respect to their lateral motion to Lane Change Left, Lane Keeping, and Lane Change Right. Each data point in the data set is assigned to one maneuver type via automatic labeling.

3.1 Labeling

Lane changes can easily be detected in a data set by searching for lane marker crossings. A more challenging aspect for reliable automatic labeling is to define start and end time of a maneuver. Before a vehicle passes the centerline of its starting lane we regard the motion as lane centering and assign it to the lane keeping maneuver. Also a possible overshoot on the target lane is treated as lane keeping. Additionally, it was determined that a maneuver bound is reached when the vehicle’s absolute lateral velocity exceeds or undercuts a velocity threshold of \(0.2\,\mathrm{m/s}\). To improve the lateral velocity estimate, we approximate the vehicle’s position course via B‑spline curve and calculate lateral velocity, \(\dot{d}\), via derivation.

3.2 Preprocessing and Clustering

Objectives of traffic-related motion pattern recognition are to automatically assign similar vehicle trajectories to the same cluster and to compute a common representation for each group, which we denote as prototype trajectory. Due to formulation of goals prototype-based clustering methods, like AHC or k‑Means, are suitable solution approaches. Both methods were investigated and compared in [10] with AHC performing superior for highway scenarios.

AHC is an algorithm iteratively merging the two closest clusters until only one single cluster exists or a termination condition is fulfilled [11]. Initially, each cluster is formed by a unique trajectory of the training data set. When two clusters are merged, they are replaced by a common prototype and their similarity to all other cluster representatives must be reevaluated. Cluster representations consist of time-dependent mean, \(\mu(t)\), and variance, \(\sigma(t)\), values. The mean of one cluster, \(C_{m}\), is calculated from its comprised trajectories by:
with \(N_{m}\) denoting the total number of trajectories in the cluster. For prototype generation, AHC is only applied to position trajectories, \(\mathbf{d}\in\mathbb{D}\). \(\mathbb{D}\) denotes the set of position trajectories relative to the centerline of the driving lane. For maneuver classification, it turned out to be beneficial to utilize both position and velocity prototypes, so in this case \(\mathbf{d}\in\mathbb{D}\times\mathbb{V}\) with \(\mathbb{V}\) denoting the set of velocity trajectories [10].
The scatter of cluster trajectories around the prototype mean is expressed via the variance
In order to express the relation between two trajectories a measure of proximity is required. We chose the average Euclidean distance between the two trajectories \(\mathbf{d}_{i},\mathbf{d}_{j}\in\mathbb{D}\) as dissimilarity measure:
$$\begin{aligned}\displaystyle&\displaystyle\qquad t_{\mathrm{min}}=\min(t_{0,i},t_{0,j})\\ \displaystyle&\displaystyle\qquad t_{\mathrm{max}}=\max(t_{0,i}+T_{i},t_{0,j}+T_{j})\\ \displaystyle&\displaystyle\qquad T=t_{\mathrm{max}}-t_{\mathrm{min}}\end{aligned}$$
where \(t_{0,i}\), \(t_{0,j}\), \(T_{i}\), and \(T_{j}\) are the starting times and the total maneuver durations of trajectories \(\mathbf{d}_{i}\) and \(\mathbf{d}_{j}\), respectively. For comparison of trajectories of unequal length, the shorter ones are extended by keeping their initial and/or final value. The factor \(\frac{1}{T}\) averages the Euclidean distance to negate the influence of duration of the two trajectories being compared on their dissimilarity value.
Fig. 3

Prototypical lane change trajectories created by Agglomerative Hierarchical Clustering. For each lane change type six clusters are created from real traffic data. a Position prototypes for changes to the left lane. b Position prototypes for changes to the right lane. c Velocity prototypes for changes to the left lane. d Velocity prototypes for changes to the right lane

A major influence factor on the dissimilarity between two lane change trajectories is their mutual alignment [10], which in the case of highway trajectories is mainly determined by the chosen labeling approach. In order to handle possible labeling inconsistencies we studied different approaches for active alignment of trajectories. Best results were achieved by an adjustment strategy minimizing the dissimilarity of each pair of trajectories by computing an individual optimal configuration [10]. In AHC the alignment of cluster representatives needs to be updated after each clustering iteration.

In order to improve cluster quality and to find the optimal number of clusters per maneuver type, conditioning constraints for cluster merging, quantifying a cluster’s cohesion and/or separation, were investigated in [10]. The average standard deviation of a cluster prototype was introduced to evaluate cluster cohesion. During the AHC, merging of two clusters was only permitted when the new prototype’s average standard deviation undercuts the threshold of \(\sigma_{\mathrm{max}}=0.2\,\mathrm{m}\) [10]. The second constraint only allows cluster merging when the overall silhouette coefficient is increased thereby. The silhouette coefficient combines both cohesion and separation. It is parameter-free and its value can vary between \(-1\) and 1, the greater the better. If a constraint impedes merging a pair of two clusters, their pairwise dissimilarity is set to infinity and the second best pair is merged. Once, when no pair of two clusters can meet the utilized constraint, the clustering ends. Both quality constraints, limitation of average standard deviation and maximization of overall silhouette coefficient, enhance cluster quality and classification results, with the former constraint marginally outperforming the latter. Figure 3 shows prototypical lane change motion patterns created using the presented AHC approach.

4 Maneuver Classification

Motion prediction for traffic participants can be split into two consecutive tasks. In the first step, the performed maneuvers need to be detected. Subsequently, the most probable future trajectories according to the maneuver estimate can be calculated. Maneuver detection is a typical classification problem. The highway topology limits the set of possible lateral maneuvers to \(C=\mathrm{\{LCL,LK,LCR\}}\). We implemented maneuver inference based on a set of features which can be derived from the observed motion of individual vehicles. Vehicle interaction is not taken into account. This approach limits computational and sensory requirements. Additionally, not relying on strong assumptions about vehicle interdependencies increases robustness. On the other hand, maneuvers can only be recognized during their execution, thus limiting the possible prediction horizon. The applied features for maneuver detection of a VOI are lateral offset within its lane, lateral velocity and proximity of its motion path to prototype lane change trajectories in the database. The proximity of a partially observed trajectory, \(\mathbf{d}_{p}\), and a cluster prototype, \(\mathbf{c}_{m}\), can be calculated via average Mahalanobis \(L_{1}\) distance:
$$\qquad T_{m}=\min(T_{\mathrm{buffer}},\tau_{m})$$

The evaluation period, \(T_{m}\), is the minimum of buffered partial trajectory and prototype trajectory length. The cluster prototype is shifted by \(\tau_{m}\) optimizing the trajectories’ adjustment by minimizing their dissimilarity within the recent \(0.5\,\mathrm{s}\) [10]. Compared to the Euclidean distance, utilized during clustering, the Mahalanobis distance measure takes the non-uniform uncertainty of the prototype course into account. Dissimilarities in regions of low variance have increased weight due to the term \(\frac{1}{\mathbf{\sigma}_{m}^{2}}\). Mahalanobis distance of position trajectories is denoted as \(\mathbf{\Delta}_{p}\). The notation in case of velocity trajectories is \(\mathbf{\Delta}_{v}\).

During feature generation a vehicle’s partial trajectory is matched individually to all prototype trajectories in the database. For each pair \(\{\mathbf{d}_{p},\mathbf{c}_{m}\}\) the optimal shifting coefficient, \(\tau_{m}\), is calculated minimizing \(\mathbf{\Delta}_{p,[-0.5:0]}(\mathbf{d}_{p},\mathbf{c}_{m})\). The velocity-dependent distance \(\mathbf{\Delta}_{v}(\mathbf{d}_{p},\mathbf{c}_{m})\) related to the same cluster is calculated for an equal value of \(\tau_{m}\). As the best matching prototypes for LCL and LCR we select those minimizing the dissimilarity \(\mathbf{\Delta}_{p}\) for the whole evaluation period. Their dissimilarity values are denoted \(\mathbf{\Delta}_{p,\mathrm{LCL}}\), \(\mathbf{\Delta}_{v,\mathrm{LCL}}\), \(\mathbf{\Delta}_{p,\mathrm{LCR}}\) and \(\mathbf{\Delta}_{v,\mathrm{LCR}}\).

In [10] a Quadratic Gaussian Discriminant Analysis (GDA) classifier is applied for maneuver inference. In this contribution its performance is compared to Boosted Decision Trees (BDT), an ensemble of shallow decision trees sequentially trained by a boosting method, described in Sect. 4.2.

4.1 Gaussian Discriminant Analysis

Discriminant Analysis is a generative classifier that derives a representation for each class \(C_{i},\,i\in\mathrm{\{LCL,\,LK,\,LCR\}}\) from the training data by calculating the empirical mean, \(\mathbf{\mu}_{i}\in\mathbb{R}^{n}\), and covariance matrix, \(\mathbf{\Sigma}_{i}\in\mathbb{R}^{n\times n}\) [12]. Classification of new data is based on the Bayesian discriminant rule
where \(\mathbf{f}\in\mathbb{R}^{n}\) denotes the n‑dimensional feature vector and \(p_{i}\) is the prior probability of class \(C_{i}\). GDA can be regarded as a “nearest centroids classifier” [13], as the conditional probability of observed features given a class representation is computed by measuring its Mahalanobis distance to the class center.
As introduced in [10], we chose a four-dimensional feature vector \(f\). The first two features are the lateral position \(d\) and velocity \(\dot{d}\) of a vehicle, respectively. For feature three and four we apply:
generating a memory for the classifier by taking into account buffered vehicle states and prior knowledge about typical lane change patterns in the form of cluster prototypes. All the features are assumed to be independent and normally distributed within each class.

The attractiveness of GDA lies in its simplicity. The classification rule is both inexpensive to compute and easily interpretable. Multi-class classification is inherently supported and no hyperparameters need to be tuned in the training phase. Quadratic GDA models were fitted to trained data utilizing the MATLAB library LIBRA. The library provides a training routine which is robust against outliers in the data [14].

4.2 Boosting

Boosting is a sequential method for consecutively fitting a set of basic learners to variations of the training data. After each iteration, the whole training set is evaluated [15]. Misclassified data points are given greater weights when used to train the next classifier. Basic learners, also referred to as “weak learners” [13], typically have low classification performance when applied individually. In an ensemble of \(N\) basic learners, each unit provides a class vote dependent on observed features. Those votes are weighted by \(\alpha_{n}\) and combined like:
with \(h_{n}\) denoting the hypothesis of the \(n\)-th basic learner. We apply the Adaboost.M2 algorithm which is applicable in classification problems with multiple class instances [16]. In AdaBoost.M2 weighting coefficients \(\alpha_{n}\) are derived from weighted pseudo-loss functions, providing more accurate basic classifiers with a greater weight.

As basic classifiers we use shallow decision trees limiting the maximum number of branch nodes to 15. The ensemble classifier comprises 90 basic decision trees. Treated individually, decisions are easily interpretable. They are not limited to a specific type of data. The flexibility of decision trees is extended by enlargement to ensemble methods at the expense of easy interpretability. AdaBoost application does not require hyperparameter training and showed best performance in an extensive empirical comparison [17].

5 Maneuver-based Trajectory Prediction

After successful driver intention recognition, the future trajectory of a vehicle is predicted according to the estimated maneuver. In [10] the potential of prototype trajectories for both longitudinal and lateral motion prediction was demonstrated. In case of detected lane changes, the best matching prototype of the respective maneuver type, LCL or LCR, was selected as an estimate for future lateral motion. Here, we extend this idea by formulating a Gaussian mixture model combining and weighting the predictions of all prototype trajectories of the estimated maneuver type. The predicted lateral course can be calculated as a weighted sum of the prototype means:
and the combined variance can be computed by:
The final values of mean and variance are applied when individual prototypes are to be evaluated outside their definition range.
$$\hat{\mu}_{m}(t)=\begin{cases}\mu_{m}(\tau_{m}+t),&\tau_{m}+t<T_{m}\\ \mu_{m}(T_{m}),&\text{otherwise}\end{cases}$$
$$\hat{\sigma}^{2}_{m}(t)=\begin{cases}\sigma^{2}_{m}(\tau_{m}+t),&\tau_{m}+t<T_{m}\\ \sigma^{2}_{m}(T_{m}),&\text{otherwise}\end{cases}.$$
The weighting factors, \(w_{m}\), are derived from the ratio of inversed Mahalanobis distance of each prototype representing a motion pattern of the detected maneuver to the partial trajectory:
Figure 4 depicts the mean and variance values of the prototype ensemble for different prediction horizons.
Fig. 4

Predicted trajectories (colored red) at different time points before a lane change left. The vehicle’s true motion is plotted in green for past states and in blue for future observations. The lane change event is indicated by a vertical red line. Bar plots depict the individual weights of each prototype in the ensemble model. a Predicted trajectory 1.66 s before the lane change. b Predicted trajectory 1.18 s before the lane change. c Predicted trajectory 0.61 s before the lane change. d Predicted trajectory 0.32 s before the lane change

The longitudinal motion during a lane change maneuver can be described by a (nearly) constant acceleration model [10], also referred to as discrete Wiener process acceleration model [18]. It is given by:
$$\begin{bmatrix}s_{k+1}\\ v_{s_{k+1}}\\ a_{s_{k+1}}\end{bmatrix}=\begin{bmatrix}1&T&\frac{1}{2}T^{2}\\ 0&1&T\\ 0&0&1\end{bmatrix}\begin{bmatrix}s_{k}\\ v_{s_{k}}\\ \overline{a}\end{bmatrix}+\begin{bmatrix}\frac{1}{2}T^{2}\\ T\\ 1\end{bmatrix}\omega_{a}$$
In [10] the constant acceleration value is calculated averaging the longitudinal accelerations of all trajectories comprised in the cluster, which is the best fit to the vehicles’ partial trajectory. \(\sigma_{\overline{a}}^{2}\) is the related variance. Using the Gaussian mixture model, the new values of \(\overline{a}\) and \(\sigma_{\overline{a}}^{2}\) can be computed via Eqs. (12) and (13), respectively.

5.1 Smooth Transition to Prototype-based Trajectory Predictions

The future lateral motion course, predicted based on the previously proposed model of mixed cluster representatives, does not necessarily pass through the currently observed state of the vehicle. Therefore, the transition from current lateral offset to future values is generally discontinuous. To solve this problem, we adapt all cluster prototypes that contribute to the predicted trajectory within the interval \([\tau_{m},\tau_{m}+1\,\mathrm{s}]\). This ensures continuity in lateral position, velocity, and acceleration also in the near future. The adaptations are based on transformation of the lateral position course into B‑splines. B‑splines and their basis functions have favorable characteristics allowing adaptations in limited scope (local support characteristic) and guaranteeing continuity of a function and their derivatives of certain degree (continuity characteristic) [19].

B-splines are piecewise polynomial curves of degree \(d\) defined on an interval \([a,b]\). They are represented by a knot vector \(\mathbf{t}=[t_{0},t_{1},{\ldots},t_{m}]\) with \(m+1\) elements and a set of \(n+1\) control points \(\mathbf{P}=\{\mathbf{P}_{0},{\ldots},\mathbf{P}_{n}\}\). The relation \(n+1=m+d\) is always valid. B‑Spline curves are represented by
where the basis functions \(N_{i,d}(t)\) are recursively defined as
$$N_{i,0}(t)=\begin{cases}1,&t\in[t_{i},t_{i+1})\\ 0,&\text{otherwise}\end{cases}$$
Initially, a cluster’s polynomial representation \(f(t)=\textbf{a}_{0}+\textbf{a}_{1}t+{\ldots}+\textbf{a}_{n}t^{n}\) is converted into a B-spline curve by scaling the polynomial to the interval \([0,1]\) and calculating the control points [20] by
Due to the local support characteristic, the control point \(\mathbf{P}_{i}\) only affects the interval \([t_{i},t_{i+p+1})\). We are using this property to modify the trajectory for the first second of prediction by adding three knots at the interval boundaries, respectively. \(\mathbf{C}(t)\) is \(d-k=2\) times continuously differentiable at these breakpoints, with multiplicity \(k=3\) corresponding to the number of identical knots in the knot vector. The insertion of a new knot \(\hat{t}\in[t_{s},t_{s+1})\) causes a new control point. As stated in [20], the elements of the control point set are updated by
$$\hat{\textbf{P}}_{i}=\begin{cases}\textbf{P}_{i},&0\leq i\leq s-d\\ (1-\alpha_{i})\textbf{P}_{i-1}+\alpha_{i}\textbf{P}_{i},&s-d+1\leq i\leq s\\ \textbf{P}_{i-1},&s+1\leq i\leq n+1\end{cases}$$
Within the first prediction second only a limited number of basis functions are unequal to zero. Due to knot insertion the non-zero basis functions are \(N_{4,5}\), \(N_{5,5}\) and \(N_{6,5}\). The control points of these basis functions need to be adjusted such that the B‑spline meets the currently observed vehicle states which are position, velocity and acceleration. According to [20], the r‑th derivative of a B-spline is defined as
with \(\textbf{P}^{0}_{i}=\textbf{P}_{i}\). The knot vector is reduced to \(t_{r},{\ldots},t_{m-r}\). This yields the following equation system with three undefined variables \(\textbf{P}_{4}\), \(\textbf{P}_{5}\), and \(\textbf{P}_{6}\):
The derivatives can be calculated via Eq. (26). The vehicle’s lateral velocity and acceleration are estimated via derivation of a third-order polynomial approximation of the partial trajectory and inserted into Eqs. (27), (28), and (29) to solve for the values of the required control points.

6 Results

For evaluation and comparison of proposed approaches for maneuver estimation and subsequent trajectory prediction we use the data set proposed in [10]. Aggregating hours of real highway footage, the whole data set comprises a total number of 434 lane change maneuvers of traffic participants of which 156 are lane changes to the left and 278 to the right neighbor lane, respectively. Data is divided with a ratio of \(70/30\) into training and test set. During data division it is specially requested that ratio of maneuver types is equal in both fractions of the data set. In terms of maneuver classification we compute performance measures juxtaposing GDA and boosting decision trees. As the distribution of classes is highly unequal, with lane keeping clearly outnumbering lane change maneuvers, balanced versions of performance measures are applied. The advances in trajectory prediction, namely combining prototype trajectories in a Gaussian mixture model and B‑spline based trajectory smoothing, are compared to the previously implemented approach [10], which utilized solely the best matching cluster representative for future motion prediction.

6.1 Evaluation of Maneuver Detection

Due to the highly unequal class incidences, balanced measures [7] must be applied for classification performance evaluation, whenever a quality criterion combines actual positive and actual negative portions of the regarded class (e.g. precision):
The True Positive Rate \(\mathrm{TPR}\), also denoted recall, states how likely a maneuver will be predicted by the respective approach. The precision \(\mathrm{PRC}\) indicates how likely a predicted lane change will actually happen. The \(\mathrm{F}_{1}\) score is the harmonic mean of both precision and recall. Those values are computed on the whole test data set. Table 1 shows the classification performance for the proposed methods, Quadratic Gaussian Discriminant Analysis and Boosted Decision Trees, using two different sets of features, respectively. As a reference, to investigate the influence of features constructed from cluster prototypes, both methods are evaluated utilizing a two-dimensional feature vector \(\mathbf{f}=[d,\dot{d}]^{T}\) comprising lateral offset, \(d\), and lateral velocity, \(\dot{d}\), of a vehicle in Frenet coordinates. In case of GDA we constructed two features utilizing distance values to the best matching prototypes by:
Thereby, the dimension of the feature vector can be kept low while achieving good prediction results. For BDT classifier we expand the feature vector to six dimensions \(\mathbf{f}=[d,\dot{d},\mathbf{\Delta}_{p,\mathrm{LCR}},\mathbf{\Delta}_{p,\mathrm{LCL}},\mathbf{\Delta}_{v,\mathrm{LCR}},\mathbf{\Delta}_{v,\mathrm{LCL}}]^{T}\), allowing the ensemble model to learn mutual dependencies of features. Application of this feature vector for GDA worsens prediction quality, most likely because model assumptions are violated. For the sake of comparison all classifiers were trained allowing a miss rate for the lane keeping maneuver of \(MCR_{LK}=0.11\). This is realized via adaptation of prior probabilities for GDA and initial weights of the data points for BDT.
Table 1

Maneuver classification performance of Quadratic Gaussian Discriminant Analysis (GDA) and Boosted Decision Trees (BDT) for different sets of features




Miss Rate




Avg. Prediction Time


\(d\), \(\dot{d}\)


















\(d\), \(\dot{d}\),




















\(d\), \(\dot{d}\)


















\(d\), \(\dot{d}\),







\(\mathbf{\Delta}_{p,\mathrm{LCR}}\), \(\mathbf{\Delta}_{p,\mathrm{LCL}}\),






\(\mathbf{\Delta}_{v,\mathrm{LCR}}\), \(\mathbf{\Delta}_{v,\mathrm{LCL}}\)






6.2 Discussion of Maneuver Detection Performance

First, the influence of features constructed from cluster prototypes on the classification performance is investigated for both learning methods. GDA strongly benefits from additional information provided by prototype trajectories. The average prediction time can be increased by \(0.14\,\mathrm{s}\) to a total of \(1.67\,\mathrm{s}\). The performance measures show improvements, as well. The increase in average prediction time of \(0.06\,\mathrm{s}\) for the BDT method by adding prototype-derived features is comparatively small, but again, the prediction quality measures denote performance improvements.

For the two-dimensional feature vector \(\mathbf{f}=[d,\dot{d}]^{T}\) BDT performs significantly better reaching an improved prediction time and higher \(\mathrm{F}_{1}\)-scores. Whereas the precision values are on equal level, the major improvements are reported by increased recall values for lane change maneuvers, which means that actual lane changes are more likely to be detected. Incorporating information provided by motion pattern recognition, both GDA and BDT achieve equal average prediction horizons of roughly \(1.7\,\mathrm{s}\), while, again, BDT method shows better values for performance measures. The optimal average prediction horizon according to the automatically labeled data can be calculated to \(2\,\mathrm{s}\).

6.3 Evaluation of Motion Prediction

An advantage of prototype-based maneuver detection lies in the usability of cluster prototypes for the future motion prediction of traffic participants. Here, the extended approach for trajectory prediction, as described in Sect. 5, is evaluated and compared to the original version presented in [10].

Figure 5 shows the mean absolute lateral and longitudinal errors between traffic participants’ predicted and actual positions for different prediction horizons. The evaluation results for the reference approach is illustrated on left, while the new motion prediction results are depicted on the right side of the figure. The prediction horizon \([0\,\mathrm{s},4\,\mathrm{s}]\), which is plotted on the horizontal axis, is subdivided into intervals of \(\Delta t=0.5\,\mathrm{s}\) . For the \(n\)-th prediction interval, the mean absolute position error is calculated by
$$\epsilon_{n}=\frac{1}{N_{n}}\cdot\sum_{\tau\in[n\Delta T,(n+1)\Delta T)}|\mathbf{x}_{t}-\hat{\mathbf{x}}_{(t|t-\tau)}|$$
where \(N_{n}\) is the total number of summands, \(\mathbf{x}_{t}=[s_{t},d_{t}]\) denotes the actual observation at time \(t\) and \(\hat{\mathbf{x}}_{(t|t-\tau)}\) is the prediction of vehicle state at time \(t\) calculated at time \(t-\tau\). The maximum evaluation time is limited by the length of the prototype ensemble used for motion prediction and capped to a value of \(4\,\mathrm{s}\). The evaluation starts when the correct lane change maneuver is detected continuously and ends at the point in time when the vehicle crosses the lane marking.
Fig. 5

Mean absolute errors between the traffic participants’ predicted and actual positions for various prediction horizons. a Lateral errors of the reference approach. b Lateral errors of the proposed approach. c Longitudinal errors of the reference approach. d Longitudinal errors of the proposed approach

6.4 Discussion of Motion Prediction Performance

For longitudinal trajectory prediction during lane change maneuvers both approaches perform equally well. Comparing both lane change types, the errors are generally larger in case of LCL maneuvers. The reason for this lies in the larger diversity of longitudinal accelerations during lane changes to the left.

The performance in lateral motion prediction is significantly improved by the extensions presented in Sect. 5. For small prediction horizons, the absolute position errors correspond approximately to the measurement accuracy. For a prediction of horizon of \(2-4\,\mathrm{s}\), the absolute lateral error is steady with an approximate value of \(0.3\,\mathrm{m}\). In case of the reference approach, error peaks for medium prediction horizons can be observed. These peaks result from incorrect selections of prototype trajectories in an early stage of the detected maneuver. At the beginning many cluster prototypes are similar so that wrong prototype selection is quite likely. The combination of prototypes in a Gaussian mixture model can compensate this issue.

Figure 6 plots the average Mahalanobis distance between traffic participants’ predicted and actual lateral offsets. The upper bound for all considered prediction horizons is \(0.7\,\sigma\), which indicates that the variance of the prototype ensemble, which is calculated utilizing Eq. (13), is a good estimate of prediction uncertainty.
Fig. 6

Mean lateral Mahalanobis distance between the traffic participants’ predicted and actual positions for various prediction horizons

7 Conclusion and Outlook

In this paper, a statistical approach is proposed, which successfully utilizes a set of prototypical lane change trajectories to realize both early maneuver detection and uncertainty-aware trajectory prediction for traffic participants. Generation of prototype trajectories from real traffic data is accomplished by Agglomerative Hierarchical Clustering, optimizing mutual cluster configurations and constraining cohesion during the clustering process [10]. The main contributions of this paper exist in targeted adaptations of prediction methods to improve utilization of motion patterns and thus enhancing prediction results. For maneuver recognition, we implemented Boosted Decision Trees classification increasing the detection rate of lane change maneuvers compared to previously applied Gaussian Discriminant Analysis. The future trajectory is predicted according to typical realizations of the estimated maneuver. We introduced a mixture model of cluster prototypes and demonstrated increased accuracy for lateral motion prediction. Additionally, a B-splines based adaptation technique is described guaranteeing continuity during transition from actually observed to predicted vehicle states. In future work, generalizability of lane change motion patterns is investigated via cross validation. Additionally, the prediction framework is extended by taking interaction between traffic participants into account to further increase the prediction horizon and lower misclassification rates.


  1. 1.
    On-Road Automated Driving (ORAD) committee (2014) Taxonomy and Definitions for Terms Related to On-Road Motor Vehicle Automated Driving Systems.
  2. 2.
    Lefèvre S, Vasquez D, Laugier C (2014) A survey on motion prediction and risk assessment for intelligent vehicles. Robomech J 1(1):1CrossRefGoogle Scholar
  3. 3.
    Liebner M, Klanner F (2015) Fahrerabsichtserkennung und Risikobewertung. In: Winner H, Hakuli S, Lotz F, Singer C (eds) Handbuch Fahrerassistenzsysteme. Springer, Wiesbaden, pp 701–719CrossRefGoogle Scholar
  4. 4.
    Schreier M, Willert V, Adamy J (2016) An integrated approach to maneuver-based trajectory prediction and criticality assessment in arbitrary road environments. IEEE Trans Intell Transp Syst 17(10):2751–2766CrossRefGoogle Scholar
  5. 5.
    Wyder T, Schildbach G, Lefevre S, Borrelli F (2015) A bayesian filter for modeling traffic at stop intersections. In: Intelligent Vehicles Symposium IEEE. vol 5, pp 1252–1257Google Scholar
  6. 6.
    Houenou A, Bonnifait P, Cherfaoui V, Yao W (2013) Vehicle trajectory prediction based on motion model and maneuver recognition. In: Intelligent Robots and Systems (IROS) 2013 IEEE/RSJ International Conference, pp 4363–4369Google Scholar
  7. 7.
    Bahram M, Hubmann C, Lawitzky A, Aeberhard M, Wollherr D (2016) A combined model-and learning-based framework for interaction-aware maneuver prediction. IEEE Trans Intell Transp Syst 17(6):1538–1550CrossRefGoogle Scholar
  8. 8.
    Bahram M (2017) Interactive maneuver prediction and planning for highly automated driving functions. Technische Universität München, München (Dissertation)Google Scholar
  9. 9.
    Vasquez D, Fraichard T (2004) Motion prediction for moving objects: a statistical approach. In: Robotics and Automation, 2004. Proceedings ICRA’04. 2004 IEEE International Conference. vol 4. IEEE, New Orleans, pp 3931–3936Google Scholar
  10. 10.
    Augustin D, Hofmann M, Konigorski U (2018) Motion pattern recognition for maneuver detection and trajectory prediction on highways. 2018 IEEE International Conference on Vehicular Electronics and Safety (ICVES), Madrid, 2018, pp. 1–8.
  11. 11.
    Tan P‑N, Steinbach M, Kumar V (2005) Introduction to data mining. Pearson, BostonGoogle Scholar
  12. 12.
    Hubert M, Van Driessen K (2004) Fast and robust discriminant analysis. Comput Stat Data Anal 45(2):301–320MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Murphy KP (2012) Machine learning: a probabilistic perspective. Adaptive computation and machine learningzbMATHGoogle Scholar
  14. 14.
    Verboven S, Hubert M (2010) Matlab library libra, Wiley interdisciplinary reviews. Comput Stat 2(4):509–515CrossRefGoogle Scholar
  15. 15.
    Bishop CM (2006) Pattern recognition and machine learning (information science and statistics). Springer, Berlin, HeidelbergzbMATHGoogle Scholar
  16. 16.
    Freund Y, Schapire RE (1997) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55(1):119–139MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Caruana R, Niculescu-Mizil A (2006) An empirical comparison of supervised learning algorithms. In: Proceedings of the 23rd international conference on Machine learning ACM, pp 161–168Google Scholar
  18. 18.
    Bar-Shalom Y, Li XR, Kirubarajan T (2004) Estimation with applications to tracking and navigation: theory algorithms and software. John Wiley & Sons, New YorkGoogle Scholar
  19. 19.
    Piegl L, Tiller W (2012) The NURBS book. Springer, Business MediazbMATHGoogle Scholar
  20. 20.
    Marsh D (2006) Applied geometry for computer graphics and CAD. Springer, Berlin HeidelbergzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Deutschland, ein Teil von Springer Nature 2019

Authors and Affiliations

  1. 1.Opel Automobile GmbHRüsselsheimGermany
  2. 2.Institute of Automatic Control and MechatronicsTechnische Universität DarmstadtDarmstadtGermany

Personalised recommendations