1 Introduction

In recent years, driver behavior has become a popular subject in road safety analysis. This popularity has resulted in research on many different innovative techniques and measures used to quantify and analyze driver behavior [1–3]. Most past efforts have focused on driver behavior under normal road weather conditions; however, few of them have considered adverse weather conditions which potentially lead to significant driver behavior adaptations.

Adverse weather conditions during winter, such as snowstorms and precipitation, have an important effect on traffic conditions and road safety. Many studies have looked at road safety issues during winter, and one can refer to [4–7] for a review of the literature dedicated to such issues. An important link has been found between crash occurrence and adverse weather. Rainy or snowy conditions increase the chance of collision. However, less severe accidents are expected to happen during adverse weather since drivers adapt their behavior [8]. More specifically, drivers faced with such conditions in general make less lane-changing maneuvers, reduce their speed, and increase their time and space gaps with other vehicles. All driver behavior adaptations are translated into less severe interactions (at lower speeds) and less severe conflicts. In other words, when accidents occur, less energy is dissipated.

A greater understanding of traffic and driver behavior in winter can improve the provision of winter maintenance services and traffic management strategies, e.g., applying variable speed limits and conducting a proactive safety analysis. To analyze driver behavior, two alternative approaches can be found in the literature to collect driving data, namely manual observation and vehicle-based tracking. In both approaches, the main idea is to collect microscopic data, e.g., speeds during short intervals or speeds at the vehicle level.

In order to get speed information, historical data from loop detectors are commonly utilized. The availability of microscopic data to investigate driver behavior is less common given the difficulties and need for more advance sensors and software for automated data collection. Several traffic data collection technologies, including the video image processor, laser sensor, ultrasonic sensor, and microwave radar sensor, experience issues during adverse weather conditions. Automated methods for generating trajectory data and deriving microscopic measures based on video are not new, but their use in winter conditions is less popular.

In order to fully understand driver behavior during adverse weather conditions, microscopic data are necessary. This data can provide more insight into the behavioral mechanics of individual drivers when interacting with the road surface, traffic controls, other vehicles, and weather factors. Accordingly, the goal of this research was to explore the feasibility of obtaining microscopic data from video sensors during winter. More precisely, this study investigates driver behavior under snow–weather conditions by adopting a trajectory-based approach with the use of video data. This study also explores the use of manual and automated video processing approaches. For automated video processing, a software called as Traffic Intelligence is used on road conditions with and without the presence of snow.

2 Literature review

Several studies have shown that snow storms greatly affect highway traffic [9–14]. Other studies have investigated the effects of various weather conditions on road safety [15–17, 19]. Research on the topics of weather conditions and vehicular traffic has generally concluded that adverse weather has a significant impact on traffic conditions (speed and volumes) and safety (collision incidence). For instance, Datla and Sharma observed a significant reduction in traffic volume due to snow events, for all types of highways [18]. Also focusing on highway traffic, Fu et al. [5] used a linear regression model to quantify the effect of snow storms on operating speed. Using an hourly snow intensity parameter, capacity and free-flow-speed (FFS) models taking into account weather and road surface conditions have been built [20].

Examining and quantifying the behavior of individual drivers are a crucial component of a thorough and comprehensive driver behavior analysis. Ahmed enhanced the existing vehicle-following and lane change models based on empirical work [21]. In his paper, microscopic lane change behavior data were used to model the lane-changing process in three steps: decision-making, choice of target lane, and gap acceptance [21]. When confronted with adverse weather conditions, road users drive more cautiously to avoid incidents. This suggests that drivers modify their behavior in such conditions. A few studies have shown that weather conditions affect driver behavior significantly [22–28]. Kilpeläinen and Summala [27] concluded that driver behavior is predominantly affected by the prevailing observable conditions. Chakrabartya and Guptab [28] studied driver behavior and crash characteristics during adverse weather conditions including rainy weather and foggy weather. The results showed that the percentage of speeding drivers reduced in both rainy weather and foggy weather.

Several different methods have been used to collect microscopic traffic data. Detailed information on most of these methods, including video image processing, as well as the use of laser, ultrasonic, and microwave radar sensors, can be found in the Traffic Detector Handbook [1]. Innovative methods not described in the book include the video-based truck detection and classification method developed by Zhang et al. [2], as well as the aerial photographic techniques used by Bham and Benekohal to analyze acceleration behavior [3].

Surrogate safety measures and video analysis have also been used in contexts, different from those where the classical road safety approach is known to give satisfactory results [29–33]. In [34], the authors presented a method which relied on such surrogate measures to analyze driver behavior on highway ramps for which crash data were not very reliable. In their study, the authors used semiautomatic trajectory collection and behavior analysis to measure time to collision (TTC).

Unfortunately, video analytics technology is still plagued by some issues and limitations [35]. This innovative approach faces challenges resulting from the complexity of the computer vision algorithms at its very core, as well as from limited field of view, low software visibility sensitivity, and congestion-related vehicle tracking problems. The effectiveness and reliability of an automated approach depends heavily on flow conditions and video quality. Weather conditions, obstacles, camera’s angle and field of view, curved roadway sections, and occlusions from dense traffic and large vehicles are all important factors that could lead to tracking errors [35]. Despite its limitations, an automated video processing method allows one to have access to a much broader range of information than the results provided by the traditional manual data processing approach. Video data can always be processed manually to check and correct suspected or confirmed errors.

3 Methodology

The methodology followed for this research includes the following four steps:

  1. (1)

    Identification of sites and winter events: Several sites were selected, and winter storms were identified in advance. An important part of the video data with snow was poor in quality and could not be processed by the automated approach. A manual method was then used. Different issues with the videos were also documented.

  2. (2)

    Processing of video data: Analysis of the driver behavior with manual approach was carried out for video with poor visibility.

  3. (3)

    Adjustment of the automated approach: Data from video with good quality were processed automatically and compared to manual video processing. By comparing the results generated from both approaches, an adjustment was made to get the automated speed data as close as possible to the manually obtained data.

  4. (4)

    Data analysis: Different surrogate measures were used including lane change ratio, TTC, and average speed.

3.1 Microscopic measures

Driver behavior and surrogate measures were defined for this study as follows:

3.1.1 Lane change ratio

Several previous studies have used the number of lane changes to analyze driver behavior [36, 37]. However, the number of lane changes cannot be directly used to evaluate the effect of a snowstorm on driver behavior because they may vary greatly based on the total traffic volume. Therefore, we used a variable indicating the percentage of lane changes among traffic known as the lane change ratio. The present study defined the lane change ratio as the number of lane change maneuvers divided by the total flow in a specified period of time, that is, R L = Q L /Q, where R L is the lane change ratio, Q L is the total observed number of lane change maneuvers, and Q is the total vehicular flow in the same travel direction. Note that the total flow refers to the number of vehicles entering the study area during the specified time period, and that the total number of lane-changing maneuvers was restricted to those maneuvers taking place inside the study area.

3.1.2 Vehicle-level operating speeds

Operating speed simply corresponds to the time that a vehicle takes to travel a short distance L, which is calculated using the straightforward equation: \(S_{i} = L/T_{i}\), where S i is the travel speed for vehicle i as it crosses the study segment, L is the length of the study segment (from the starting line to the ending line), and T i is the time that a vehicle takes to cross the study segment. The average operating speed is then computed as the sum of all the measured speed values divided by the total number of vehicles. In this study, the average operating speed was used to validate the two approaches, as well as to analyze the effect of adverse winter weather conditions (snow) on highway driver behavior. Note that Traffic Intelligence (TIS) registers the spot speed, which is the speed of a moving object when its presence is detected. In the automated approach, the results for collision points and speed heat-maps were all generated based on the temporal speeds. Average temporal speeds are discussed in the result section as part of the result summary for the automated approach.

3.1.3 TTC as a surrogate measure

TTC can be defined as “the time required for two vehicles to collide if they continue at their present speeds and on the same path” [38, 39]. Following the approach adopted in previous research conducted by St-Aubin et al. [35], this study uses TTC as a surrogate safety measure. As shown in Fig. 1, TTC measures are classified according to two types of conflict: rear-end (type A) and lateral-diagonal (type C). The measures are computed using trajectories obtained from a computer vision tracking algorithm and represented on heat-maps.

Fig. 1
figure 1

Classification of TTC measures [35]. a Type A rear-end converging. b Type C diagonal converging

Full size image

In our study, TTC was used as a surrogate measure of driver behavior adaptations. Drivers make speed adaptations to avoid potential conflicts with other vehicles. These potential conflicts occur when a vehicle gets too close to the vehicle in front of it or merges into the other lane with vehicles already in it. Therefore, TTC was used to capture the safety effect of both speed adaptations and lane change behaviors.

3.2 Video processing

Video processing was carried out using both a manual and automated process. The manual process manually determines vehicle speeds and the lane change ratio. The driver behavior measures utilized with the manual approach include travel speeds and number of lane changes. By setting a start and end line on the video records, study segments were defined. The lengths of the segments were measured using Google Earth. By recording the time it took for randomly selected vehicles to pass the segments, the speeds of these vehicles were calculated based on the segment lengths. In addition, the number of vehicles and the total number of vehicles changing lanes were counted for each lane.

Regarding the automated process, this was done using computer vision algorithms implemented in the tracker. The techniques used in this software are explained by Saunier [29] and Shi and Tomasi [40].

The tracker algorithm used in this paper can be summarized in two steps:

  1. (1)

    Individual pixels are detected and tracked from frame to frame and recorded as feature trajectories using the Kanade–Lucas–Tomasi feature tracking algorithm [40].

  2. (2)

    The feature trajectories are then grouped based on consistent common motion in order to define a single moving object.

The parameters of this algorithm were calibrated through trial and error, leading to a trade-off between over-segmentation (one object being tracked as many objects) and over-grouping (many objects being tracked as one object). After the second step, the object data set was saved as a Sqlite file which included coordinates of each object and its corresponding features in time. Traffic information, such as average travel speed, and behavior information were calculated.

3.3 TTC estimation

TTC was computed under the assumption that velocity was constant. A threshold value of 5 s was then used to classify TTC values that lead to potential conflicts. Potential conflict points were defined as those conflicts with a TTC less than 5 s. Theoretically, areas with a higher density of potential conflict points experience more behavioral adaptations and have a higher possibility of an accident occurring.

A procedure framework to generate TTC results was created based on the automated approach. The framework is described below, which gives an example of a practical application of using video data with an automated analysis approach to study driver behavior in winter road conditions. The step for calibrating the parameters is quite essential in the framework. Due to the error caused by matching the pixels from the camera view to the points in the real world and the potential fish-eye effect caused by the camera, the trajectory data, especially the speed generated by the automated approach, may have a 10 % error range. This difficulty can be controlled by adjusting parameters like the number of frames per second. Since TTC is generated from the speed data, this measure has the same source of error as speed.

  • Step 1 Data preparation

    • Video data collection

    • Highway resolution aerial map

  • Step 2 Data preprocessing

    • Object trajectory tracking

    • Test analysis with a sample video

  • Step 3 Parameter calibration

    • Travel speed comparison from two approaches

    • Parameter adjustment to calibrate the automated approach

  • Step 4 Data post-processing

    • Generating behavior related results, TTC and speed using the tracker

    • Analysis of results

  • Step 5 TTC mapping

    • Mapping the location of high risk conflicts according to TTC (5 s)

    • Defining the areas requiring priority treatments

4 Data and results

This research makes use of the open source software TIS to obtain trajectories and identify conflicts. Two case studies were conducted. The first case study focused on identifying the main issues when trying to extract accurate information from video data collected with surveillance traffic cameras during adverse weather conditions, as well as some potential solutions. The video data were also manually processed to evaluate the impact of snowy weather conditions on the behavior of drivers at highway access ramps, as well as to compare the results with the results obtained with TIS. The second case study was conducted to analyze some higher quality videos using the automated approach. An aerial TTC map showing the location of cumulated potential conflicts was also generated as an example of using the automated analytical approach for the practical winter road maintenance.

4.1 Issues with automated approach: case study in Toronto

As previously mentioned, video data were recorded using permanent CCTV cameras installed at three sites in the Toronto area. To control the impact of other factors that may affect the analysis results apart from weather conditions, data were collected over four time periods on weekdays: 05:30–06:30 in the morning off-peak, 08:00–09:00 in the morning peak, 13:00–14:00 in the afternoon off-peak, and 16:00–17:00 in the afternoon peak. A brief description of the three selected sites is provided in Table 1 including site location, camera angle, and video ID. Table 2 describes all 15 videos recorded from these locations.

Table 1 Description of sites
Full size table
Table 2 Description of video recordings
Full size table

The weather condition and surface were used to classify each video; this method helped to classify whether surfaces were clear or covered with snow. Traffic conditions were also determined based on volumes and operating speeds. In general, videos taken during rush hours (07:00–10:00 and 14:00–19:00) were classified as congested traffic conditions.

A preliminary analysis of video quality was carried out to identify the most common data quality issues for each site involved in the study. Some of the common weather-related issues associated with most cameras were (i) intense and direct sunlight, (ii) presence of frost or rain drops on the camera lens, and (iii) the reflection of light from the wet road surface. These issues usually led to poor video quality, which made automated data processing more difficult. In fact, some weather conditions resulted in overly complicated and unclear video recordings, from which algorithms were unable to recognize distinct features needed to hypothesis objects. Among the four videos collected during snowy weather, only two videos represent traffic in free-flow conditions. Table 3 lists the issues encountered when the tracker tried to process the collected video data. Potential solutions to some of these issues are discussed later in the document.

Table 3 Typical camera issues in video data
Full size table

In addition to weather-related problems, congestion was another common issue with video data processing. Since the tracker used in this study only detects moving objects, automated video data processing could not extract trajectories under congested conditions. During traffic jams, the velocity of some vehicles was near zero. In such conditions, the tracker was incapable of tracking the majority of vehicles, which rendered the video recording useless. In addition to failing to properly track slow-moving vehicles, the algorithm could not track the vehicles under most night-time conditions, since darkness caused many objects in the video to appear indistinguishable. One potential solution to this issue could be to use a thermal imaging camera.

4.1.1 Behavior analysis with manual approach

The results of the average speeds across the three observed weather conditions are presented in Fig. 2. The three weather conditions were good weather without any form of precipitation, good weather with a wet road surface (covered with snow or water), and poor weather with snow precipitating. For comparison purposes, video data from same data and the same data collection period under different weather conditions were selected. Therefore, videos 3 (good weather without any form of precipitation) and 4 (poor weather with snow precipitation) and videos 7 (good weather with wet surface), 9 (poor weather with snow precipitation), and 10 (good weather without any form of precipitation) were compared to each other, respectively. As initially hypothesized, significant reductions were observed in speeds and lane change ratios under snowy conditions. Average speeds of lanes under good weather conditions were higher than those under good weather conditions with wet road surface. Meanwhile, average speeds under snowy conditions were the lowest. For instance, on average, a reduction of more than 10 km/h along the site at Site—Camera 71 could be observed. Interestingly, the impact was also observed in each of the lanes; however, the left lane was the lane with the greatest reduction. These results provide clear evidence of drivers adapting to snowstorms and wet road surface conditions.

Fig. 2
figure 2

Comparison of driver behaviors. a Lane change ratio of the two sites under different weather conditions. b Average speeds on different lanes of the two sites under different weather conditions

Full size image

Regarding the lane-changing ratio, the results also show a clear reduction in the lane-changing behavior. As illustrated in Fig. 2, the lane-changing ratio went down from 0.12 (in good weather) to 0.07 (in snowy weather) at Site—Camera 71. This result again confirms the initial hypothesis that drivers seem to make less lane changes when it is snowing or when the road surface is wet or covered with ice, compared to dry road and clear weather conditions.

4.1.2 Automated approach for speed analysis

In the automated approach, the number of frames per second (fps) in video was an essential parameter, based on which speeds and TTCs were calculated. Speeds were calculated based on the measured length divided by the travel time which was calculated by total number of the frames divided by the fps. Due to error caused in matching the video to a map, the measured length used in calculating speeds in video was also subject to error. Therefore, the automated approach needs to be calibrated. Fps is a right parameter to be adjusted to calibrate the automated approach.

In order to represent the calibration procedures, a 3 min sample from video 11 is taken as an example to show how the automated results were matched with the manual results. The post-processing results after the calibration are presented. The first step was to run the speed analysis with the tracker using the original fps value (30 fps in this case). The second step was to get the actual speeds based on manual counts and make a comparison. The results generated manually and automatically are presented in Table 4. From the table, the speeds generated by the tracker on average were 8 % higher than those calculated manually.

Table 4 Comparison of average operating speeds before and after calibration
Full size table

Based on the comparison, the fps can be adjusted to reduce the error. Note that the fps of the video and the total number of frames do not change, and the fps value to be adjusted is the fps value used in calculating speeds. By knowing that the speeds are 8 % higher by the automated approach, the fps has to be decreased by 8 %. In this case, the value of adjusted fps is 27.78.

The speed results from automated approach after calibration are also presented in Table 4. The results show that the corresponding values of average operating speeds in each lane, obtained from the automated and manual methods, were very similar. The two approaches led to approximately equivalent results after calibration.

4.2 Automated data processing: case study in Montreal

A second case study was performed to test the automated approach for analyzing video data in snow conditions. An example procedure of winter maintenance guidance was applied. In consideration of the weather-related issues, high quality videos were recorded from a site in Montreal area. A brief description of the selected site and the videos is given in Table 5. Videos were recorded in snowy weather and good weather, respectively.

Table 5 Description of site and videos
Full size table

Following the same approach, a preprocessing video analysis was conducted, and the fps was adjusted for the automated approach. After calibration, both of the videos were analyzed by the tracking software with the adjusted fps. The results are presented in Figs. 3 and 4.

Fig. 3
figure 3

Speed results for two weather conditions. a Speed distribution—good weather condition. b Speed distribution—snowy weather condition. c Speed heat-map—good weather condition. d Speed heat-map—snowy weather condition

Full size image
Fig. 4
figure 4

Accumulative TTC distributions for both weather conditions

Full size image

From the automated approach, a comparison was made between driver behavior in snow and good weather-related conditions. Considering the speed, the average speed decreased from 84.7 km/h in good conditions to 77.2 km/h in snow conditions, a reduction of 8.8 %. Figure 3 shows the speed for two weather conditions with and without snow. Speed distributions and speed heat-maps generated using the same analysis automation and trajectory filtering tools as [41] were presented as an intuitive way of the results.

Figure 4 presents the accumulative distributions of TTCs for two weather conditions with and without snow. In good weather conditions, a total number of 563 TTCs were detected in the video, while in snowy weather, this number was 505. Among all interactions (the blue line), drivers had more intensive TTCs (smaller than 3 s) under good weather conditions, which is proof that under snow conditions, drivers drive more cautiously and adapt their behavior to avoid potential conflicts.

As discussed, in snowy weather, areas with higher occurrences of potential conflicts have a higher possibility of occurring of behavioral adaptations, such as variable accelerations and decelerations, and therefore should be prioritized in winter maintenance operations. With the location of potential conflict points known, known as the TTC heat-map, these areas can be identified on an aerial map, helping agencies in the road treatment process.

As an example, the heat-map in this case study generated by the software [41] was combined with an aerial map of the studying sites. The aerial map with the locations of the potential collision points, named the aerial TTC map, is presented in Fig. 5. On the map, areas with the highest density of potential collision points should be the first road segments to be addressed by winter road maintenance crews. These areas also require to be salted more often because as more behavior adaptations occur, more salt is dispersed on the pavement surface.

Fig. 5
figure 5

Aerial TTC map

Full size image

5 Conclusions and future work

As part of the contributions of this research, a methodology was proposed for analyzing the effect of winter on highways using microscopic measures such as vehicle-level speeds, lane change ratios, and surrogate safety measures.

A procedure for extracting driver behavior information from video data was demonstrated. The speed results in both case studies showed that surface conditions during a snowstorm influenced drivers to significantly lower their speeds. Based on these results, it can be inferred that drivers employed a certain degree of risk compensation in order to avoid potential severe conflicts caused by a lack of tire friction on the road. Despite their limitations, the video-based results showed a certain risk compensation effect. Vehicle-level speeds significantly decreased during the snowy weather, in particular when weather and road conditions were poor. This risk compensation phenomenon should translate into less severe crashes due to the longer braking distance which results in less kinetic energy. In other words, the consequences of a crash (probability of being injured or die) are proportional to the speed. As discussed by Wallman et al. [42], if the speed of a vehicle is reduced from 100 to 90 km/h (10 %), braking distance is reduced by 19 %, and injuries and fatal accidents are expected to be reduced by 27 % and 34 %, respectively.

The computer vision techniques have the advantage of being able to process large amounts of data and compute surrogate measures such as TTCs and post-encroachment times (PETs). The first case study demonstrated the issues of collecting high-accurate video data in adverse weather. Using videos with appropriate quality, the results were then validated in the second case study. The use of TTC as a surrogate behavior measure was illustrated for an analysis of winter driver behavior. An analytical process which offers an example of winter road maintenance guidance was provided and presented in the methodology section of this paper and tested in the second case study to be a successful example of linking the automated approach to practical winter road maintenance.

The lane change ratio was also used as an indicator. Based on this indicator, it was observed that the proportion of lane-changing maneuvers significantly decreased with snow precipitation and the presence of snow on the road surface. The automated method has the advantage of having low cost and being much less time consuming. However, it has limitations under adverse weather conditions especially considering that video data collected from regular surveillance equipment (including highway cameras) are not of great quality and high resolution. Several typical issues were encountered during the processing of the video data, which were documented in this paper. Potential solutions to the limitation of surveillance video data include i) applying special treatments to the camera to prevent the accumulation of snow or rain on its lens, ii) adjusting the angle of the camera to help reduce water reflection and direct sunshine, and iii) using thermal video cameras which can detect and track objects regardless of visible illumination using infrared radiation emitted from moving objects. Infrared technology works by detecting differences in temperature between people, vehicles, and the road and is unaffected by the glare of the sun and vehicle headlights as well as reflections from water accumulated on the road surface.

More efficient automatic methods are expected to help in the identification of locations requiring the priority of winter road maintenance operations as well as in the evaluation of the effectiveness of alternative treatment strategies.

More work concerning different aspects of road safety needs to be studied such as getting higher quality videos, defining and applying a proper measurement method for time gaps, and building lane-changing models using trajectory analytics software. The use of thermal video cameras and computer vision techniques will help in data collection accuracy in both good and snowy weather conditions. Automated methods will help reduce the need for manual analysis and collect more detailed driver behavior information. In our future work, with the help of the thermal camera collecting video data under snowstorm conditions, a detailed surrogate safety analysis on highway sections and ramps can be generated to investigate different targets such as evaluating the economic and safety impacts of winter maintenance treatments, determining storm durations, calculating occurrences of storms at nighttime, and evaluating driver adaptation to snowy conditions. A proactive approach to safety through the use of surrogate measures could help in the implementation of more effective winter maintenance operations.