Estimation Method of Parking Space Conditions Using Multiple 3D-LiDARs

Abstract

In the early stages of the spread of autonomous vehicles, it is conceivable to operate an automated valet parking system in parking lots where autonomous vehicles and non-autonomous vehicles coexist. Since non-autonomous vehicles may park beyond the parking space, it is necessary to estimate parking space conditions three-dimensionally. This paper proposes a method to estimate the parking space conditions using multiple 3D-LiDARs that can detect the space three-dimensionally. In the evaluation experiment, multiple 3D-LiDARs were installed in the parking lot of a public facility, and the estimation accuracy of the proposed method was evaluated in various situations.

Introduction

Automated valet parking is one of the future visions of fully automated autonomous vehicles [1]. An autonomous vehicle automatically drives in a parking lot and parks in a specified empty parking space. Passengers can request their autonomous vehicles from any place. All works done by conventional valet parking employees are done automatically.

An automated valet parking system is detected each parking space condition from sensors installed in a parking lot. Accurate detection of parking space condition is important to appropriately provide a parking space position for autonomous vehicles. There are many technologies to detect parking space conditions. The number of vehicles that have passed through the gates installed at the parking lot’s entrance and exit is used to estimate the parking space condition in a gated parking lot [2]. It is possible to determine the availability of parking spaces, but it is impossible to determine the condition of each parking space. Magnetic sensors [3, 4] and inductive-loop vehicle detectors [5] are buried in the ground and detect the presence or absence of a vehicle above the sensors by changing the magnetic force. Ultrasonic-based vehicle detectors [6] are installed on the top and sides of parking spaces and detect the presence or absence of a vehicle based on the reflection of the ultrasonic wave emitted from the sensor. These sensors only scan a portion of the parking space and detect the presence or absence of a vehicle. Parking lots where autonomous vehicles and non-autonomous vehicles coexist are conceivable in the early stages of the spread of autonomous vehicles. Non-autonomous vehicles may park beyond a parking space, and parking spaces adjacent to the vehicle may be inaccessible. Sensors that scan only a portion of the parking space are ineffective in this situation. To change the position of the parking spaces in the parking lot, it is necessary to dig up or remove the sensor. Vision-based vehicle detectors [7,8,9] are installed in places where the vast area can monitor the parking space conditions using image processing technologies and deep learning. Since visual-based vehicle detectors can three-dimensionally detect parking spaces, it can handle the protrusion of the parking space of the parked vehicle. Even if the parking space position is changed in the parking lot, it is not necessary to change the sensor position. However, in places where the light source is insufficient, such as parking lots without lights at night, nothing is reflected in the camera image, so the vehicle cannot be detected.

Three-dimensional light detection and ranging (3D-LiDAR) are also sensors that can detect spaces in three-dimensions. A 3D-LiDAR measures range by irradiating lasers and measuring the time for the reflected light that hits objects to return to the receiver. A 3D-LiDAR outputs point cloud data that summarizes the reflection points. Since a 3D-LiDAR uses lasers, it can accurately detect space in three dimensions, even in places with no light source. There are several methods to detect parking space conditions using a 3D-LiDAR. Lee et al. [10] proposed a method for detecting the condition of parking spaces using a 3D-LiDAR mounted on the top of the autonomous vehicle. In this method, an autonomous vehicle equipped with a 3D-LiDAR approaches a parking space, and the condition of the parking space is determined based on the presence or absence of point cloud data from the road surface to the height of the 3D-LiDAR. Because this method assumes that the road surface is horizontal with respect to the vehicle’s posture, it cannot be used in areas where the road surface is uneven. Thornton et al. [11] also proposed a method for detecting the condition of a parallel parking space on a public road using a 3D-LiDAR mounted on the top of the autonomous vehicle. Based on the curbs, this method detects the parking space condition.Parking spaces with detectable curbs are determined to be vacant, while parking spaces with undetectable curbs are determined to be occupied.This method cannot be used in the parking spaces without curbs. In addition, there is no literature reporting the detection of parking spaces only using 3D-LiDARs installed at fixed points in a parking lot.

This paper proposes estimation method of parking space conditions using multiple 3D-LiDARs installed at various fixed points in a parking lot. The contributions of this paper are two points.

The first point is classification method of point cloud data using a high-precision map. A high-precision map is one that has lane-level detail and is used by autonomous vehicles for self-position estimation and route selection. A high-precision map reflects the actual size of parking spaces and road surface height. As a result, using the high-precision map’s parking space road surface information, the point cloud data in the parking space can be extracted and classified as observation points of objects and road surfaces.

The second point is an estimation method of parking space condition by coordinating multiple 3D-LiDARs. The presence or absence of point cloud data in a parking space is related to the parking space’s condition. After identifying the point cloud data in each 3D-LiDAR, the number of objects and road surfaces point cloud data in each parking space are calculated. When the same parking space is detected in a 3D-LiDAR with different installation locations, the number of object and road surface observation points is summed. Finally, the parking space condition is estimated based on the number of object and road surface observation points.

The remainder of this paper is organized as follows. Frist, Section 2 describes the assumptions of this study. Next, Section 3 introduces our proposed method Subsequently, Section 4 evaluates the method in an actual environment, and Section 5 presents the results. Finally, Section 6 discusses our proposed system, and Section 7 concludes the paper.

Assumptions

Figure 1 shows a schematic diagram of the monitoring system of parking space condition assumed in this study. The parking lot is assumed to have a specific size, such as a shopping mall parking lot. 3D-LiDARs are installed at fixed points in various places in the parking lot. Each 3D-LiDAR is connected to a personal computer (PC) that processes the point cloud data output from 3D-LiDARs and sends detection results to a management terminal. In addition, a management terminal estimates the parking space conditions.

Fig. 1

Schematic diagram of the monitoring system of parking space condition

All PCs are assumed to have the same high-precision map containing parking space information. A high-precision map is expressed with lane level detail. Autonomous vehicles often use it for self-position estimation and route selection. The parking spaces are represented by quadrangular and assigned a unique ID. Each quadrangular vertex　has position information expressed by the altitude value of the road surface and the coordinate value represented by the plane orthogonal coordinate system.

Proposed Method

This section describes a method for classifying point cloud data using a high-precision map and a method for estimating the condition of a parking space by coordinating multiple 3D-LiDARs.

Classification of Point Cloud Data Using a High-precision Map

The point cloud data output from a 3D-LiDAR are represented by a coordinate system with the 3D-LiDAR as the origin. In this data representation, it is impossible to know which region the point cloud data is observed. Therefore, the point cloud data are converted into the plane rectangular coordinate system similar to the high-precision map. The position and orientation of a 3D-LiDAR in the plane rectangular coordinate system are estimated in advance using normal distribution transform scan matching [12], which is often used for self-position estimation of autonomous vehicles [13].

The road surface information of the parking space on the high-precision map is used to classify the point cloud data as observation points of objects and road surfaces. Figure 2 shows a schematic diagram of the point cloud data classification method using a high-precision map. First, the parking space with the observation point is estimated using the parking space information on the high-precision map. Next, the height of the observation point and the average altitude of quadrangular vertices that represents the parking spaces on a high-precision map are compared. The height of observation point of the road surface and the altitude of road surface on the high-precision map do not completely match in reality due to the unevenness of the roadway surface and the vibration of a 3D-LiDAR. Thus, the observation points within a specific range from the road surface on the map are defined as the observation points of the roadway surfaces, and the observation points higher than that range defined as the observation points of objects.

Fig. 2

Classification of point cloud data using high-precision map

Estimation of Parking Space Conditions Based on Multiple 3D-LiDARs

As an example, the proposed method is explained using the environment shown in Fig. 1. Figure 3 shows a schematic diagram of the proposed method’s system in the environment shown in Fig. 1. A PC classifies the point cloud data using the method described in Section 3.1, and then sends the parking space ID, number of object and road surface observation points to a management terminal. When multiple 3D-LiDARs are connected to a PC, the number of object and road surface observation points is combined and transmitted to a management terminal. The management terminal integrates the number of object and road surface observation points with the same parking space ID.

Fig. 3

Schematic diagram of the proposed system

The condition of a parking space is estimated based on the number of object and road surface observation points. Figure 4 shows that the point cloud data are classified as the object and road surface observation points using the classification method in Section 3.1. In Fig. 4, square areas surrounded by white lines are the parking spaces, the green points are the observation points of objects, and the red points are the observation points of the road surface. Furthermore, a vehicle is parked in the left parking space, and there is nothing in the center and right parking space. In the parking spaces where the vehicle exists, there are observation points of objects. In the parking space where the objects don’t exist, there are no observation points of objects, only observation points of the road surface. So, the parking spaces where the number of observation points of road surface below the threshold value set in advance defined as unknown.

Fig. 4

Definition of parking space condition

From the above characteristics, a parking space condition is estimated as follows:

• Non-Parkable:

  A parking space where object observation points exist.

• Parkable:

  A parking space where there are no object observation points, and the number of road surface observation points is over the threshold value.

• Unknown:

  A parking space where there are no object observation points, and the number of road surface observation points is under the threshold value.

Evaluation Experiments

The estimation accuracy of the proposed method was evaluated. The experiment was conducted in a parking lot of a public facility in Aichi, Japan. Multi 3D-LiDARs are installed at a fixed point in this experimental environment. The estimation accuracy of the proposed method was estimated in various situations. The evaluation experiment was conducted from 11:00 to 15:00 on February 9, 2021.

The presence or absence of vehicles in each parking space was recorded at 1-min intervals by a camera installed at a position overlooking the entire parking lot. The parking space condition obtained by the proposed method was compared with this recorded data. Then, the match rate, the mismatch rate, and the percentage of the proposed method outputting unknown (unknown rate) were calculated. The system was built with the robot operating system.

Experiment Environment

Figure 5 indicates the parking lot where the experiment was conducted. The parking lot is 20 m long and 50 m wide and has 55 parking spaces. In the evaluation experiment, three 3D-LiDARs were installed at the height of approximately 3.5 m in a position where the entire parking lot could be covered (see Figs. 5 and 6).

Fig. 5

The experimental parking lot

Fig. 6

3D-LiDARs installed at a fixed point

Each 3D-LiDAR is connected to a PC via a hub (see Fig. 7). Tables 1, 2 and 3 demonstrate the specifications of the PC, 3D-LiDAR, and hub used in the experiment. The laser irradiation unit of the 3D-LiDAR rotates 360 degrees and goes back to its original position every 0.1 s. The 3D-LiDAR outputs point cloud data that summarizes the reflection points obtained during one rotation of the laser irradiation unit. The position and orientation of each 3D-LiDAR were estimated using normal distribution transform scan matching in advance. The position and orientation of each 3D-LiDAR are shown in Table 4.

Fig. 7

Configuration of system

Table 1 Specification of the PC

Table 2 Specification of the 3D-LiDAR

Table 3 Specification of the hub

Table 4 Position and orientation of the 3D-LiDARs

Vehicle stops are installed in parking spaces throughout the lot. Vehicle stops are approximately 9 cm high. As a result, the threshold value for classification of observation points was set to 10 cm to ensure that vehicle stops were not recognized as objects. That is, observation points within 10 cm of the road surface height on the map are defined as the road surface observation points, while observation points higher than 10 cm are defined as object observation points.

The threshold value for detecting the parking space condition was defined as 10 points by a trial experiments. Parking space condition was estimated as follows:

• Non-Parkable:

  A parking space where object observation points exist.

• Parkable:

  A parking space with no object observation points and more than 10 points road surface observation.

• Unknown:

  A parking space with no object observation points and less than 10 points road surface observation.

Point Cloud Data Obtained From 3D-LiDARs

Figure 8 is a visualization of the point cloud data obtained from 3D-LiDARs in the experimental environment. In Fig. 8, the rectangular regions surrounded by the white lines are the parking lot spaces, the yellow points are the point cloud data output by lidar a, the red points are the point cloud data output by lidar b, and the blue points are the point cloud data output by lidar c.Consequently, three 3D-LiDARs can detect the entire parking lot area. It can be confirmed that there are parking spaces with sparse or dense point cloud data.It can also be confirmed that there are parking spaces detected by one 3D-LiDAR or multi 3D-LiDARs. Various situations can be created.

Fig. 8

Point cloud data obtained from 3D-LiDARs in the experimental environment

Results

Table 5 lists the match, mismatch, and unknown results when parking spaces are parkable or non-parkable. The match rate in the non-parkable space is high, but there is some estimation mismatch. Figure 9 presented the state of the parking space when the mismatch occurred in a non-parkable space. In Fig. 9, the place surrounded by the red frame is where the mismatch occurred. The green points represent the object observation points. Even though a vehicle was parked in the red frame, there was no object observation points, only road surface observation points. At this time, the vehicle was in the gap of the laser emitted from 3D-LiDAR (see Fig. 10). In addition, part of a laser passed under the vehicle and detected the road surface. So, the system determined that the parking space is parkable.

Table 5 Experimental results

Fig. 9

The parking space at the mismatch occurs in a non-parkable space

Fig. 10

The positional relationship between the vehicle and 3D-LiDAR at the mismatch occurs in a non-parkable space

Figure 11 shows the condition of the parking space when the mismatch occurred in a parkable space. In Fig. 11, the mismatch occurred in the area surrounded by the red frame. In Fig. 11, the vehicle’s point cloud data extends beyond the parking space, and it can determine whether the proposed method’s result is correct. The proposed method accurately detected the condition of parkable parking spaces.

Fig. 11

The parking space at the mismatch occurs in a parkable space

There are also parking spaces that could not be determined the parking space conditions. Figure 12 shows the condition of the parking space that proposed method could not be detected in a non-parkable space. In Fig. 12, the place surrounded by the red frame is where the system could not be determined the parking space condition. Even though a vehicle was parked in the red frame, there was no observation points. At this time, there were vehicles in the parking space on both sides. The laser emitted from 3D-LiDARs did not impact the vehicle parking in the space due to the positional relationship between the surrounding objects and 3D-LiDARs.

Fig. 12

The parking space at the unknown occurs in a parkable space

The match rate is low, and the unknown rate is high in parkable spaces because lasers were blocked before impacting the road surface due to the positional relationship between the surrounding objects and 3D-LiDARs.

The proposed method can detect the parking space conditions except for the parking spaces which is not completely invisible due to occlusion and parking spaces with few observation points.

Discussion

This section discusses the technical specifications and the advantage of the proposed method.

Influence of Threshold Values

The impact of the threshold for classifying point cloud data and the threshold for estimating parking space conditions is discussed.

As the classification threshold value increases, the probability of identifying the observation points of objects as the observation points of the road surface increases. To properly classify the observation points, it is preferable to keep the classification threshold low. When the classification threshold value is small, the probability of the parkable parking space being identified as a non-parkable space increases due to unevenness in the road surface, an error in posture estimation, and vehicle stop. As a result, when determining the classification threshold value, the approximate value is determined based on road surface unevenness and vehicle stops. The threshold value is then adjusted to account for sensor position and attitude error.

When the threshold value for estimating the parking space condition is small, the parking space condition is detected as a parkable space with a small number of road surface observation points. The unknown rate decreases, but the possibility of mismatch at parkable space increases. This threshold value must be determined by a trade-off between the mismatch rate and the unknown rate at parkable spaces.

Advantage of Proposed Method

The proposed method’s advantages are discussed. The proposed method estimates the condition of parking spaces based on the number of road surface and objects observation points classified using a high-precision map. This allows you to deal with the vehicle that is protruding from the parking space. Sensors that cannot detect three-dimensional space cannot deal with this. The proposed method can respond to changes in the location of parking spaces in a flexible manner. A 3D-LiDAR sensor measures space in three-dimensions. Because a high-precision map is required for autonomous vehicle position estimation and route setting, it will be updated if the parking space is changed. By reading the updated high-precision map without changing the position of sensors, it is possible to estimate the condition of parking spaces.

The frequency with which parking spaces are scanned is determined by the time required to classify the point cloud data. The time it takes to classify the point cloud data is affected by the number of parking spaces and observation points. The number of parking spaces in this paper’s experimental environment was 55, and the average number of point clouds output by a 3D-LiDAR per 0.1 s was about 14,900 points. The point cloud data was classified in 0.04 s. The time required to classify point cloud data grows in direct proportion to the number of parking spaces and observation points. The processing time can be reduced by distributing the classification of point cloud data among multiple processors. As a result of distributed processing, even if the number of parking spaces and observation points increases, the time required to detect the condition of parking spaces does not increase.

The proposed method determines the parking spaces using a high-precious map and 3D-LiDARs. These are usually owned by autonomous vehicles. When the number of autonomous vehicles increases in the future, the parked autonomous vehicles are used as a fixed-point 3D-LiDAR. It is possible to increase the number of 3D-LiDARs without increasing the number of 3D-LiDARs installed in the parking lot. Because the parking space can be detected without occlusion, the probability of accurately detecting the parking space condition increases.

Conclusions

This paper proposes a parking space condition estimation method using multiple 3D-LiDARs installed at various fixed points in a parking lot. The point cloud data output by a 3D-LiDAR are extracted for each parking space and classified into object and road surface observation points using a high-precision map. The number of object and road surface observation points is used to estimate the parking space condition. Because 3D-LiDARs can detect parking spaces in three-dimensions, they can handle the protrusion of the parked vehicle’s parking space and respond flexibly to changes in the position of the parking spaces. In the evaluation experiment, the proposed method was confirmed that the condition of parking spaces can be estimated appropriately in the parking spaces without occlusion.

The proposed method has poor accuracy in detecting the condition of parking spaces with occlusion. A method for estimating the parking space condition by tracking and route prediction using a high-precision map are considering. A method for estimating the condition of parking spaces without observation points using ray casting is considering.