Equipment and Software
The equipment used to obtain low-level aerial imagery included a multirotor unmanned aerial platform Tarot X6, equipped with a Panasonic DMC-GH4 camera with a 16.1 Mpx matrix and a DJI Zenmuse X5 lens (Fig. 2). The obtained images were processed using the Agisoft Lens and PhotoScan software, and the geovisualisation in the VR system was created with the aid of the Unity 3D software, which is a 3D game engine.
Research Procedure
Creating a geovisualisation in the VR system entails a sequence of activities. The complexity of the conducted study required us to devise a specially designed geomatic workflow of the research process, illustrating the various research stages (Fig. 3). The individual components of the workflow will be described in more detail in later parts of the paper.
Planning the UAV Flight
At the office stage of planning the flight, an important role is played by cartometric map bases, which are helpful in the situational orientation of the studied area (Kędzierski et al. 2014). Moreover, in order to define the area of the flight and the ground control station, it is a good idea to use satellite photographs of the studied area (Ostrowski and Hanus 2016). During the office stage of flight planning, we designed the path of the UAV flight over the studied natural aggregate pit (Fig. 4). Based on the analysis of the area and the studied object, we decided to take oblique photographs of the pit from its every side (west, east, north, south) as well as vertical photographs from directly above it. The object was divided into five observation sides: northern, southern, eastern, western, and vertical. Eight oblique observation posts were defined for the UAV’s flight around the object (two for each side) as well as two vertical observation posts. A flight conducted in such a way would yield a series of overlapping photographs of the studied object from every aspect.
In the conducted study, we used a cartometric map base as well as orthophotomaps downloaded from the national geoportal, which served to identify potential spots for establishing ground control. While planning the photogrammetric flight, it was decided that in order to obtain a model representing the relief with a sub-centimetre accuracy, photogrammetric control would be established in the area—a necessary step for conducting the georeferencing process, and eventually generating a numerical terrain model (Uysal et al. 2015). In addition, an intermediate aim of this paper was to conduct an independent check of the situational and height accuracy of the photogrammetric model obtained in the georeference process. That was why additional check points were established in the area of the study, constituting an independent control network (Gonçalves and Henriques 2015). It was assumed that the check of the obtained photogrammetric model would consist in comparing the coordinates of the afore-mentioned control points measured with the Global Navigation Satellite System Real Time Kinematic (GNSS RTK) technique with the coordinates of the same points obtained from the numerical terrain model.
Establishing Ground Control
During the surveying activities performed with the help of UAVs, what is important is not only the proper design of the flight itself, but also the network of points constituting ground control (Nex and Remondino 2014). In a situation when the unmanned platform used in the flight is not equipped with an inertial measurement unit (IMU), or when the received satellite signal is interrupted, establishing ground control is a necessity, even at the office stage of planning the flight (Anai et al. 2012; Barazzetti et al. 2010; Eugster and Nebiker 2008; Wang et al. 2008). The process is especially necessary when the aim is to generate a cartometric terrain model. Ground control should be established using geodetic techniques such as satellite observation or tacheometry. It is believed, however, that nowadays the best and most effective measurement method is to conduct satellite observations in real time, which also contributes to reducing image distortions (de Kock and Gallacher 2016).
In the study in hand, ground control points (GCPs) were established using the GNSS RTK technique, with the aid of the Trimble R4 Model 3 receiver. Taking into consideration the surveying technique used and the planar coordinate system EPSG:2177 (the one applicable in Poland, used in geodetic works) (Bosy 2014), it was decided that all the measurements made would be defined in this system for practical reasons. Due to the fact that UAV-obtained images are often oblique, it is recommended that a great number of control points be established in the surveyed area (Ruzgienė et al. 2015). Taking into account the size of the studied object and the planned altitude of the flight, four targeted GCPs were established (Fig. 5). They were located on every side of the observed aggregate heap. The ground control points were established in the field with the use of wooden poles, additionally marked with fluorescent orange paint so they could be easily identified in the photographs (Siebert and Teizer 2014). Additionally, in order to perform a situational and height accuracy check of the obtained photogrammetric model, seven independent check points (ICPs) were established in the studied area and labelled with numbers from 1 to 7 (Fig. 5). The decision to establish seven independent check points (ICP) was motivated by several factors:
-
(a)
Viewing the heap as a three-dimensional object, one might notice that (1) in terms of its vertical structure it consists of a foot and a summit. Hence, there should be points established in both those parts; (2) from the north, the object is shaded; from the south—lit by sunlight. Hence, establishing a single control point on each of those sides could be insufficient. Consequently, two points were established on each side (the shaded and the sunlit one). Two additional control points were established on the summit.
-
(b)
One of the objectives of the study was to check whether for the applied research methodology, increasing the distance from which the photograph was taken would affect the accuracy of the obtained 3D model. To this end, one additional point (a third one) was established on the southern side, as photographs of this side were taken from a greater distance than those of the northern side (Fig. 6).
ICPs were not used in the process of georeferencing the images (Toutin and Chénier 2004).
Conducting the Flight
The platform used in the study was equipped with an inertial measurement unit (IMU) and a GPS receiver; however, the coordinates were only used to stabilize the flight in the GPS mode and on the basis of available research (Gonçalves and Henriques 2015). It was assumed that the currently attainable accuracy levels are not sufficient for photogrammetric purposes. Hence, the flight was carried out in the manual mode, thus acquiring photographs on the basis of image obtained in real time. Taking into account the size of the studied object and the legal restrictions in Poland related to UAV flights, we decided that a sufficient elevation of the flight would be 100 m AGL (above ground level) and that the photographs for further analysis would be taken from all sides of the object. In accordance with the currently binding regulations in Poland (Regulation of the Minister of Infrastructure and Construction, Dz. U. 2016, Item 1993), carrying out UAV flights with a platform not exceeding 25 kg of take-off weight is allowed up to an altitude of 100 m above ground level (AGL) and at a distance of more than 6 km away from the nearest airport.
The weather conditions during the flight were favourable: wind speed approximately 3 m/s and a clear sky. During the 30-min flight, a total of 103 images of the object were obtained. The given time of the flight included 10 min taken to prepare the platform for flight and run a checklist ensuring flight safety.
Manual Selection of the Images for Further Analysis
Before proceeding to further stages of the study, the researchers performed a manual harmonisation of the obtained set of images, reducing their number while making sure to retain the ones which presented the studied object in the best possible way. As a result of this stage, ten photographs were chosen for further analysis—two images from every direction and two vertical images presenting the natural aggregate heap (Fig. 6).
Georeferencing the Selected Images
The next stage consisted in georeferencing the chosen images into a metric coordinate system. The first step towards georeferencing them into a coordinate system, identical to the one in which the ground control points had been defined, was to recreate their relative internal orientation. This process was performed on the basis of the EXIF file containing metadata about the photographs which made it possible to estimate the location of the images in space. Owing to the necessity of generating a cartometric DTM, the next stage was to recreate the relative orientation of the points, but already on the basis of the established GCPs. The task was completed using Agisoft PhotoScan Professional software. This programme is especially useful for processing UAV-obtained images, as it enables its user to generate a DTM or an orthophotomap in any chosen coordinate system.
It must be added, however, that before the calculation process was performed in the above mentioned software, a twin programme, Agisoft Lens, had been used to calibrate the camera used during the UAV flight. Brown’s distortion model implemented in this software made it possible to determine and factor into further calculations the radial and tangential distortion of the images.
After performing the georeferencing process on the basis of the established ground control, the RMSE (root-mean-square error) was calculated—a value specifying the deviation of the original tie-up points from the corresponding points calculated on the basis of the generated model. For calculating the RMSE, the following formula was used (Smaczynski and Medynska-Gulij 2017):
$${\text{RMSE}} = \sqrt {\mathop \sum \limits_{i = 1}^{n} \frac{{\left( {X_{{i, {\text{est}}}} - X_{{i, {\text{in}}}} } \right)^{2} + \left( {Y_{{i, {\text{est}}}} - Y_{{i, {\text{in}}}} } \right)^{2} + \left( {Z_{{i, {\text{est}}}} - Z_{{i, {\text{in}}}} } \right)^{2} }}{n}} ,$$
where X/Y/Zi,in—the actual value of a particular coordinate, X/Y/Zi,est—the estimated value of this coordinate.
On the basis of the obtained results, it was concluded that the ground control point whose total error of georeferencing had the lowest value was point D, and the point burdened with the greatest error—point B (Table 2). The further part of the study involved a check on the basis of independent check points.
Table 2 The calculated georeferencing errors of the GCPs
Generating Point Clouds
The next step towards the creation of the terrain model was to use the obtained images as the basis for generating a set of three-dimensional point clouds of the studied object. At this stage, three point clouds representing the landform were created, differing in their construction parameters. The first point cloud was created on the basis of photographs in their original size, i.e., 4608 × 2592 pixels (Model 1). The second cloud (Model 2) was generated after reducing the size of the images 4 times (which meant that the length of each side of the photo was reduced twice). The last, smallest point cloud (Model 3), was created from images reduced 16 times in size, i.e., 4 times on every side.
Choosing the Optimum Point Cloud and Calculating the 3D Model
Table 3 presents the characteristics of the three point clouds and the photogrammetric 3D models generated from them. As expected, the time needed to calculate Model I with the greatest precision in representing the terrain was the longest, amounting to nearly four hours. A disadvantage of this model, directly connected with the size of the created point cloud (over 23.5 mln points), was the very large size of the output file (363 MB), which could render it impossible for implementation into the VR system. Bearing in mind also the fact that Model I yielded the greatest value of the RMSE error (2.05 cm), the researchers decided to exclude it from further stages of the study. In Model II, the point cloud was much smaller (over 1.5 mln points), which reduced the file size to only 6.83 MB. The calculation time was also greatly reduced, to below 4 min, yet the model provided a very accurate representation of the terrain (RMSE 1.26 cm). Model III was constructed from the lowest number of points (approximately 0.4 mln), which, as in the previous cases, influenced file size (1.21 MB). However, it yielded worse results in terms of the RMSE error (1.58 cm). Taking into consideration the much worse parameters of Model III and the insignificant difference in the size of the output files, the model chosen by the authors as more appropriate for further geovisualisation in the VR system was Model II.
Table 3 Parameters of the generated 3D models Exporting the 3D Model into the Unity 3D Environment
Based on the results obtained in the previous stage, the model chosen to undergo further analyses was Model II, calculated from a cloud of 1,537,422 points (90 points/m2). On the basis of the cloud, a 3D model of the studied area was generated, consisting of 50,806 vertices, translating into 101,128 triangles making up the area of the model. Next, a texture was applied to the generated model, to reflect the real appearance of the natural aggregate heap (Fig. 7). The last step was to export the resultant model into the *.OBJ format, which enabled its subsequent implementation into Unity 3D.
Importing the Model into the VR Environment in Unity 3D
Activities at this stage were carried out in the programming environment of Unity 3D, which is an engine for the creation of computer games. The Unity project was first upgraded with the Google VR package, which boosts the functionality of the Unity engine, enabling it to display content in the VR system. The Google VR package provides additional features such as: user head tracking, side-by-side stereo rendering, detection of user interaction with the system, distortion correction for a VR viewer’s lenses. After the upgrade, the 3D model of the aggregate heap was imported and set to a proper scale. Because by default the model was displayed without any texture, texture was added in order to give the user an impression of greater realism while visually analysing the terrain. Finally, the virtual world was made complete by adding the sky as the background. The authors used Google Cardboard, which is the cheapest and easiest VR platform to obtain. This smartphone-based headset uses the user’s smartphone as a device to display the VR content.
Implementing the Virtual Reality First-Person View Mode (VR FPV)
A key element of the geovisualisation in the VR system is the FPV mode which enables the observation of objects from the pedestrian perspective. This works by displaying on the HMD two slightly different angles of the scene for each eye, which generates an illusion of depth (Fig. 8). This, along with other methods of simulating depth, such as parallax (objects farther from the observer seen to move more slowly) and shading, creates an almost lifelike experience. In this mode, the user becomes the main animator of the geovisualisation; all his/her movements are registered by sensors in the HMD and faithfully rendered in the geovisualisation. The VR system generates two stereoscopic images, one for each eye. This creates an impression that the user is immersed in the geovisualisation; in other words, the geovisualisation reaches its recipient directly. An additional advantage of the VR FPV mode is the possibility of adjusting the height of the observed picture to the actual height of the observer.