Introduction

Currently, UASs play an increasingly pivotal role in the realm of spatial data collection. Unlike conventional air and satellite-based remote sensing methods, the UAS offers significant advantages in facilitating data acquisition and adapting to the specific requirements of different survey studies with a variety of spatial and temporal resolutions. This versatility is further underscored by the utilization of a broad spectrum of sensor types, all of which are achieved at a comparatively low cost (Yao et al. 2019; Iglhaut et al. 2019; Nex et al. 2022). The SfM method is a key solution for processing aerial imagery obtained by UASs. Notably, the parameters are integral to the processing workflow, including camera calibration, bundle adjustment, and orientation parameter adjustments, all of which are implemented in a user-friendly manner (Uysal et al. 2015; Iglhaut et al. 2019; Deliry and Avdan 2021).

The increasing adoption of UAS-based SfM techniques for mapping and modeling is evident from its growing utilization across various domains, including agriculture, environmental studies (Akay et al. 2022; Dai et al. 2022), and industrial applications (Haas et al. 2016; Rauhala et al. 2017), as well as urban environments (Ahmed et al. 2021). In contrast to previous methods dominated by aerial and terrestrial laser scanning for high-resolution topographic data production, UAS SfM demonstrates superior efficiency through its capability for flexible, target-oriented point cloud generation at resolutions approaching sub-decimeter scales, thereby facilitating more intricate analyses (Manfreda and Eyal 2023).

The efficacy of UAS SfM compared with laser scanning has been substantiated in various contexts. For instance, a comparison of the accuracy of the UAS-based SfM and the LiDAR digital surface model (DSM) in an urban area for flood estimation yielded inconsequential disparities (Escobar Villanueva et al. 2019). Studies also investigated the feasibility of employing the UAS-based SfM for pre-disaster condition assessment, determining that its vertical error tolerance (sub-decimeter) qualifies it for comparative analyses before and after disasters (Kucharczyk and Hugenholtz 2019). Furthermore, the method’s morphometric accuracy was evaluated alongside a terrestrial laser scanner (TLS), particularly for documenting historic districts (Parrinello and Picchio 2019). Additionally, UAS-based aerial mapping has proven invaluable in generating foundational data for estimating the roof area of agricultural structures, facilitating the efficient planning of solar panel installations (Koc et al. 2020). In built-up areas, Liu et al. (2022a) investigated the effects of UAS flight altitude and image overlap degrees, while Ahmed et al. (2022) explored the influence of varying camera angles and flight directions on the accuracy of UAS-derived outputs, namely, DSM and orthophoto mosaics.

The accuracy of the method must be rigorously assessed to ensure the efficiency and reliability of UAS SfM-based data. This entails quantification of the horizontal errors inherent in the produced orthophoto mosaics and vertical errors within the DSMs. The exigency for such assessments is grounded in the profound impact of these parameters on the overall uncertainty associated with the generated models (Inzerillo et al. 2018; Belcore et al. 2021). Various factors pertaining to UAS-based SfMs can influence the reliability of generated models. These include properties associated with direct or indirect external orientation parameters, such as the scale of GNSS measurement errors, the number and spatial distribution of ground control points (GCPs), camera characteristics (e.g., tilt of the camera principal axis, focal length, lens distortions), diverse weather conditions (e.g., lighting and atmospheric properties), and the quality of the resultant images (e.g., sharpness, resolution, blur). In addition, the processing software environment significantly contributes to the quality of the derived datasets (Barkóczi and Szabó 2017; Sanz-Ablanedo et al. 2018; Lehoczky and Siki 2020; Casella et al. 2020).

The significance of external orientation has been prominently underscored by numerous studies dedicated to revealing its details and effects. These investigations have collectively determined that an optimal spatial distribution of GCPs is achieved through their homogeneous and nonlinear dispersion within the study area (Manfreda et al. 2019; Villanueva and Blanco 2019; Gomes Pessoa et al. 2021; Ulvi 2021). The establishment of universally accepted principles dictating the minimum and optimal quantity of GCPs remains elusive, as studies employ varied benchmarking systems. For instance, Sanz-Ablanedo et al. (2018) investigated the influence of the number of GCPs per image on accuracy, revealing that beyond 3–3.5 GCPs per 100 images, no statistically significant improvement in accuracy was observed. Conversely, contrasting findings have been reported, indicating that there is no statistically significant enhancement in accuracy when the number of GCPs surpasses 10 per square kilometer (Liu et al. 2022a, b). In addition to the size of the area under consideration, the topography of the site markedly influences the required number of GCPs. Consequently, a site characterized by heterogeneous topography, with an average slope of 19–30%, may necessitate a higher number of GCPs (Agüera-Vega et al. 2017), in contrast to a homogeneous flat area (Reshetyuk and Mårtensson 2016).

An increasing number of investigations have emphasized the benefits associated with utilizing the oblique data acquisition method for evaluating UAS-based aerial data (Li et al. 2018; Wu et al. 2018; Vetrivel et al. 2018). The primary justification for this preference stems from the unique capability of oblique data acquisition to capture detailed information not only from the top but also from the sides of objects (Svennevig et al. 2015; Jaud et al. 2019; Cao et al. 2023), contrasting with nadir imaging. Its application is favored in scenarios where vertical or near-vertical surfaces dominate the study area. Research dedicated to quarry wall surveys has demonstrated that integrating nadir and oblique imagery improves the accuracy of DSMs while simultaneously addressing data gaps caused by shadowing effects (Rossi et al. 2017; Zapico et al. 2021). Similar results have been noted in investigations carried out in naturally high-relief regions (Nesbit and Hugenholtz 2019; Bi et al. 2021; Nesbit et al. 2022). The enhanced accuracy and heightened informational content evident in models created by integrating nadir and oblique images support the utility of this approach in computing the leaf area index (LAI) (Che et al. 2020; Lin et al. 2021). Furthermore, its progressive deployment in urban settings is attributed to its ability to provide adequate data on building frontages, which can be harnessed for precise generation of 3D building models (Verykokou and Ioannidis 2016; Wu et al. 2018; Pepe and Costantino 2020).

In addition to the aforementioned factors, subsequent investigations into the evaluation of UAS-SfM methods have focused on additional categories of variables that influence accuracy. These include parameters such as flight altitude and image overlap (Jeong et al. 2020; Anders et al. 2020; Liu et al. 2022b). However, the novelty of our research lies in (1) its comprehensive comparison of three distinct factors influencing accuracy, (2) the comparison of two different built-in study areas with different characteristics, and (3) the utilization of three reference groups, one horizontal and two vertical. Our study is unique in its simultaneous examination of these variables, which have not been studied together in this manner previously. This holistic approach allows for a more nuanced understanding of the interplay between various factors affecting accuracy in UAS-based surveys, particularly in urban environments. Furthermore, our investigation highlights the impact of processing software settings and duration on model quality and processing time, aspects often overlooked in accuracy studies. By addressing these gaps, our research not only advances the current understanding of UAS-based survey methodologies but also provides valuable insights for optimizing aerial mapping campaigns in urban settings. Considering our results, we aimed to underscore the universal limitations in applying the investigated survey conditions to enhance the general efficacy of urban aerial mapping campaigns with UASs.

Materials and Methods

Study Area

This research was conducted in two distinct study areas within Debrecen, Hungary, as illustrated in Fig. 1. The first area, designated as “Csapókert,” is situated in the north-eastern part of the city. It is characterized by a regular street network, expansive gardens, lower building density, and predominantly one- or two-floored structures. The second study area, “Úrrétje,” is in the north-western part of the city. This area features a denser and more irregular road network with smaller gardens and increased building density. It includes multistory blocks of flats, as well as terraced and detached houses. In contrast to Csapókert, Úrrétje exhibits a homogeneous flat terrain. Despite Debrecen’s overall flat landscape, there were disparities in the topography of the two study areas. Csapókert comprises two sand mounds ranging in height from 1.5 to 2 m, contributing to its topographical variation. On the other hand, Úrrétje represents a consistently flat terrain. The selection of these two areas was intentional and based on prior research findings (Kucharczyk and Hugenholtz 2019), which suggested that varying characteristics, such as building height, built-in area density, and spatial patterns, may influence the accuracy of both orthophoto mosaics and DSMs.

Fig. 1
figure 1

Study area

Full size image

UAS Data Acquisition

Image acquisition was executed using a DJI Mavic Pro quadcopter, equipped with an integrated 12-megapixel 1/2.3″ sensor featuring a focal length of 26 mm. The camera is fixed to the drone through a 3-axis gimbal, enabling the adjustment of the camera tilt along the X-axis from − 90° (nadir) to + 30°. Aerial surveys were conducted in the two designated study areas at almost simultaneous times, with merely a 5-day difference between the two assessments. To ensure comparability, both study areas were surveyed during the same time of day (early afternoon), and strict consistency was maintained in the flight parameters, encompassing various aspects. Throughout the entirety of the data collection process, a consistent flight altitude of 90 m above ground level was maintained, with a frontlap and sidelap of imagery set at 70%. Image acquisitions were performed using three different flight modes (Fig. 2):

  • - Single grid: the UAS flies over the area to be mapped in a series of parallel paths in a defined direction.

  • - Double grid: at the end of the initial flight pass, the UAS flies along a series of passes but perpendicular to the previous one.

  • - Perimeter 3D: the UAS flew a single-grid plan and a square bounding the area to be mapped, the former with a camera angle of 90° (nadir) and the latter with a camera angle positioned at the spatial center of the area (Fig. 2).

Fig. 2
figure 2

The applied data acquisition methods

Full size image

For data collection, two single-grid flights were implemented, one conducted in an east–west (E-W) flight direction and the other in a north–south (N-S) flight direction, in addition to a double-grid flight pattern. All three flight modes were executed with a 90° nadir and a 60° oblique camera angle. The number of photos captured for each data acquisition method is listed in Table 1. To establish accurate external orientation parameters, the aerial data collection process was complemented with GCP surveys. Prior to aerial data collection, the GCPs were established by positioning the signs along the roadway. The coordinates of these GCPs were surveyed using a Stonex S9i RTK GNSS system, which demonstrated an average positioning error of approximately ± 2 cm. In total, 23 GCPs were distributed for the Úrrétje area, while 25 GCPs were designated for the Csapókert area (Fig. 3).

Table 1 The numbers of the images of the different acquisition methods
Full size table
Fig. 3
figure 3

The distribution of GCPs in the two areas

Full size image

Reference Data

To evaluate the accuracy of the orthophoto mosaics and DSMs produced via SfM processing, two distinct types of reference data were utilized. In both study areas, an RTK GNSS device (Stonex S9i) was used to survey the GCPs and collect reference coordinates, as previously described. Given the nature of the areas, which are characterized by a high number of enclosed gardens, reference points were surveyed exclusively in public areas, typically on roadways. These points were employed to evaluate both the horizontal and vertical accuracies of the resulting orthophoto mosaics and DSMs. A uniform sampling approach was adopted for the analysis, involving the measurement of 100 points in each of the two study areas (Fig. 4). Additionally, to assess the vertical data accuracy, a LiDAR point cloud, acquired from a survey conducted using a RIEGL VQ-780II sensor, was utilized. Airborne laser scanning (ALS) data collection occurred 2 months after the UAS-based survey, with a flight altitude of 1078 m above ground level and an average point density of 16 points per square meter. DSMs were generated for both study areas at a spatial resolution of 0.25 m per pixel using LiDAR point cloud data. For this type of reference, 100 points were randomly generated for each area (Fig. 4). These points were then used to extract the height values from the rooftops of the buildings.

Fig. 4
figure 4

The position of the two types of references in the two areas

Full size image

Data Processing

UAS-based aerial imagery was processed using the Agisoft Metashape Professional (v1.5.1) software environment.

In our SfM processing workflow using Metashape, four distinct stages are included, which encompass the following procedure:

  1. I)

    During the Align Photos stage, the software identifies tie points to establish connections between different images, thereby creating a block for subsequent block adjustments. The outcome of this process is a sparse point cloud with default software settings. Following this, georeferencing of the models was conducted using GCPs.

  2. II)

    A dense point cloud was created using the built-in dense point cloud option. This involved employing low, medium, and high processing modes for various types of acquisitions, with aggressive depth filtering applied in all cases. The processing modes correspond to different image resolutions, with the software calculating 1/8 of the pixels for low,1/4 for medium, and 1/2 for high.

  3. III)

    The generation of DSMs followed, utilizing the dense point cloud data with interpolation enabled, and the default resolution specified by the software according to the ground sampling distance characteristics of the lens and image acquisition.

  4. IV)

    Finally, orthophotos were generated using the previously derived DSMs. The mosaic blending mode was employed, and the resolution was determined using software.

SfM processing was performed on a dedicated Workstation PC with an Intel Xeon E5-2670 CPU (12/24 cores/thread), 64 GB DDR5 RAM memory, and a Nvidia Quadro K420 GPU (2 GB VRAM) configuration.

Accuracy Assessment

The horizontal accuracies of the orthophoto mosaics were determined by calculating the absolute distances between the reference and model coordinates along the X- and Y-axes using trigonometry. These distances were treated as absolute (i.e., positive) values and did not incorporate directional information. For the DSMs, the disparities between the reference data (elevation data obtained from GPS measurements for roads and extracted elevation values from LiDAR DSM for roofs) and elevation values derived from the generated models were assessed.

Initially, the obtained difference values were analyzed using standard descriptive statistical methods. The RMSE was subsequently determined for both vertical and horizontal tests by evaluating the differences between the reference and model data.

Modeling with Residuals

The residuals, which denote the disparities between the SfM models and the reference date, were subjected to evaluation using random forest regression. In this analysis, the dependent variable was the residual values, while the independent variables included the resolution (level), flight pattern (image acquisition), flight direction (direction), and the study area. The reference data were divided into training and testing datasets at a 70:30 ratio, and the models were trained using the training set. The modeled values for the testing data were subsequently calculated, and the minimum, median, and maximum values were selected for further analysis by employing explainable machine learning techniques.

The models were evaluated using explainable AI metrics. The rank of the feature importance was determined by performing 50 permutations, and the contribution of the independent variables was evaluated using breakdown plots. These plots provide a visual representation of the variables’ contributions to a specific value of the dependent variable, indicating both positive and negative contributions. The calculations were carried out using R 4.2.2 (R Core Team) with DALEX and the forester packages.

The whole data collection, processing, and analysis process is shown in Fig. 5.

Fig. 5
figure 5

Flow chart of the steps performed during the study

Full size image

Results

Horizontal Accuracy

Comparing the error ranges between the two study areas using the same method and processing level, it was observed that in 19 modeling cases, the Úrrétje area exhibited smaller differences. However, for the Csapókert area, only the MPL (0.23 m) and LPL (0.37 m) processing levels of the P3D method demonstrated a better performance (Fig. 6). In terms of the RMSE values for distances, it was noted that only the P3D LPL mode yielded better results (0.5 cm) for the Csapókert area compared to the Úrrétje area. Importantly, for the same processing mode and level between the two study areas, the discrepancy in the RMSE values was less than 2 cm.

Fig. 6
figure 6

The result of the horizontal accuracy assessment

Full size image

The results demonstrated consistent patterns when comparing data collection methods in both study areas. Specifically, at the same processing level, the NG1 method consistently yielded the smallest error range, whereas the P3D method consistently yielded the largest error range. Correspondingly, for RMSE values, the NG1 method consistently provided the most accurate results, whereas the P3D method resulted in the least accurate outcomes (Table 2). However, when specifically considering RMSE values for oblique images, it was observed that the double-grid flight employed for both study areas outperformed the single-grid flight at all three processing levels, yielding better overall results.

Table 2 Values in meters of the median, range, and RMSE of the horizontal precision test
Full size table

Comparing the three processing levels examined when looking at the range values for both study areas, there were pairs of results: for example, in the NG1 processing Úrrétje area and in the OG2 method’s Csapókert. In the investigation of RMSE values, there were no cases in which lower processing levels resulted in better accuracy of the Úrrétje results. In addition, for Csapókert, only the OG3 method had a 0.2 cm higher MPL than the HPL.

According to the feature importance, the image acquisition method (0.0303) had the greatest impact, whereas the processing level (0.0298) had the least impact on the error rates. Except for the acquisition mode, the other three features were observed with minimal differences (Fig. 7). Breakdown analysis revealed that the Csapókert area and the oblique acquisition mode produced the best results with the smallest error.

Fig. 7
figure 7

The result of the feature importance test (A) and the breakdown analysis (minimum value (B), median value (C), maximum value (D)) of the horizontal accuracy

Full size image

The processing level (medium) did not significantly affect the error, whereas the double-grid method contributed to the error reduction. In terms of the average error value, the error was reduced by factors such as the Úrrétje area, camera angle, and N-S flight direction. Conversely, a low processing level increases error. Regarding the maximum error, the Úrrétje location contributed to error reduction, whereas the P3D method, low processing level, and the associated flight direction with P3D increased the error.

Vertical Accuracy—Road Surface

When comparing the two study areas, variations in error ranges were observed, depending on the data collection method and processing level. Specifically, the Úrrétje area showed lower error values in 12 instances, while the oblique and P3D methods yielded superior results for the Csapókert area. Regarding RMSE, the Úrrétje area demonstrated better outcomes in 18 cases, with only three exceptions. These exceptions occurred in OG1 LPL, where the difference between the two areas was 3.5 cm, and in OG2 for OG3 LPL, with a difference of less than 0.4 cm. Compared to the error ranges of different data collection methods at the same level, the NG1 method consistently demonstrated the highest accuracy. However, in both cases, the error range exceeded 1 m. This was notably observed with the NG3 LPL method (1.04 m) for the Csapókert area and the P3D LPL method (1.15 m) for the Úrrétje area (Fig. 8). In both areas, the utilization of the double-grid approach for the oblique method showed a significant improvement in the range values compared to the single-grid approach.

Fig. 8
figure 8

The result of the vertical accuracy assessment on the road surface

Full size image

Regarding RMSE, our results did not identify a universally superior method. Instead, they indicated that the OG3 method provided the lowest values of HPL in the Csapókert area, while the NG1 method was the best for MPL and LPL. In the Úrrétje area, NG3 achieved the best results for HPL and MPL, whereas NG1 obtained the best results for LPL (Table 3).

Table 3 Values in meters of the median, range, and RMSE of the vertical precision test of the road
Full size table

When comparing different processing levels, it was observed that in the Csapókert area, MPL yielded lower values than HPL for the NG1 and NG2 methods. In the Úrrétje area, only the NG2 method demonstrated lower values in the MPL than in the HPL. In terms of RMSE, only the NG1 and NG2 methods in the Csapókert area failed to exhibit lower errors for the HPL than for the MPL. Notably, we achieved 2.4 times lower errors in the error range (MPL-HPL) using the P3D method in the Csapókert area and 2.9 times lower errors using the OG3 method in the Úrrétje area. However, similar improvements were not observed for the RMSE values.

According to the feature importance test results, the processing level exerted the most substantial influence on the alteration of vertical accuracy values for roadways, registering a value of 0.1307. In contrast, the study area had the least impact, with a value of 0.1148. There was a more noticeable disparity in the significance of the different factors compared to the previous reference type. Further examination through the breakdown analysis unveiled intriguing insights.

When examining the minimum value (negative error) for the Csapókert area, the P3D mode, LPL, and E-W directional flight increased the magnitude of the error (Fig. 9). In contrast, for the median error in the Úrrétje area, the tilted camera, HPL, and double grid slightly increased the difference from zero. At the maximum value, except for the study area (Csapókert), the other three factors—oblique camera angle, MPL, and E-W directional flight—increased the error.

Fig. 9
figure 9

The result of the feature importance test (A) and the breakdown analysis (minimum value (B), median value (C), maximum value (D)) of the vertical accuracy of the road

Full size image

Vertical Accuracy—Roof

Assessment of the accuracy of the roof surfaces revealed substantial errors and range values. Upon comparison of the two study areas, it was evident that the Úrrétje area consistently demonstrated superior performance over the Csapókert area across all processing methods and levels, with disparities frequently exceeding several tens of centimeters. However, regarding RMSE values, the Csapókert area exhibited lower errors in six instances, specifically for the NG1 HPL, MPL, LPL, NG2 HPL, MPL, and Perimeter 3D LPL methods. The differences in RMSE between the two areas are notably smaller than the range of error values. When comparing the various data collection methods, it was found that the P3D method consistently yielded the smallest range of errors at the same processing levels in both study areas. Additionally, when focusing on the HPL and MPL, the OG3 method demonstrated considerable improvement in results compared to the other methods, with differences of only a few centimeters lower than those of P3D. Regarding the RMSEs in the Csapókert area, the OG1 method exhibited the lowest values for HPL (21.6 cm) and MPL (22.7 cm). The NG1 method produced the lowest values for LPL (23.8 cm). In the Úrrétje area, the NG3 method consistently performed the best across all three processing levels, with RMSEs of 15.3 cm for HPL, 15.6 cm for MPL, and 16.4 cm for LPL. Conversely, the P3D method yielded the poorest RMSE results for both the areas.

Concerning various processing levels, the considered reference type exhibited a noteworthy number of instances where the anticipated hierarchy (HPL > MPL > LPL) was not consistently observed. Regarding the range of errors, in the Csapókert area, MPL produced superior results for the NG1 and NG2 methods, while the LPL level yielded the best outcomes for OG1 (refer to Fig. 10). Conversely, in the Úrrétje area, the expected relationship was only observed for OG1. Upon analyzing the models generated from the three nadir flights, it was noted that the LPL resulted in the narrowest range of errors, while the MPL proved most effective for the OG2, OG3, and P3D methods.

Fig. 10
figure 10

The result of the vertical accuracy assessment on the roof tops

Full size image

Examining the RMSEs in the Csapókert area, it was found that the HPL yielded the lowest error values in three cases (NG2, OG1, and OG2), with no discernible difference between the HPL and MPLs in one case (NG3) (Table 4). MPL resulted in the lowest errors for NG1 and OG3, whereas LPL produced the lowest errors for the P3D method. In the Úrrétje area, the best results were obtained with the HPL for NG2, NG3, OG1, and OG2 methods, while the LPL yielded the lowest RMSE values in the remaining three cases.

Table 4 Values in meters of the median, range, and RMSE of the vertical precision test of the roof
Full size table

The analysis of feature importance revealed valuable insights regarding the vertical accuracy test conducted on the rooftops (Fig. 11). Image acquisition exerted the most substantial influence on accuracy, with a value of 0.2247. In contrast, the processing level had the smallest impact, with a value of 0.1543, which was notably lower than that of the other three factors.

Fig. 11
figure 11

The result of the feature importance test (A) and the breakdown analysis (minimum value (B), median value (C), maximum value (D)) of the vertical accuracy of the roof

Full size image

Further investigation through breakdown analysis indicated intriguing findings. In the case of the minimum value, it was found that the Csapókert area, P3D mode, LPL, and P3D flight direction increased from zero. Considering the median value, the Csapókert area and LPL decreased the error; however, the tilted camera had no discernible effect, while the E-W flight direction resulted in an increase. For the maximum, it was observed that the Úrrétje area, nadir camera angle, MPL, and E-W flight direction increased the error.

Data Collection and Processing Time

Considerable disparities in flight duration were evident in the analysis of the data collected. Single-grid flights demonstrated the shortest flight durations, as they followed the same trajectory irrespective of the camera angles, yielding negligible differences. In contrast, P3D flights were characterized by the longest duration. The duration of double-grid flights was less than twice that of single-grid flights, as detailed in Table 5. By comparing the two surveyed areas, it was observed that the average flight time in the Úrrétje area was 17% longer. Notably, within this area, the flight time associated with P3D exceeded the recommended overall flight time of 21 min by the drone manufacturer (Internet 1).

Table 5 The flight times of the different acquisition methods
Full size table

Upon analysis of processing times, our findings revealed an approximately threefold increase in computation time across distinct levels, regardless of the data collection methods employed. In the comparison between the two surveyed regions, the processing time for the Úrrétje data was observed to be 34% longer on average (Fig. 12). Specifically, at identical processing levels, NG3 processing required the greatest amount of time, whereas OG1 processing required the least time. Remarkably, data acquired through the double-grid method took approximately three times longer to process compared to each single grid for all three processing levels in both areas.

Fig. 12
figure 12

The processing times of the different processing levels and data acquisition methods within the two study areas

Full size image

Discussion

Experimental investigations were conducted in two distinct urban study areas with different built-up characteristics. This approach facilitated the comparative analysis and validation of general statements regarding the accuracy of UAS-based mapping outputs in urban environments. Overall, our study observed better accuracies and modeling reliability in the case of the Úrrétje area compared with Csapókert. This discrepancy can be attributed to the spatial distribution of the GCPs. In the Csapókert area, the GCP distribution followed the surface of four parallel roads, resulting in a relatively regular pattern that was ultimately deemed suboptimal. Consistent with prior studies by Hugenholtz et al. (2016) and Ahmed et al. (2022), our investigation confirmed that the horizontal accuracy rates exceeded the vertical accuracy rates.

In the context of the horizontal accuracy test results, it was evident that the accuracy values improved with higher processing levels, consistent with our anticipated outcomes. The chosen data acquisition methods played a pivotal role in the surveys, with NG1 yielding the most favorable results across all three tested factors (RMSE, error range, and median), whereas P3D demonstrated comparatively inferior outcomes (Table 2).

These findings notably differ from those reported in studies, where the tilted camera and the double-grid method were reported to achieve the highest accuracy. In our study, the double-grid method only demonstrated accuracy improvement for the tilted camera angle, in contrast to the results of Ahmed et al. (2022) and Strząbała et al. (2022), where the double-grid method enhanced the accuracy of nadir acquisition.

The disparities in the results could be attributed to variations in data collection methods. Specifically, our study employed a flight altitude that was 30 m higher, and the camera tilt angle was 10° more pronounced during oblique shooting than in the study by Ahmed et al. (2022). Similarly, Jaud et al. (2018) found that a tilted camera provided better horizontal accuracy in Agisoft PhotoScan (the older version of Agisoft Metashape software). The variation in outcomes between our study and theirs could also originate from differences in data acquisition methods. Their oblique survey employed a 40-m lower flight altitude and a 20° higher camera tilt angle compared to ours. A recent study by Mueller et al. (2023) introduced innovative UAS flight design scenarios, yet it is important to highlight that their conceptual framework was specifically tailored for application within urban environments.

The results of the vertical accuracy test revealed distinct impacts of flight direction and camera angle factors on the RMSE values across the two study areas and the three processing levels. Notably, in the Csapókert area, the tilted double grid (OG3) method produced superior results for ground points at the HPL, consistent with the findings reported in studies by Jaud et al. (2018) and Ahmed et al. (2022). In contrast, in the Úrrétje area, the NG3 method achieved the highest accuracy, mirroring the third-best vertical accuracy result obtained in the study by Ahmed et al. (2022).

For vertical accuracy on road surfaces in the Úrrétje area, our results corroborated a similar trend observed by Dai et al. (2023) study, indicating that a tilted camera does not necessarily lead to increased accuracy when a substantial number (> 5–8) of GCPs are employed. The optimal method for rooftop points was OG1 in the Csapókert area and NG3 in the Úrrétje area. In conclusion, in line with the findings of Nesbit and Hugenholtz (2019), our results emphasize that utilizing both a double-grid flight direction and tilted camera angle contributes to improved vertical accuracy (Tables 3 and 4) compared to employing a single grid and nadir configurations for these two reference point types. Previous studies have shown that in situations involving oblique camera angles, optimal accuracy levels are attained when the automatic flight plan is supplemented by manual flight operations for additional image acquisition (Kovanič et al. 2023).

Jakovljevic et al. (2019) observed that point clouds generated through UAS-SfM exhibited larger vertical errors, resulting in an overestimation of elevations compared to point clouds derived from LiDAR surveys. According to the authors, these errors limited the suitability of SfM-generated point clouds for applications such as accurately modeling flood hazard areas. However, in our study, the vertical errors of the ground surface points were, on average, 17 cm smaller than those in Kameyama and Sugiura (2020). Kameyama and Sugiura (2020) examined different data acquisition parameters, including flight altitude and image overlap to assess the height and volume of forest trees. They discovered that the methods of data collection had minimal impacts on error magnitudes, which, in their study, proved to be excessively large for the intended measurements. In terms of non-ground surface vertical accuracy, our models produced RMSEs within the sub-meter range, contrasting with the RMSEs exceeding a meter defined by Kameyama and Sugiura (2020).

It is essential to note that our study primarily focused on the quantitative analysis of the unprocessed results generated by the SfM process. In contrast, prior research has emphasized the potential for substantial enhancements in the vertical accuracy of UAS-derived DEMs through the incorporation of weighted averaging and additive median filtering algorithms (Ajibola et al. 2019). In our investigation of accuracy factors, we observed that the recording method held the greatest significance for two reference types (horizontal and vertical roofs), whereas the processing level emerged as the most influential factor for the vertical roof reference. Unlike Mora-Felix et al. (2020), we found that flight direction played a minor role in our study. The discrepancies in the results between the two studies may be attributed to variations in the specific factors investigated.

The software processing time for the images produced by various methods was aligned with the specifications provided by the software developer (Internet 2). According to these specifications, projects with larger image datasets and higher processing levels generally require longer processing times. However, it is noteworthy that the accuracy of the models produced from these processes does not consistently surpass that of models produced with lower processing levels. In contexts where the timely delivery of results is of significant importance, as in applications such as agriculture and monitoring, the optimization of SfM processing time becomes paramount. Based on our findings, we infer that employing MPL can offer a satisfactory level of accuracy for situations where expeditious results are of primary concern.

Conclusions

The primary objective of this study was to evaluate and compare the impact of various UAS flight orientations and camera angles on the accuracy of SfM models, while considering different processing levels. This investigation was conducted across two urban study areas, with distinct characteristics enabling a thorough examination of their combined effects.

In the present study, we generated 42 distinct models including both digital surface models (DSMs) and orthophoto mosaics, by varying image acquisition and processing parameters. Our evaluation of horizontal accuracy, specifically regarding ground elevation, through orthophoto mosaics, consistently showed that the nadir camera angle combined with the single-grid flight mode provided the highest accuracy across all processing levels. Notably, we found that image acquisition emerged as the primary factor influencing accuracy values, with minimal disparities among the outcomes of the four tested factors.

On the other hand, our vertical accuracy assessment on road surfaces did not identify a single optimal method for all the processing levels. We observed instances where the double-grid and oblique camera angles produced superior results. The processing level had a more profound impact on accuracy. Similarly, our examination of roof surface measurements revealed that different data acquisition methods across varied processing levels yielded optimal results, with image acquisition exerting the greatest impact on accuracy.

When analyzing the processing levels for both vertical references, we noted that, in several cases, the medium processing level (MPL) outperformed the high processing level (HPL) in terms of accuracy. This suggests that MPL could serve as a viable alternative, particularly when rapid results are required. Despite the potential benefits of oblique camera angles and double grids in certain scenarios, it is essential to consider the significant increase in processing time, associated with these methods, which may be a crucial factor in practical applications.

Our research findings offer a comprehensive understanding of the potential accuracy variations associated with different flight patterns and image acquisition modes. Collectively, these results contribute substantially to the broader applicability of UAS-based aerial mapping in urban settings.

According to our experiences, urban mapping is constrained by several factors. One of the most significant challenges is the regulatory framework that may impede lawful surveys. Additionally, UAV flight time restrictions can pose obstacles when mapping large areas. To further investigate this subject, our team intends to explore the feasibility of employing various external orientation techniques in urban settings and their effect on model precision.