1 Introduction

The beginnings of our working group, the Application Center for Machine Learning and Sensor Technology (AMLS), and our technical equipment already date back more than ten years ago. In 2010, our university had the opportunity to acquire its own manned aircraft, funded by the German government’s economic stimulus package II. We applied for a so-called gyrocopter, and the proposal was positively reviewed by the German Research Foundation (DFG). The gyrocopter is an ultralight rotorcraft that differs from a helicopter in that the wing is not actively propelled by an engine but only by the incoming airflow. This effect is known as autorotation and provides the necessary lift and flight capability. Furthermore, due to the simple design principle compared to helicopters, gyrocopters are significantly cheaper to purchase and operate. They, therefore, represent a very economical alternative to helicopters. In particular, the excellent low-speed flight characteristics and maneuverability suggests using the gyrocopter for observation and survey flights. The gyrocopter flies at an altitude ranging from 150 to 3000 m at speeds between 30 and 150 km/h. If operated without a copilot, it can fly for several hours with a payload of up to 100 kg. Due to these characteristics and its continuing widespread use in the ultralight community, its flight dynamics are the subject of ongoing research to extend its applicability to the regulated professional sector (DLR 2021; Sachs 2013). After a particular evaluation phase, we finally decided on the MTOsport gyrocopter from the manufacturer AutoGyro (www.auto-gyro.com), Germany (Fig. 1). One of the key advantages of the developed remote sensing system is that it is not restricted to a single MTOsport gyrocopter. Instead, it can be installed in any MTOsport without the need for modifications to the aircraft. Besides, more than 2,000 units of this model can be found worldwide, including around 500 in Germany alone, potentially suitable as aerial carrier platforms. This makes the FlugKit remote sensing platform very flexible, as only this needs to be transported to the planned destination, and a locally available MTOsport can be chartered. The gyrocopter is a full-fledged participant in air traffic with a trained pilot on board and equipped with a flight radio system and a transponder. In contrast, UAVs are increasingly subject to regulations limiting, for example, their use within the pilot’s visual line of sight (VLOS). Moreover, they are not allowed to fly over or nearby certain areas or only with special permission. The following sections describe the components we have developed in detail and present typical application scenarios.

Fig. 1
figure 1

Full-view of the MTOsport and the temporarily installed FlugKit equipped with the PanX.2 sensor system before take-off

Full size image

2 Sensor Carrier Platform

2.1 FlugKit

The authors developed the modular sensor carrier FlugKit to temporarily convert the MTOsport into a full-fledged remote sensing platform (Kneer et al. 2016). The FlugKit can be installed in the legroom of the copilot’s seat of any MTOsport within 30 minutes without irreversible modifications to the aircraft. It combines all necessary components to ensure an autonomous, GPS waypoint-guided operation of the optical instruments. Therefore, it contains a gyrocopter-independent power supply, inertial navigation systems (INS), and a computing unit that runs the self-developed sensor control software.

Fig. 2
figure 2

FlugKit sensor carrier platform for the AutoGyro MTOsport gyrocopter

Full size image

Figure 2 shows the current version of the carrier with the gyrocopter mounting bracket (center), the battery unit (left), the control unit (right), and an external camera payload mounted into a gimbal (rightmost). Besides the camera system, various environmental sensors are integrated to monitor the incident solar spectrum, temperature, air pressure, and humidity. External parts of the FlugKit, in the form of a flight navigation system and a status monitor, are installed in the front of the pilot’s cockpit for the survey duration (Fig. 3b). The camera systems can be mounted as external loads in wind-protected gimbal devices on both sides of the FlugKit. A vibration damper is used to decouple the external payload from the mainframe of the gyrocopter.

Fig. 3
figure 3

a Fast installation of the FlugKit carrier system on the MTOsport mainframe. b View of the cockpit with the two external components of the FlugKit: Flight navigation system (red box) and status monitor (green box). c MTOsport and the temporarily installed FlugKit equipped with the PanX.2 sensor system before take-off. d MTOsport and installed FlugKit equipped with the PanX.3 during flight

Full size image

The system was recently upgraded with a new custom irradiance sensor consisting of two mini-spectrometers (Hamamatsu C12880MA), installed on both ends of the carrier, and a 200\(^\circ\) cloud monitoring fisheye camera (Sony IMX219PQH5-C). Furthermore, a GNSS splitter is incorporated to distribute the GPS signal from one antenna (Trimble AV37) to two GNSS receivers (Trimble BX982, Emlid Reach M+ Rover) and an INS (Xsens MTi7 GNSS/INS). The accuracy of GNSS data is improved, either by real-time kinematic (RTK) or post-processed kinematic (PPK), depending on the application. Figure 3 shows an overview of the system installation and the location of the components on the ground and in the air.

2.2 Gimbal

To accomplish a permanent nadir viewing angle of the sensors, regardless of the current orientation of the aircraft during flight operation, an electromechanical stabilization device, a so-called gimbal, is used. The agile maneuverability of the gyrocopter puts high demands on such gimbal systems. Which are, mainly, an instant response to the fast-changing orientation of the aircraft, a high angle deflection of up to 90\(^\circ\), and a specific resistance to vibrations. Furthermore, the system should be capable of balancing multiple camera setups, feature a rigid and safe structure, and achieve a payload of up to 5 kg. Additionally, it should be highly resistant to electromagnetic interference (EMI) since the aircraft radio system, in particular, offers a potential source of such interference. However, as most commercial systems are built for use onboard drones, e.g., for cinematography, no suitable product was available for the specific application on a gyrocopter. Because of this, a dedicated gimbal device was developed that fits seamlessly into the overall concept and connects the optical instruments to the FlugKit. Figures 4 and 7 show the latest development stage of the gimbal system with built-in DSLR and PanX.3 sensor system, both with vibration dampers. The two-axis (pitch and roll) gimbal is built from CNC-machined aluminum and 3D-printed parts combined with powerful brushless motors and a gimbal controller. The system has been constantly improved and updated over the years, and there are currently three copies in operation.

3 Optical Sensor Systems

In this section, the individual optical systems designed by the AMLS are presented. To date, three systems have been developed and used in several studies.

3.1 High-Resolution RGB Imaging

A high-quality DSLR camera (Nikon D800e) was integrated into the gimbal system for high-resolution RGB color images (see Fig. 4). So far, three different lenses with a focal length of 25, 35, and 85 mm can be selected, depending on the parameters of the intended application.

Fig. 4
figure 4

High-resolution DSLR camera mounted in the self-designed gimbal system (left). The self-explanatory icons on the right visually summarize the most important features of the system

Full size image

Due to the high spatial resolution of the image data, digital elevation models (DEMs) with a ground resolution in the centimeter range can be derived using photogrammetric software. The point clouds generated from these image data, together with data from other systems (UAV-LiDAR and RGB imagery), were successfully evaluated in a comparative study to determine the accuracy and precision with which various individual tree parameters can be derived (Ganz et al. 2019). Results of another case study are shown in Fig. 5. Here, a part of an orthomosaic of an agro-sylvo-pastoral system in Andalusia (Spain) is displayed with two zoomed views to illustrate the resolution capacity of the system. A ground sampling distance (GSD) of approximately 0.027 m was achieved to perform, for example, a single tree detection (Bareth et al. 2017).

Fig. 5
figure 5

a Part of an RGB orthomosaic of the Dehesa San Francisco (Andalusia, Spain), b two zoomed views of the RGB orthomosaic

Full size image

3.2 Multispectral VNIR Imaging–PanX Series

The most widely used camera system of the AMLS is the so-called PanX system and, according to Aasen et al. (2018), can be defined as a multispectral multi-camera 2D imaging system. The spectral instrument is sensitive to the visible and near-infrared spectral range (VNIR, 400–1000 nm), has undergone several stages of improvement, and is currently in version 3. The advantage of this multispectral camera system is that the spectral resolution is reduced in favor of high spatial resolution and image quality. For each wavelength, four channels for the PanX, a complete camera sensor including lens and corresponding broad- or narrow-band interference filters are used. Thus, these systems are particularly suitable for applications where specific, well-established vegetation indices are required to effectively monitor survey areas in the tens of square kilometers. In addition, the use of high-quality optical components provides high-resolution spectral image data.

Fig. 6
figure 6

Version 2 of the spectrally adaptable multispectral multi-camera system, PanX. Ready assembled prototype with gimbal for operational use (left). Icons on the right summarize the most important features of the system

Full size image

The PanX system was the first fully integrated camera system developed by AMLS. The version PanX.2 (see Fig. 6) was flight-ready at the beginning of 2016. The interference filters are mounted in front of the lenses to guarantee easy and fast exchangeability. In 2020, the system was redesigned and updated for higher resolution industrial board-level cameras (Allied Vision, Manta G-1236). Furthermore, a sensor-temperature stabilization was integrated. The position of the interference filters was changed between lens and sensor to achieve a more homogeneous sensor illumination and wavelength stability of the passband.

Fig. 7
figure 7

PanX.3, the latest version of the multispectral multi-camera system of the PanX series. Finished prototype with gimbal, mounted on the FlugKit carrier system. Icons on the right summarize the most important features of the system

Full size image

Figure 7 shows the latest development stage of the PanX system. Table 1 provides a comprehensive overview and comparison of the two camera systems.

Table 1 Comparison of both PanX versions
Full size table

Figure 8 shows the same scene captured on different days with PanX.2 and PanX.3. Both flights were performed at comparable altitudes, and the improvement in GSD is clearly visible.

Fig. 8
figure 8

Comparison of a PanX.2 (678 m AGL, GSD: 29 cm/pix) and b PanX.3 (625 m AGL, GSD: 16 cm/pix) orthomosaics

Full size image

3.3 Thermal Imaging - PanTIR

The PanTIR camera system is a dual-camera system consisting of a microbolometer camera for thermal infrared (TIR) imaging combined with a 4-megapixel RGB camera (see Fig. 9).

Fig. 9
figure 9

PanTIR: Thermal Imager combined with an RGB camera. Finished gimbal-mounted PanTIR camera system prototype (left). Icons on the right summarize the most important features of the system

Full size image

The system was developed in 2014 to monitor the surface temperatures of water bodies and vegetation canopies (Weber et al. 2015). The former application is used for concluding temperature distributions within the water body. The latter can be utilized as an indicator of plant stress or for early detection of disease infestation (Hornero et al. 2021). At that time, significant development efforts were required to support synchronization and triggering, especially in the light of creating a system able to be taken to international campaigns, where export restrictions might apply. Table 2 lists the parameters of both camera types. The thermal image data can be fused with the higher resolution RGB imagery and then further processed to orthomosaics in photogrammetric software for low contrast thermal scenery.

Table 2 Parameter overview of both cameras of the PanTIR imager
Full size table

4 Applications–Case Studies

4.1 Airborne Thermal Survey of the Hahnhöfer Nebenelbe

In 2015, an aerial survey was conducted in cooperation with the Federal Institute of Hydrology (BfG) to obtain information on water surface temperature over the Hahnöfer Nebenelbe, a part of the Elbe estuary near Hamburg. For this purpose, the gyrocopter was carried to Stade airfield by truck and equipped with the FlugKit on site.

Fig. 10
figure 10

Survey Area of the Hahnhöfer Nebenelbe near Hamburg a thermal infrared orthomosaic b RGB orthomosaic

Full size image

For two consecutive days, the approximate 24 km\(^2\) wide area was overflown three times per day with the PanTIR camera system. The objective was to capture the temperature changes at the water surface over the tidal cycle during a day. Figure 10 displays the two orthomosaics (TIR and RGB) for the first flight (10:42 am) on the first survey day (30.06.2015). A detailed description of the study design and the further data analysis by the BfG can be found in Fricke et al. (2016) and Fricke et al. (2021).

4.2 Case Study Agro-Sylvo-Pastoral System

As part of a joint research campaign to monitor cork and holm oak decline in Mediterranean silvopastoral systems, the FlugKit remote sensing system was successfully applied to monitor the 700 hectares wide Dehesa San Francisco (www.fundacionmontemediterraneo.com) north of Seville in Andalusia, Spain. Spectral data should typically be recorded in a two-hour time window around solar noon. Therefore, it was not possible to spectrally cover the entire Dehesa with a drone, which would have required several days. Professional aircraft with appropriate spectral sensors, if available, on the other hand, are very cost-intensive and must be ordered in advance on a long-term basis. So for this particular application scenario, again, the advantages of the gyrocopter outweighed. The FlugKit and three optical instruments were brought to Spain and temporarily installed in a chartered MTOsport, that was located in the region of Andalusia (see Fig. 11). Within three days, the entire Dehesa was monitored several times a day with three different camera systems (RGB, PanTIR, PanX.2). Figure 12 shows the main section of the Dehesa San Francisco in the three resulting spectral modalities captured with the applied sensors.

Fig. 11
figure 11

FlugKit installed in a chartered MTOsport in the region of Andalusia (Spain)

Full size image
Fig. 12
figure 12

Main section of the Dehesa San Francisco in three spectral modalities: a RGB orthomosaic b PanX.2 NDVI orthomosaic c PanTIR thermal infrared orthomosaic

Full size image

4.3 Path Flights for Monitoring Waterways

Contracted in a project of the BfG, a survey of the section of the Rhine from Lahnstein via Koblenz to Bad Honnef as well as parts of the Moselle and the Lahn took place. On a total of three flight days, spread over twelve months, both the PanX.2 and PanTIR camera systems were deployed on the river sections, which were approximately 72 km long in total. This path was flown individually with each of the two camera systems per flight date. Figure 13 shows both orthomosaics from the second flight date in August 2017. The thermal and VNIR orthomosaics were used by the BfG to create a workflow for thermal sharpening of Landsat-8 TIRS surface temperatures for inland water bodies (Fricke et al. 2018).

In such projects, the gyrocopter is able to show its full potential, as effectively flying along winding river courses is a demanding task. Larger fixed-wing aircraft, in particular, require several passes (and turns) to master such conditions, resulting in inefficient coverage of the survey area and thus higher costs. In contrast, the gyrocopter is particularly suitable for these tasks due to its high maneuverability. The main difficulty here lies in the pre-planning of the flight. Since pure water surfaces are not suitable for processing in photogrammetry software due to non-existent or constantly changing structural features, a certain amount of land area is also always necessary in the images. Therefore, the flight parameters must be set in advance so that a sufficient part of, e.g., the riverbank is still contained in the images.

Fig. 13
figure 13

Path flight of the Rhine, Moselle and Lahn. a PanX.2 color infrared (CIR) orthomosaic with zoomed views b PanTIR thermal infrared orthomosaic with zoomed views

Full size image
Fig. 14
figure 14

Path flight of the river Rhine with PanX.3 for the mDRONES4rivers project: NDVI orthomosaic with zoomed views

Full size image

Figure 14 again shows a path flight over nearly the same Rhine section as in Fig. 13, without the section of the river Lahn. This survey flight was carried out with the PanX.3 as part of the joint research project mDRONES4rivers, funded by the Federal Ministry of Transport and Digital Infrastructure (BMVI). In addition to effectively flying along waterways, other possible applications include monitoring highways, railroads, pipelines, or power lines.

5 Conclusion and Outlook

Over the past ten years, the AMLS has developed a versatile optical remote sensing platform for use in combination with a widespread ultralight aircraft, the gyrocopter AutoGyro MTOsport. Its aircraft-independent centerpiece consists of the FlugKit sensor carrier platform and a versatile gimbal system. In addition, three spectral sensor systems have been developed or implemented, ranging from high-resolution RGB imaging to multispectral camera systems in the VIS and NIR and in the LWIR. Moreover, other sensor modalities such as LiDAR and RADAR can also be integrated depending on the required application. Several projects in Germany and abroad were carried out in joint research projects or by contract.

In the near future, it is planned to extend the spectral range of the developed optical sensor systems to the short-wave infrared (SWIR). A UAV-based VNIR-SWIR multispectral camera system has already been developed by (Jenal et al. 2019). It has been successfully tested in a permanent grassland experimental field (Jenal et al. 2020) as well as in a winter wheat field trial (Jenal et al. 2021) and is currently being implemented in the FlugKit ecosystem.